Event-triggered dual-link robotic arm control method and system under cloud-edge-end collaboration

Abstract

The present invention discloses a control method and system for a double-link robotic arm triggered by events under cloud-edge-end collaboration; the terminal collects real-time data of the double-link robotic arm and sends it to the edge end; the edge end inputs the data into a model-free controller or a trigger holder to obtain a control instruction according to a preset event trigger rule; the terminal controls the movement of the double-link robotic arm according to the control instruction, collects real-time data of the double-link robotic arm and sends it to the edge end; the cloud end selects to optimize and update the parameters of the multivariable model-free controller according to the preset event trigger rules based on the real-time data fed back by the edge end until the control task is completed; the control method and system provided by the present invention are not only suitable for double-link robotic arms, reduce their modeling difficulty, improve control accuracy, enhance computing efficiency, and save system resources, but also have the application prospect of being extended to other multi-input and multi-output systems.

Description

Technical Field

[0001] The present invention belongs to the field of robotic arm control, and in particular, to a method and system for controlling a double-link robotic arm triggered by events under cloud-edge-end collaboration. Background Art

[0002] With the advancement of science and technology and the accelerated development of industrial automation, robotic arms have become an indispensable part of the automation field, undertaking various important tasks such as assembly, handling, and welding. The two-link robotic arm has been widely used in industrial production, medical rehabilitation, and service robots. However, the strong time-varying, strong nonlinearity, and strong coupling brought by its double-joint structure have brought great challenges to the research of its decoupling control method. The control method of the two-link robotic arm is divided into model-based control and data-driven control. The model-based control method requires the establishment of an accurate dynamic model. However, the complex internal factors and external environment make modeling difficult, which affects its in-depth research and application. Data-driven control directly uses rich input and output data to directly design the controller, which has received widespread attention from scholars.

[0003] In 2013, Hou Zhongsheng and Zhu Yuanming first proposed a direct single-variable model-free adaptive control method for single-input single-output systems in their paper "Controller-dynamic-linearization-based model free adaptive control for discrete-time nonlinear systems." IEEE Transactions on Industrial Informatics, 2013, 9(4): 2301-2309. After 7 years, Yu Xian from Beijing Jiaotong University developed a direct multivariable control method for multi-input multi-output systems in his doctoral dissertation "Data-driven iterative learning control and application based on controller dynamic linearization [D]", Beijing Jiaotong University, 2020. It has now become a typical data-driven control method, attracting scholars to explore its potential in combination with neural networks to improve the control performance of multi-input multi-output controlled systems. With the development of cloud computing, edge computing and direct multivariable model-free adaptive control methods, new ideas have been provided for the control of dual-link robotic arms. However, in actual applications, cloud-edge systems are limited by system bandwidth and communication resources. Traditional periodic sampling and data transmission schemes will consume a lot of network bandwidth and energy even if the system status changes slightly, resulting in waste of communication resources and increased system operating costs.

[0004] In order to solve the above problems, an event-triggered control scheme came into being. This scheme performs data transmission and control operations only when there is a significant change in the system state, effectively reducing unnecessary communication and calculation, thereby saving system resources; on the other hand, in order to further improve the control performance of the double-link robotic arm and combine the advantages of the cloud-edge, the present invention deeply integrates direct multivariable model-free adaptive control and reinforcement learning, and proposes a control method based on event triggering; in view of the fact that such methods have not been involved in existing research, the present invention proposes a double-link robotic arm control method and system triggered by events under the collaboration of cloud-edge. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a double-link robotic arm control method triggered by an event under cloud-edge-end collaboration, wherein the cloud-edge-end includes a cloud end, an edge end, and a terminal, and includes the following steps:

[0006] Step (1): the terminal collects real-time data of the double-link robot arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link, and sends the real-time data to the edge end;

[0007] Step (2): the edge end preprocesses the real-time data; designs a dynamic event trigger mechanism; if the trigger rule is met, the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and a control instruction of the double-link robotic arm is calculated, the control instruction includes a control instruction of the first link and a control instruction of the second link; if the trigger rule is not met, the control instruction at the last triggering moment is maintained; the control instruction is sent to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instruction of the first link and the control instruction of the second link; at the same time, the edge end sends the preprocessed data to the cloud;

[0008] Step (3): The cloud optimizes and updates the parameters of the multivariable decoupled model-free controller based on the data fed back by the edge and in combination with the dynamic event trigger mechanism, if the trigger rules are met, and transmits the optimized and updated multivariable decoupled model-free controller parameters to the multivariable decoupled model-free controller at the edge; if the trigger rules are not met, no operation is performed;

[0009] Repeat the above steps (1) to (3) until the control task is completed.

[0010] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, the dynamic event triggering mechanism designed in step (2) is used to determine the dynamic event triggering mechanism for the next triggering moment:

[0011]

[0012] in, For the s A trigger moment, s is a positive integer, for Sampling time The trigger error of the connecting rod, For the s The trigger moment The real-time operating joint angle of the connecting rod, for k Sampling time The real-time operating joint angle of the connecting rod, , double-link robotic arm ; is an internal dynamic variable, and the update rule is:

[0013]

[0014] in, and All are event trigger parameters. , , is the preset threshold parameter;

[0015] The triggering rule described in step (2) introduces the indicator factor Indicates whether the triggering rule is met:

[0016] .

[0017] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, if the triggering rule is met in step (2), the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and the control instructions of the dual-link robotic arm are calculated, wherein the control instructions include the control instructions of the first link and the control instructions of the second link, and the steps are as follows:

[0018] Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is:

[0019]

[0020] in, k is the sampling time, For the A trigger moment, s is a positive integer, a two-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; For the The control command of the first link of the double-link robot arm at the triggering moment, For the Control instructions for the second link of the double-link robotic arm at the triggering moment; is the indicator factor; for k The sampling time s +2 time-varying block matrices, For the s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s +2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; L is the controller pseudo order, L is a positive integer; , , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator Real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ;

[0021] Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network;

[0022] The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​within the All inputs of the online Actor-Critic reinforcement learning subnetwork; the The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

[0023] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, in step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge end in combination with the dynamic event trigger mechanism if the triggering rules are met. The optimization and update process includes the following steps:

[0024] Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics:

[0025] ;

[0026] in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , is the Jacobian matrix of the two-link manipulator, and the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them;

[0027] Jacobian matrix of the two-link manipulator The iterative learning law is:

[0028]

[0029] in, For the A trigger moment, For the The Jacobian matrix of the two-link manipulator at the triggering moment; is the indicator factor, is the step size factor, is the penalty factor; , ; represents the two-norm;

[0030] Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is:

[0031]

[0032] Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k +1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time;

[0033] Step (3.3): In the optimization and updating process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the Jacobian matrix of the two-link manipulator is:

[0034]

[0035] in, for against The derivative of , ;

[0036] Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is:

[0037] ;

[0038] Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize the update The weights of the online Actor-Critic reinforcement learning subnetwork, which contains the Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork;

[0039] Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks :

[0040] If the controller is pseudo-order ,but

[0041]

[0042] If the controller is pseudo-order ,but

[0043]

[0044] in, For the The Actor network weight at the triggering moment, ; is the indicator factor, For the said The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node;

[0045] Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks :

[0046]

[0047] in, For the The Critic network weight at the triggering moment, , is the indicator factor, For the said The learning rate of the Critic network;

[0048] Step (3.6): Optimize updates The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time:

[0049] If the controller is pseudo-order ,but

[0050]

[0051] Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix;

[0052] If the controller is pseudo-order ,but

[0053]

[0054] in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

[0055] According to the event-triggered dual-link robotic arm control method under cloud-edge collaboration provided by the present invention, the Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer, that is, a three-layer network structure consisting of an input layer, a single hidden layer, and an output layer.

[0056] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, the edge end in step (2) preprocesses the real-time data, including filtering and denoising the real-time data.

[0057] According to the event-triggered double-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, in step (1), the terminal collects real-time data of the double-link robotic arm through a sensor signal acquisition device.

[0058] The present invention also provides a double-link robotic arm control system triggered by an event under cloud-edge-end collaboration, comprising:

[0059] The terminal comprises a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module;

[0060] A data acquisition module, used to collect real-time data of the double-link robotic arm, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link;

[0061] A first data output module, used to send the real-time data to an edge end;

[0062] A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod;

[0063] The edge end includes a second data input module, a data preprocessing module, a first dynamic event triggering module, a multivariable decoupling model-free controller computing module, a control instruction holding module, and a second data output module;

[0064] A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal;

[0065] A data preprocessing module, used to preprocess the real-time data and then send it to the dynamic event triggering module and the cloud;

[0066] A first dynamic event triggering module is used to design a dynamic event triggering mechanism. If the triggering rules are met, the pre-processed real-time data is input into a multi-variable decoupling model-free controller calculation module, otherwise the control instruction holding module is triggered;

[0067] A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link;

[0068] A control instruction holding module, used to hold the control instruction at the last triggering moment;

[0069] A second data output module, used for sending the control instruction to the terminal;

[0070] The cloud comprises a third data input module, a data storage module, a second dynamic event triggering module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module;

[0071] The third data input module is used to receive the pre-processed real-time data uploaded by the edge end;

[0072] A data storage module, used to store the pre-processed real-time data uploaded by the edge end;

[0073] The second dynamic event triggering module is used to determine whether the triggering rules are met. If so, the multi-variable decoupling model-free controller optimization and updating module is triggered, otherwise no operation is performed;

[0074] A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller;

[0075] The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

[0076] Furthermore, the present invention adopts the following technical solutions:

[0077] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the event-triggered double-link robotic arm control method under cloud-edge-end collaboration as described above.

[0078] Furthermore, the present invention adopts the following technical solutions:

[0079] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the event-triggered dual-link robotic arm control method under cloud-edge collaboration as described above is implemented.

[0080] The beneficial technical effects of the present invention are:

[0081] For multi-input and multi-output controlled systems, existing research has not yet involved combining direct multivariable model-free adaptive control with reinforcement learning networks, and solving the problem of limited system bandwidth and communication resources faced by the cloud-edge through an event triggering mechanism. The event-triggered dual-link robotic arm control method and system under cloud-edge collaboration provided by the present invention at least include the following beneficial technical effects:

[0082] (1) Reduce the difficulty of modeling: The present invention adopts a data-driven control method, which does not require the establishment of an accurate dynamic model, thus reducing the difficulty of modeling. This method has more obvious advantages when dealing with multivariable systems with strong time-varying, strong nonlinearity, and strong coupling, and is easier to apply in practice;

[0083] (2) Improve control accuracy: By deeply integrating multivariable model-free adaptive control and reinforcement learning, this method can achieve stronger learning ability, thereby improving the control accuracy of the dual-link robotic arm and meeting higher precision application requirements;

[0084] (3) Improve computing efficiency and save system resources: Utilize the powerful computing power of the cloud computing platform for model training and data analysis, and use edge computing devices to achieve real-time data processing and rapid response to control instructions, effectively improving computing efficiency. On the other hand, by introducing an event trigger mechanism, data transmission and control operations are performed only when there is a significant change in the system status, effectively reducing unnecessary communication and computing, thereby saving system resources.

[0085] (4) Broadening the scope of application: The control method and system proposed in the present invention are not only applicable to dual-link robotic arms, but can also be extended to other multi-input and multi-output control systems, and have broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a structural schematic diagram of a double-link robotic arm;

[0087] Figure 2 A schematic flow chart of a method for controlling a dual-link robotic arm triggered by an event under cloud-edge-end collaboration provided by the present invention;

[0088] Figure 3 A schematic diagram of the connection of the double-link robotic arm control system module triggered by events under cloud-edge-end collaboration provided by the present invention. DETAILED DESCRIPTION

[0089] The present invention discloses a control method and system for a double-link robotic arm triggered by an event under cloud-edge collaboration; the terminal collects the real-time data of the double-link robotic arm and sends it to the edge end; the edge end inputs the data into a model-free controller or a trigger holder to obtain a control instruction according to a preset event trigger rule; the terminal controls the movement of the double-link robotic arm according to the control instruction, collects the real-time data of the double-link robotic arm and sends it to the edge end; the cloud end selects to optimize and update the parameters of the multivariable model-free controller according to the preset event trigger rule based on the real-time data fed back by the edge end until the control task is completed. The control method and system provided by the present invention are not only applicable to double-link robotic arms, reduce their modeling difficulty, improve control accuracy, enhance computing efficiency, and save system resources, but also have the application prospect of being extended to other multi-input and multi-output systems.

[0090] The following is a further clear and complete description of the model-free control method and system of the double-link robotic arm under cloud-edge-end collaboration provided by the present invention in conjunction with the accompanying drawings:

[0091] Figure 1 The structural diagram of the double-link robotic arm is given. Figure 2 A schematic flow chart of a double-link robotic arm control method triggered by an event under cloud-edge-end collaboration provided by the present invention is given; the present invention provides a double-link robotic arm control method triggered by an event under cloud-edge-end collaboration, wherein the cloud-edge-end includes a cloud, an edge, and a terminal, and includes the following steps:

[0092] Step (1): the terminal collects real-time data of the double-link robot arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link, and sends the real-time data to the edge end;

[0093] Step (2): the edge end preprocesses the real-time data; designs a dynamic event trigger mechanism; if the trigger rule is met, the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and a control instruction of the double-link robotic arm is calculated, the control instruction includes a control instruction of the first link and a control instruction of the second link; if the trigger rule is not met, the control instruction at the last triggering moment is maintained; the control instruction is sent to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instruction of the first link and the control instruction of the second link; at the same time, the edge end sends the preprocessed data to the cloud;

[0094] Step (3): The cloud optimizes and updates the parameters of the multivariable decoupled model-free controller based on the data fed back by the edge and in combination with the dynamic event trigger mechanism, if the trigger rules are met, and transmits the optimized and updated multivariable decoupled model-free controller parameters to the multivariable decoupled model-free controller at the edge; if the trigger rules are not met, no operation is performed;

[0095] Repeat the above steps (1) to (3) until the control task is completed.

[0096] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, the dynamic event triggering mechanism designed in step (2) is used to determine the dynamic event triggering mechanism for the next triggering moment:

[0097]

[0098] in, For the s A trigger moment, s is a positive integer, for Sampling time The trigger error of the connecting rod, For the sThe trigger moment The real-time operating joint angle of the connecting rod, for k Sampling time The real-time operating joint angle of the connecting rod, , double-link robotic arm ; is an internal dynamic variable, and the update rule is:

[0099]

[0100] in, and All are event trigger parameters. , , is the preset threshold parameter;

[0101] The triggering rule described in step (2) introduces the indicator factor Indicates whether the triggering rule is met:

[0102] .

[0103] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, if the triggering rule is met in step (2), the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and the control instructions of the dual-link robotic arm are calculated, wherein the control instructions include the control instructions of the first link and the control instructions of the second link, and the steps are as follows:

[0104] Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is:

[0105]

[0106] in, k is the sampling time, For the A trigger moment, s is a positive integer, a two-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; For the The control command of the first link of the double-link robot arm at the triggering moment, For the Control instructions for the second link of the double-link robotic arm at the triggering moment; is the indicator factor; for k The sampling time s +2 time-varying block matrices, For the said s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s +2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; L is the controller pseudo order, L is a positive integer; , , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator The real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ;

[0107] Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network;

[0108] The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​within the All inputs of the online Actor-Critic reinforcement learning subnetwork; the The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

[0109] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, in step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge end in combination with the dynamic event trigger mechanism if the triggering rules are met. The optimization and update process includes the following steps:

[0110] Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics:

[0111]

[0112] in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , is the Jacobian matrix of the two-link manipulator, and the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them;

[0113] The Jacobian matrix of the two-link manipulator The iterative learning law is:

[0114]

[0115] in, For the A trigger moment, For the The Jacobian matrix of the two-link manipulator at the triggering moment; is the indicator factor, is the step size factor, is the penalty factor; , ; represents the two-norm;

[0116] Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is:

[0117]

[0118] Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k +1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time;

[0119] Step (3.3): In the optimization and updating process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the Jacobian matrix of the two-link manipulator is:

[0120]

[0121] in, for against The derivative of , ;

[0122] Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is:

[0123] ;

[0124] Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize the update The weights of the online Actor-Critic reinforcement learning subnetwork, which contains the Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork;

[0125] Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks :

[0126] If the controller is pseudo-order ,but

[0127]

[0128] If the controller is pseudo-order ,but

[0129]

[0130] in, For the The Actor network weight at the triggering moment, ; is the indicator factor, For the The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node;

[0131] Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks :

[0132]

[0133] in, For the The Critic network weight at the triggering moment, , is the indicator factor, For the The learning rate of the Critic network;

[0134] Step (3.6): Optimize updates The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time:

[0135] If the controller is pseudo-order ,but

[0136]

[0137] Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix;

[0138] If the controller is pseudo-order ,but

[0139]

[0140] in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

[0141] According to the event-triggered dual-link robotic arm control method under cloud-edge collaboration provided by the present invention, the Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer, that is, a three-layer network structure consisting of an input layer, a single hidden layer, and an output layer.

[0142] According to the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, the edge end in step (2) preprocesses the real-time data, including filtering and denoising the real-time data.

[0143] According to the event-triggered double-link robotic arm control method under cloud-edge-end collaboration provided by the present invention, in step (1), the terminal collects real-time data of the double-link robotic arm through a sensor signal acquisition device.

[0144] Figure 3 A schematic diagram of the connection module of a double-link robotic arm control system triggered by an event under cloud-edge-end collaboration provided by the present invention is given. The present invention also provides a double-link robotic arm control system triggered by an event under cloud-edge-end collaboration, including:

[0145] The terminal includes a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module;

[0146] A data acquisition module, used to collect real-time data of the double-link mechanical arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link mechanical arm, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link;

[0147] A first data output module, used to send the real-time data to an edge end;

[0148] A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod;

[0149] The edge end includes a second data input module, a data preprocessing module, a first dynamic event triggering module, a multivariable decoupling model-free controller computing module, a control instruction holding module, and a second data output module;

[0150] A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal;

[0151] A data preprocessing module, used to preprocess the real-time data and then send it to the dynamic event triggering module and the cloud;

[0152] The first dynamic event triggering module is used to design a dynamic event triggering mechanism. If the triggering rules are met, the pre-processed real-time data is input into the multi-variable decoupling model-free controller calculation module, otherwise the control instruction holding module is triggered; Figure 3 As shown, in this embodiment, =0 as an example, when the data output by the edge data preprocessing module is judged by the dynamic event triggering mechanism designed by the first dynamic event triggering module not to meet the event triggering condition, the instruction holding module is triggered;

[0153] A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link;

[0154] A control instruction holding module, used to hold the control instruction at the last triggering moment;

[0155] A second data output module, used for sending the control instruction to the terminal;

[0156] The cloud comprises a third data input module, a data storage module, a second dynamic event triggering module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module;

[0157] The third data input module is used to receive the pre-processed real-time data uploaded by the edge end;

[0158] A data storage module, used to store the pre-processed real-time data uploaded by the edge end;

[0159] The second dynamic event trigger module is used to determine whether the triggering rules are met. If so, the multi-variable decoupling model-free controller optimization update module is triggered, otherwise no operation is performed; Figure 3 As shown, in this embodiment, =0, for example, when the data output by the cloud data storage module is judged by the dynamic event trigger mechanism to not meet the event trigger condition, no operation is performed;

[0160] A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller;

[0161] The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

[0162] Furthermore, the present invention adopts the following technical solutions:

[0163] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the event-triggered double-link robotic arm control method under cloud-edge-end collaboration as described above.

[0164] Furthermore, the present invention adopts the following technical solutions:

[0165] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the event-triggered dual-link robotic arm control method under cloud-edge collaboration as described above is implemented.

[0166] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that the facilities of the present invention can be implemented by means of software plus the necessary general hardware platform. The embodiments of the present invention can be implemented using an existing processor, or by a dedicated processor used for this purpose or other purposes for an appropriate system, or by a hard-wired system. The embodiments of the present invention also include non-transitory computer-readable storage media, which include machine-readable media for carrying or having machine-executable instructions or data structures stored thereon; such machine-readable media can be any available medium that can be accessed by a general or special-purpose computer or other machine with a processor. For example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures, and can be accessed by a general or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hard-wired, or wireless, or a combination of hard-wired and wireless), the connection is also considered a machine-readable medium.

[0167] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for controlling a double-link robotic arm triggered by an event under cloud-edge-end collaboration, wherein the cloud-edge-end includes a cloud end, an edge end, and a terminal, and is characterized in that: The following steps are involved: Step (1): The terminal collects real-time data of the double-link robot arm, wherein the real-time data includes the real-time expected joint angle of the first link, the real-time operating joint angle of the first link, the real-time expected joint angle of the second link, and the real-time operating joint angle of the second link, and sends the real-time data to the edge end; Step (2): the edge end preprocesses the real-time data; designs a dynamic event trigger mechanism; if the trigger rule is met, the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and a control instruction of the double-link robotic arm is calculated, the control instruction includes a control instruction of the first link and a control instruction of the second link; if the trigger rule is not met, the control instruction at the last triggering moment is maintained; the control instruction is sent to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instruction of the first link and the control instruction of the second link; at the same time, the edge end sends the preprocessed data to the cloud; Step (3): The cloud optimizes and updates the parameters of the multivariable decoupled model-free controller based on the data fed back by the edge and in combination with the dynamic event trigger mechanism, if the trigger rules are met, and transmits the optimized and updated multivariable decoupled model-free controller parameters to the multivariable decoupled model-free controller at the edge; if the trigger rules are not met, no operation is performed; Repeat the above steps (1) to (3) until the control task is completed; The dynamic event triggering mechanism designed in step (2) is used to determine the dynamic event triggering mechanism for the next triggering time as follows: ； in, For the s A trigger moment, s is a positive integer, for Sampling time The trigger error of the connecting rod, For the s The trigger moment The real-time operating joint angle of the connecting rod, for k Sampling time The real-time operating joint angle of the connecting rod, , double-link robotic arm ; is an internal dynamic variable, and the update rule is: ； in, and All are event trigger parameters. , , is the preset threshold parameter; The triggering rule described in step (2) introduces the indicator factor Indicates whether the triggering rule is met: 。 2. The event-triggered dual-link robotic arm control method under cloud-edge-end collaboration according to claim 1 is characterized in that: If the triggering rule is met in step (2), the preprocessed data is input into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and a control instruction of the two-link manipulator is calculated, wherein the control instruction includes a control instruction of the first link and a control instruction of the second link, and the following steps are included: Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is: ； in, k is the sampling time, For the A trigger moment, s is a positive integer, a two-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; For the The control command of the first link of the double-link robot arm at the triggering moment, For the Control instructions for the second link of the double-link robotic arm at the triggering moment; is the indicator factor; for k The sampling time s +2 time-varying block matrices, For the said s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s +2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; L is the controller pseudo order, L is a positive integer; , , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator The real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ; Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network; The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​within the All inputs of the online Actor-Critic reinforcement learning subnetwork; the The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

3. The event-triggered dual-link robotic arm control method under cloud-edge-end collaboration according to claim 2 is characterized in that: In step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge and in combination with the dynamic event trigger mechanism, if the triggering rules are met. The optimization and update process includes the following steps: Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics: ； in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , is the Jacobian matrix of the two-link manipulator, and the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them; Jacobian matrix of the two-link manipulator The iterative learning law is: ； in, For the A trigger moment, For the The Jacobian matrix of the two-link manipulator at the triggering moment; is the indicator factor, is the step size factor, is the penalty factor; , ; represents the two-norm; Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is: ； Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k +1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time; Step (3.3): In the optimization and updating process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the Jacobian matrix of the two-link manipulator is: ； in, for against The derivative of , ; Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is: ； Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize the update The weights of the online Actor-Critic reinforcement learning subnetwork, which contains the Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork; Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks : If the controller is pseudo-order ,but ； If the controller is pseudo-order ,but ； in, For the The Actor network weight at the triggering moment, ; is the indicator factor, For the said The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node; Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks : ； in, For the The Critic network weight at the triggering moment, , is the indicator factor, For the said The learning rate of the Critic network; Step (3.6): Optimize updates The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time: If the controller is pseudo-order ,but ； Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix; If the controller is pseudo-order ,but ； in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

4. The event-triggered dual-link robotic arm control method under cloud-edge-end collaboration according to claim 1 is characterized in that: The Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer.

5. The event-triggered dual-link robotic arm control method under cloud-edge-end collaboration according to claim 1 is characterized in that: In step (2), the edge end preprocesses the real-time data, including filtering and denoising the real-time data.

6. The event-triggered dual-link robotic arm control method under cloud-edge-end collaboration according to claim 1 is characterized in that: In the step (1), the terminal collects real-time data of the double-link robotic arm through a sensor signal acquisition device.

7. A double-link robotic arm control system triggered by an event under cloud-edge-end collaboration, executing a double-link robotic arm control method as described in any one of claims 1-6, characterized in that: include: The terminal includes a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module; A data acquisition module, used to collect real-time data of the double-link robotic arm, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link; A first data output module, used to send the real-time data to an edge end; A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod; The edge end includes a second data input module, a data preprocessing module, a first dynamic event triggering module, a multivariable decoupling model-free controller computing module, a control instruction holding module, and a second data output module; A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal; A data preprocessing module, used to preprocess the real-time data and then send it to the dynamic event triggering module and the cloud; A first dynamic event triggering module is used to design a dynamic event triggering mechanism. If the triggering rules are met, the pre-processed real-time data is input into a multi-variable decoupling model-free controller calculation module, otherwise the control instruction holding module is triggered; A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link; A control instruction holding module, used to hold the control instruction at the last triggering moment; A second data output module, used for sending the control instruction to the terminal; The cloud comprises a third data input module, a data storage module, a second dynamic event triggering module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module; The third data input module is used to receive the pre-processed real-time data uploaded by the edge end; A data storage module, used to store the pre-processed real-time data uploaded by the edge end; The second dynamic event triggering module is used to determine whether the triggering rules are met. If so, the multi-variable decoupling model-free controller optimization and updating module is triggered, otherwise no operation is performed; A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller; The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the event-triggered dual-link robotic arm control method under cloud-edge-end collaboration as described in any one of claims 1 to 6.

Images