A time-varying optimal formation fault-tolerant control method for multi-euler-lagrange systems

Abstract

The application provides a time-varying optimal formation fault-tolerant control method of a multi-Euler-Lagrange system, and relates to the field of formation control.The application constructs a communication topology graph of N follower mobile robots, and then constructs a target mathematical model of an Euler-Lagrange system of each follower mobile robot, a sensor fault model and an actuator fault model of each follower mobile robot, and a target coordinate transformation equation, and then derives an optimal virtual control law, an approximate value of an optimal output signal and an actuator optimal input signal approximate value.The application can enable the follower mobile robots to adjust the relative distance with the leader according to the environmental changes as needed.Through adjusting the distance, the dynamic change of the topology structure is realized, so that obstacle avoidance is realized and the reliability in the collaborative work process is improved.The application fuses the optimal backstepping control and the fault-tolerant mechanism, so that the time-varying formation with high precision can be maintained when the sensor and the actuator simultaneously fail.

Description

Technical Field

[0001] This invention relates to the field of formation control, specifically to a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems. Background Technology

[0002] Eulerian-Lagrange systems can be used to describe the motion behavior of various physical systems, such as mechanical and electronic systems. The study of formation control for multi-Eulerian-Lagrange systems has wide applications in engineering. Taking mobile robots as an example, by designing effective formation control protocols, multiple robot systems can form and maintain specific formations, thereby collaboratively completing complex tasks far exceeding the capabilities of a single robot. Formation control mainly falls into two categories: time-invariant formation and time-varying formation. Time-invariant formation is typically suitable for applications where the formation shape is relatively stable and does not require frequent adjustments. However, the formation shape often needs to be dynamically adjusted according to task requirements to adapt to different working scenarios. The flexibility of time-varying formation allows the formation system to adapt to different task requirements and environmental changes. Furthermore, in the formation control problem of multi-Eulerian-Lagrange systems, practical application scenarios are often more complex. The system is not only continuously affected by external nonlinear disturbances, but also has unknown nonlinear dynamics within the model. These factors significantly increase the difficulty of controller design and implementation. In addition, such systems often involve high energy consumption when performing tasks; therefore, incorporating optimization into the design principles to minimize energy consumption is particularly important.

[0003] In recent decades, optimal control has become an important research direction, aiming to design control protocols for dynamic systems to minimize overall control costs while accomplishing specified tasks. Mathematically, optimal control typically requires solving the Hamilton-Jacobi-Bellman (HJB) equations. However, due to the complex nonlinear dynamics of Eulerian-Lagrange systems, these equations are often difficult to solve analytically. Against this backdrop, reinforcement learning offers a new approach to address these challenges. This method continuously interacts with the environment and uses feedback signals to evaluate the control effect, thereby optimizing the control strategy and progressively learning and generating appropriate control actions. A typical reinforcement learning framework includes two networks: an actor and a critic. The actor generates control actions, while the critic evaluates the value of the actions and provides feedback to the actor, continuously improving the system's control performance. The uncertainties in Eulerian-Lagrange systems often do not satisfy matching conditions, making traditional optimal control methods difficult to directly apply or generalize to such scenarios. To address this challenge, optimal backstepping control techniques, which combine reinforcement learning with backstepping methods, provide an effective solution to the control problems of such systems.

[0004] On the other hand, multi-Euler-Lagrange formation control systems may experience sudden failures during operation, typically occurring in system components such as actuators and sensors. These failures can damage system performance or even lead to instability. Therefore, the safety and reliability of the system face significant challenges, making fault tolerance considerations essential in optimal backstepping control design. The literature [“Optimised backstepping fault-tolerant control based on reinforcement learning for a class of nonlinear strict feedback systems with actuator and sensor failures.” (International Journal of Systems Science (2025): 1-18)] designs an optimal fault-tolerant control scheme for nonlinear strict feedback systems suffering from sensor and actuator bias failures. An adaptive update law is designed to dynamically compensate for sensor and actuator bias failures. Compared to traditional optimization methods, the proposed reinforcement learning-based optimization method is simple and easy to implement, and more readily integrated with fault-tolerant mechanisms.

[0005] However, the aforementioned methods are primarily geared towards strictly feedback nonlinear systems with single input and single output. In practical engineering, Eulerian-Lagrangian systems often exhibit complex characteristics of multiple inputs and multiple outputs, such as multi-joint robotic arms, drones, and mobile robots. These systems generally exhibit strong dynamic coupling, high variable dimensionality, and are susceptible to external disturbances and model uncertainties. In such real-world scenarios, the aforementioned optimal fault-tolerant control schemes are often difficult to apply directly. Furthermore, existing methods only design optimal fault-tolerant tracking control strategies for a single system, which becomes even more limiting when facing multi-system coordination tasks. Therefore, researching an optimal formation fault-tolerant control method capable of handling time-varying dynamics and meeting the requirements of cooperative control for multi-Eulerian-Lagrangian systems has significant theoretical value and practical engineering implications. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to propose a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems. This method aims to compensate for the effects of faults online by designing adaptive parameters and overcome the controller design difficulties caused by strong coupling and high dimensionality of the system. Thus, while minimizing control costs, it ensures that the multi-Euler-Lagrange system can form and accurately maintain a predetermined time-varying formation.

[0007] This invention provides a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems, comprising:

[0008] Step 1: Obtain a virtual leader and N following mobile robots, construct a communication topology graph of the N following mobile robots, and then obtain the adjacency matrix and the communication weight matrix; the virtual leader is a preset smooth time-varying reference trajectory, and there is communication interaction between the virtual leader and the following mobile robots;

[0009] Step 2: Construct the target mathematical model for each Euler-Lagrange system following the mobile robot;

[0010] Step 3: Based on the target mathematical model, construct a sensor fault model for each following mobile robot, and simultaneously construct an actuator fault model;

[0011] Step 4: Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, construct the target coordinate transformation equation;

[0012] Step 5: Derive the optimal virtual control law based on the target coordinate transformation equation. The objective expression for the optimal virtual control law The parameters in the objective expression are updated to obtain the optimal virtual control law. ;

[0013] Step 6: Based on the target coordinate transformation equation and the optimal virtual control law The approximate expression for the optimal output signal is derived, and the parameters in the approximate expression for the optimal output signal are updated to obtain the approximate value of the optimal output signal and the approximate value of the optimal input signal of the actuator.

[0014] Optionally, the communication topology diagram described in step 1 is represented as follows: ,in, Communication topology diagram The point set, ,in, This represents the 1st node, the Nth node, and one node corresponds to one following mobile robot. Communication topology diagram edge set, , This represents the undirected edge between the i-th node and the j-th node. It is an adjacency matrix. ,in, Let be an element in the adjacency matrix, and if ,but ,like ,but ;

[0015] The communication weight matrix is ​​obtained in the following way:

[0016] Obtain the communication weight between the virtual leader and each following mobile robot. , This indicates that the virtual leader communicates directly with the i-th following mobile robot, and all communication weights form a communication weight matrix. .

[0017] Optionally, step 2 specifically includes:

[0018] The initial mathematical model for the i-th Eulerian-Lagrange system following the mobile robot is expressed as:

[0019] ;

[0020] in, , Let m represent the position vector and velocity vector, respectively, and m represent the vector dimension. Represents the acceleration vector; The inertia matrix; The resultant force vector of the Coriolis force and the centripetal force; It is the gravity vector; The control force vector that has failed;

[0021] make The initial mathematical model is transformed to obtain the target mathematical model, which is expressed as:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] in, For coupling nonlinear uncertain vectors, Given that it is symmetric positive definite The control gain matrix.

[0027] Optionally, the sensor fault model described in step 3 can be represented as follows:

[0028] ;

[0029] in, , These are the position vector and velocity vector output by the sensor after the fault, respectively. , It is a bounded bias signal;

[0030] The actuator fault model is represented as follows:

[0031] ;

[0032] in, The control force vector that has failed. For the input of the actuator, It is a bounded bias signal.

[0033] Optionally, step 4 specifically includes:

[0034] Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, the initial coordinate transformation equation is constructed as follows:

[0035] ;

[0036] ;

[0037] ;

[0038] in, This represents the tracking error of the i-th following mobile robot after compensation. , and They are , and The estimate; This represents the time-varying parameter of the formation distance of the i-th following mobile robot. This represents the position vector output by the sensor of the j-th following mobile robot, which is directly connected to the i-th following mobile robot in the communication topology, after a failure. express The estimate, Let represent the bounded bias signal of the j-th robot following the mobile robot. This represents the time-varying parameter of the formation distance of the j-th following mobile robot. This represents the smooth, time-varying reference movement trajectory of the virtual leader. This represents the virtual error of the i-th following mobile robot after compensation. It is the optimal virtual control law. express Approximate value;

[0039] Transforming the initial coordinate transformation equation yields the target coordinate transformation equation, expressed as:

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] in, This represents the actual tracking error of the i-th following mobile robot. Let be the total coupling weight of the i-th following mobile robot; This represents the actual position vector of the j-th following mobile robot that is directly connected to the i-th following mobile robot in the communication topology diagram; This represents the actual virtual error of the i-th following mobile robot. express The estimation error, express The estimation error, This represents the actual formation tracking error. express The estimation error.

[0049] Optionally, step 5 specifically includes:

[0050] Step 5.1: Construct the first optimal performance index function , represented as:

[0051] ;

[0052] in, Let be the cost function, s represent the integration variable, and t represent the lower limit of integration. It is a compact set containing the equilibrium point. It is a permissive control set. It is a virtual control law;

[0053] Differentiating both sides of the equation for the first optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as:

[0054] ;

[0055] in, express Hamiltonian function, Represents the tracking error term. , for The derivative of for The derivative of for The derivative of for The derivative of for The derivative of , where T denotes transpose;

[0056] make Solving for it yields the optimal virtual control law. The first expression is represented as:

[0057] ;

[0058] right By decomposing the equations, we obtain the first decomposition equation:

[0059] ;

[0060] in, This indicates that the control gain is a positive design parameter. Indicates uncertain terms;

[0061] Will Approximation is obtained through neural networks. The formula is expressed as:

[0062] ;

[0063] in, It is the weight matrix of an ideal neural network; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error;

[0064] Based on the first decomposition equation and The formula for the optimal virtual control law By rearranging the first expression, we obtain the optimal virtual control law. The second expression is represented as:

[0065] ;

[0066] A reinforcement learning algorithm using an actor-critic neural network is employed to determine the optimal virtual control law. Asymptotically approximating the second expression yields the optimal virtual control law. The target expression is represented as:

[0067] ;

[0068] in, This represents the ideal actor weight estimate;

[0069] Step 5.2: Construct the first weight update equation, expressed as:

[0070] ;

[0071] in, The rules for updating actor weights are indicated. , and All are positive design parameters. It is the identity matrix;

[0072] Construct the first parameter update law, expressed as:

[0073] ;

[0074] in, and All are positive design parameters. A positive design parameter vector. express Update rules;

[0075] Step 5.3: Update the equation using the first weight. To update, update the law using the first parameter. Update it, and the updated version will be available. Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substitute into the optimal virtual control law The objective expression is used to obtain the optimal virtual control law. .

[0076] Optionally, step 6 specifically includes:

[0077] Step 6.1: Construct the second optimal performance index function , represented as:

[0078] ;

[0079] in, It is the optimal output signal of the actuator. The cost function;

[0080] Differentiating both sides of the equation for the second optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as:

[0081] ;

[0082] in, express Hamiltonian function, for The derivative of Given that it is symmetric positive definite The control gain matrix;

[0083] make Solve for it to obtain the optimal output signal of the actuator. The first expression is represented as:

[0084] ;

[0085] in, Represents the composite gradient term. ;

[0086] right Decomposing the equation yields the second decomposition equation, which is expressed as:

[0087] ;

[0088] in, Positive design parameters Indicates uncertain terms;

[0089] According to the second decomposition equation, for The first expression, after being rearranged, yields... The second expression is represented as:

[0090] ;

[0091] in, Represents the extended state vector. ;

[0092] Will Approximation is obtained through neural networks. The formula is expressed as:

[0093] ;

[0094] in, It is an ideal neural network weight matrix; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error;

[0095] according to The formula, The second expression is rearranged to obtain... The third expression is represented as:

[0096] ;

[0097] A reinforcement learning algorithm using an actor-critic neural network is employed to... By asymptotically approximating the third expression, we obtain an approximate expression for the optimal output signal, which is expressed as:

[0098] ;

[0099] in, This represents the ideal actor weight estimate. for Approximate value;

[0100] Step 6.2: Construct the second weight update equation, expressed as:

[0101] ;

[0102] in, The rules for updating actor weights are indicated. , and All are positive design parameters, and It is the identity matrix;

[0103] Construct the second parameter update law, expressed as:

[0104] ;

[0105] ;

[0106] in, , , , and All are positive design parameters. express Update rules, express Update rules;

[0107] Step 6.3: Update using the second weight update equation Updated by the second parameter update law The updated version Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substituting the approximate expression for the optimal output signal, we obtain an approximate value for the optimal output signal. ;

[0108] Step 6.4: Update according to the second parameter update law Then, based on the approximation of the optimal output signal... And the updated Calculate the approximate value of the optimal input signal for the actuator. Specifically, this is achieved through the following formula:

[0109] .

[0110] The beneficial effects of adopting the above technical solution are as follows:

[0111] 1. This invention studies a time-varying optimal formation fault-tolerant control method for multiple Euler-Lagrange systems. Euler-Lagrange systems generally possess characteristics such as strong dynamic coupling and high-dimensional variables, and are applicable to the modeling and control of practical physical systems such as multi-joint robotic arms, UAV swarms, and multi-mobile robots. This invention, based on low-level dynamic modeling of the Euler-Lagrange equations, significantly improves the universality of this control method across different types of intelligent agents.

[0112] 2. The method proposed in this invention enables a following mobile robot to adjust its relative distance to the leader as needed based on environmental changes. By adjusting this distance, dynamic changes in the topology are achieved, thereby enabling obstacle avoidance. Therefore, this method can significantly improve the reliability of multi-Euler-Lagrange systems in cooperative operations.

[0113] 3. The method proposed in this invention integrates optimal backstepping control and fault-tolerant mechanisms, enabling the system to maintain high-precision time-varying formation even when both sensors and actuators fail simultaneously. Simultaneously, this method achieves an optimal balance between energy consumption and control cost, significantly improving the robustness and control efficiency of multi-Euler-Lagrange systems in complex dynamic environments. Attached Figure Description

[0114] Figure 1 This is a flowchart illustrating a time-varying optimal formation fault-tolerant control method for a multi-Euler-Lagrange system according to an embodiment of the present invention.

[0115] Figure 2 This is a communication topology diagram of the virtual leader and five following mobile robot agents in an embodiment of the present invention;

[0116] Figure 3 This is a time-varying formation motion trajectory diagram of five Eulerian-Lagrange agents and a virtual leader on a plane in an embodiment of the present invention;

[0117] Figure 4 This is an adaptive parameter curve diagram in an embodiment of the present invention, wherein (a) is The estimated L2 norm curve, (b) is The estimated L2 norm curve;

[0118] Figure 5 As described in the embodiments of the present invention The estimated L2 norm curve;

[0119] Figure 6 This is a weight convergence curve of an actor-critic neural network according to an embodiment of the present invention, where (a) represents the critic weights. The estimated L2 curve, (b) represents the actor weights. The estimated L2 norm curve;

[0120] Figure 7 This is a weight convergence curve of another actor-critic neural network in an embodiment of the present invention, where (a) represents the critic weights. The estimated L2 curve, (b) represents the actor weights. The estimated L2 norm curve. Detailed Implementation

[0121] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0122] To address the problems existing in the prior art, this invention provides a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems. The purpose of this invention is to overcome the limitation that existing optimal fault-tolerant control methods are only applicable to single-input single-output strictly feedback nonlinear systems. Specifically, for a class of multi-Euler-Lagrange systems with strong dynamic coupling and high variable dimension characteristics in practical engineering, this invention solves the problem that when performing cooperative tasks, they are difficult to achieve time-varying optimal formation control due to simultaneous sensor and actuator bias faults.

[0123] This invention provides a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems, combined with... Figure 1 This may include the following steps:

[0124] Step 1: Obtain a virtual leader and N following mobile robots, construct the communication topology graph of the N following mobile robots, and then obtain the adjacency matrix and the communication weight matrix;

[0125] The virtual leader is a preset smooth time-varying reference movement trajectory, and the virtual leader and the following mobile robot have communication interaction;

[0126] A virtual leader and Consider a following mobile robot as a multi-Euler-Lagrange system; its communication relationship can be described as a graph. Communication diagram It contains at least one directed spanning tree with the leader as the root node, meaning there is a path between the leader and all the following mobile robots.

[0127] The communication topology diagram is represented as follows: ,in, Communication topology diagram The point set, ,in, This represents the 1st node, the Nth node, and one node corresponds to one following mobile robot. Communication topology diagram edge set, , This represents the undirected edge between the i-th node and the j-th node. It is an adjacency matrix. ,in, Let be an element in the adjacency matrix, and if ,but ,like ,but At the same time, an undirected graph means that there is a bidirectional edge between any pair of nodes, i.e. Then there must be .

[0128] Obtain the communication weight between the virtual leader and each following mobile robot. , This indicates that the virtual leader communicates directly with the i-th following mobile robot, and all communication weights form a communication weight matrix. ;

[0129] Furthermore, the present invention also constructs a positive definite matrix based on the adjacency matrix of the communication topology graph, specifically including:

[0130] Construct a degree diagonal matrix , , For each element in a degree diagonal matrix, the element in the degree diagonal matrix is... The calculation is based on the adjacency matrix, specifically implemented using the following formula:

[0131] ;

[0132] Constructing the Laplace matrix , Then construct a positive definite matrix. , ;

[0133] Step 2: Construct the target mathematical model for each Euler-Lagrange system following the mobile robot;

[0134] Consider a virtual leader and The initial mathematical model for the Eulerian-Lagrange system of the i-th following mobile robot, consisting of several follower robots, is expressed as:

[0135] ;

[0136] in, , Let m represent the position vector and velocity vector, respectively, and m represent the vector dimension. Represents the acceleration vector; The inertia matrix; The resultant force vector of the Coriolis force and the centripetal force; It is the gravity vector; The control force vector that has failed;

[0137] make The initial mathematical model is transformed to obtain the target mathematical model, which is expressed as:

[0138] ;

[0139] ;

[0140] ;

[0141] ;

[0142] in, For coupling nonlinear uncertain vectors, Given that it is symmetric positive definite The control gain matrix; at the same time, if and If it is continuous and bounded, then and It is continuous and bounded.

[0143] Step 3: Based on the target mathematical model, construct a sensor fault model for each following mobile robot, and simultaneously construct an actuator fault model;

[0144] Because the target mathematical model suffers from sensor failure, its precise state cannot be obtained. Therefore, in the design of subsequent control strategies, only the fault state is considered. It can be used. Therefore, a sensor fault model is constructed, which is expressed as:

[0145] ;

[0146] in, , These are the position vector and velocity vector output by the sensor after the fault, respectively. , For bounded bias signals, respectively satisfy and ,and and All are positive numbers.

[0147] The actuator fault model is represented as follows:

[0148] ;

[0149] in, The control force vector that has failed. For the input of the actuator, For a bounded bias signal, satisfying ,and It is a positive number.

[0150] Step 4: Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, construct the target coordinate transformation equation;

[0151] For a multi-Euler-Lagrange system suffering from actuator and sensor failures, the real state Input to the actuator Neither can be obtained; the system can only obtain the fault status. With faulty actuator output This presents an obstacle to designing the optimal backstepping control strategy. The core idea of ​​this strategy is to ensure that each virtual control law and actual controller in the backstepping design becomes the optimal solution for the corresponding subsystem, based on reinforcement learning algorithms. Therefore, directly using fault states and faulty actuator outputs in the reinforcement learning algorithm for controller design will lead to a significant deviation between the control output and the desired target, and may even cause system instability. To effectively compensate for the impact of sensor and actuator failures, this invention introduces an adaptive method to construct coordinate transformation equations, specifically through the following method:

[0152] Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, the initial coordinate transformation equation is constructed as follows:

[0153] ;

[0154] ;

[0155] ;

[0156] in, This represents the tracking error of the i-th following mobile robot after compensation. , and They are , and The estimate; This represents the time-varying parameter of the formation distance of the i-th following mobile robot. This represents the position vector output by the sensor of the j-th following mobile robot, which is directly connected to the i-th following mobile robot in the communication topology, after a failure. express The estimate, Let represent the bounded bias signal of the j-th robot following the mobile robot. This represents the time-varying parameter of the formation distance of the j-th following mobile robot. This represents the smooth, time-varying reference movement trajectory of the virtual leader. This represents the virtual error of the i-th following mobile robot after compensation. It is the optimal virtual control law. express Approximate value;

[0157] Transforming the initial coordinate transformation equation yields the target coordinate transformation equation, expressed as:

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] in, This represents the actual tracking error of the i-th following mobile robot. The total coupling weight of the i-th following mobile robot is the sum of all its inbound communication weights and the leader connection weight. This represents the actual position vector of the j-th following mobile robot that is directly connected to the i-th following mobile robot in the communication topology diagram; This represents the actual virtual error of the i-th following mobile robot. express The estimation error, express The estimation error, This represents the actual formation tracking error. express The estimation error;

[0167] Step 5: Derive the optimal virtual control law based on the target coordinate transformation equation. The objective expression for the optimal virtual control law The parameters in the objective expression are updated to obtain the optimal virtual control law. ;

[0168] Step 5.1: Construct the first optimal performance index function , represented as:

[0169] ;

[0170] in, Let be the cost function, s represent the integration variable, and t represent the lower limit of integration. It is a compact set containing the equilibrium point. It is a permissive control set. It is a virtual control law;

[0171] in, and For the same parameter, and They are the same parameter.

[0172] Differentiating both sides of the equation for the first optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as:

[0173] ;

[0174] in, express Hamiltonian function, Represents the tracking error term. , for The derivative of for The derivative of for The derivative of for The derivative of for The derivative of , where T denotes transpose;

[0175] make Solving for it yields the optimal virtual control law. The first expression is represented as:

[0176] ;

[0177] right By decomposing the equations, we obtain the first decomposition equation:

[0178] ;

[0179] in, This indicates that the control gain is a positive design parameter. Indicates uncertain terms;

[0180] Will Approximation is obtained through neural networks. The formula is expressed as:

[0181] ;

[0182] in, It is the weight matrix of an ideal neural network; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error;

[0183] Based on the first decomposition equation and The formula for the optimal virtual control law By rearranging the first expression, we obtain the optimal virtual control law. The second expression is represented as:

[0184] ;

[0185] However, The second expression contains unknown neural network weights. This control law is infeasible. Therefore, a reinforcement learning algorithm using an actor-critic neural network is employed to find the optimal virtual control law. Asymptotically approximating the second expression yields the optimal virtual control law. The target expression is represented as:

[0186] ;

[0187] in, This represents the ideal actor weight estimate;

[0188] Furthermore, this invention can also employ a reinforcement learning algorithm based on an actor-critic neural network to... By asymptotic approximation, we obtain:

[0189] ;

[0190] in, This represents an approximation of the gradient of the optimal cost function output by the critic network. This represents an estimate of the ideal critic weights;

[0191] Step 5.2: Construct the first weight update equation, expressed as:

[0192] ;

[0193] in, The rules for updating actor weights are indicated. , and All are positive design parameters. It is the identity matrix; where, and They have the same meaning.

[0194] In this invention There is also a corresponding update equation, expressed as:

[0195] ;

[0196] in, The update rule for representing the critic weights; where, and They have the same meaning.

[0197] Construct the first parameter update law, expressed as:

[0198] ;

[0199] in, and All are positive design parameters. A positive design parameter vector. express Update rules;

[0200] Step 5.3: Update the equation using the first weight. To update, update the law using the first parameter. Update it, and the updated version will be available. Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substitute into the optimal virtual control law The objective expression is used to obtain the optimal virtual control law. ;

[0201] Next, we will prove step 5 by constructing the following Lyapunov function. :

[0202] ;

[0203] in, This represents the actual tracking error. Describes a positive definite matrix. express An identity matrix of order 1. Represents the trace operation of a matrix. This represents the weight estimation error of the critic network. This represents the weight estimation error of the actor network. This represents the ideal weight of the actor network. This represents the ideal weight of the critic network. , ,and ;

[0204] also, satisfy ,in, , This represents the column vector stacking operator. This represents the actual formation tracking error. This represents the position vector of the first robot following the mobile robot;

[0205] For Lyapunov functions Perform differentiation, and apply the optimal virtual control law. Target expression First weight update equation The update equation and the first parameter update law can be used to obtain...

[0206] ;

[0207] in, Representing Lyapunov functions The derivative of It is a positive number. Representation matrix The smallest eigenvalue, These are positive design parameters. Indicates a compound bounded term;

[0208] and

[0209] ;

[0210] ;

[0211] ;

[0212] ;

[0213] , ;

[0214] in, These are positive design parameters. These are positive design parameters. The derivative of the time-varying parameter of formation distance is given. The derivative of the smooth time-varying reference signal representing the virtual leader. These are positive design parameters. It is a positive design parameter vector. Represents the ideal neural network weights. Represents the smallest eigenvalue of the matrix. These are positive design parameters.

[0215] Step 6: Using optimal backstepping control technology, design the optimal controller, the weight update law of the actor-critic network, and the adaptive parameter update law. Specifically, based on the target coordinate transformation equation and the optimal virtual control law... The approximate expression for the optimal output signal is derived, and the parameters in the approximate expression for the optimal output signal are updated to obtain the approximate value of the optimal output signal and the approximate value of the optimal input signal of the actuator.

[0216] Step 6.1: Construct the second optimal performance index function , represented as:

[0217] ;

[0218] in, It is the optimal output signal of the actuator. Let be the cost function; where, and For the same parameter, and They are the same parameter.

[0219] Differentiating both sides of the equation for the second optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as:

[0220] ;

[0221] in, express Hamiltonian function, for The derivative of Given that it is symmetric positive definite The control gain matrix;

[0222] make Solve for it to obtain the optimal output signal of the actuator. The first expression is represented as:

[0223] ;

[0224] in, Represents the composite gradient term. ;

[0225] right Decomposing the equation yields the second decomposition equation, which is expressed as:

[0226] ;

[0227] in, Positive design parameters Indicates uncertain terms;

[0228] According to the second decomposition equation, for The first expression, after being rearranged, yields... The second expression is represented as:

[0229] ;

[0230] in, Represents the extended state vector. ;

[0231] Will Approximation is obtained through neural networks. The formula is expressed as:

[0232] ;

[0233] in, It is an ideal neural network weight matrix; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error;

[0234] according to The formula, The second expression is rearranged to obtain... The third expression is represented as:

[0235] ;

[0236] We can also obtain:

[0237] ;

[0238] A reinforcement learning algorithm using an actor-critic neural network is employed to... By asymptotically approximating the third expression, we obtain an approximate expression for the optimal output signal, which is expressed as:

[0239] ;

[0240] in, This represents the ideal actor weight estimate. for Approximate value;

[0241] Furthermore, we can also obtain:

[0242] ;

[0243] in, This represents the ideal critic weight estimate. Represents the composite gradient term The neural network estimate;

[0244] Step 6.2: Construct the second weight update equation, expressed as:

[0245] ;

[0246] in, The rules for updating actor weights are indicated. , and All are positive design parameters, and It is the identity matrix; where, and They have the same meaning.

[0247] In this invention There is also a corresponding update equation, expressed as:

[0248] ;

[0249] in, The update rule for representing the critic weights; where, and They have the same meaning.

[0250] Construct the second parameter update law, expressed as:

[0251] ;

[0252] ;

[0253] in, , , , and All are positive design parameters. express Update rules, express Update rules;

[0254] Step 6.3: Update using the second weight update equation Updated by the second parameter update law The updated version Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substituting the approximate expression for the optimal output signal, we obtain an approximate value for the optimal output signal. ;

[0255] Step 6.4: Update according to the second parameter update law Then, based on the approximation of the optimal output signal... And the updated Calculate the approximate value of the optimal input signal for the actuator. Specifically, this is achieved through the following formula:

[0256] ;

[0257] Next, we will prove step 6 by constructing the following Lyapunov function. :

[0258] ;

[0259] in, This represents the weight estimation error of the critic network. This represents the weight estimation error of the actor network. Indicates bias signal The estimation error, Indicates bias signal The estimation error, , , ,and , This represents the ideal weight of the actor network. This represents the ideal weight of the critic network. Indicates bias signal The estimate, The bias signal indicates an actuator malfunction.

[0260] Then, for the Lyapunov function Differentiate and update the equation according to the second weight. The update equation, the second parameter update law, and the approximate value of the optimal input signal for the actuator. You can get

[0261] ;

[0262] in, Representing Lyapunov functions The derivative of It is a positive number. Indicates the actual virtual error. Indicates a positive design parameter. This indicates a compound bounded term. It is a positive number;

[0263]

[0264] ;

[0265] ;

[0266] ;

[0267] , ;

[0268] ;

[0269] ;

[0270] in, The derivative of the approximate value of the optimal virtual control law output by the actor network is represented by the derivative of the virtual control law output by the actor network. Representation matrix The absolute value of the largest eigenvalue, This indicates a sensor fault bias signal. This represents the ideal neural network weight matrix. Representation matrix The largest eigenvalue;

[0271] Based on Lyapunov stability theory, it is proved that the proposed time-varying optimal formation fault-tolerant control method can ensure the stability of the closed-loop system.

[0272] Based on the derivatives of the Lyapunov functions obtained in steps 5 and 6 above, the derivative of the total Lyapunov function can be summarized in the following form:

[0273] ;

[0274] make , and Then you can obtain

[0275] ;

[0276] Then, by integrating both sides of the above equation, we can obtain...

[0277] ;

[0278] Inequality means , , , , , All of these are semi-globally consistent and eventually bounded. Therefore, the time-varying optimal formation fault-tolerant control method designed in this invention can ensure the stability of the closed-loop system while enabling all following mobile robots to track the leader signal and form the desired time-varying formation.

[0279] To further illustrate the effectiveness of the proposed time-varying optimal formation fault-tolerant control method, simulation experiments were conducted on five mobile robots with Eulerian-Lagrange dynamics. The dynamics of the mobile robot are shown below:

[0280] ;

[0281] and

[0282] ;

[0283] ;

[0284] ;

[0285] In the formula, .

[0286] The sensor and actuator failure model is given as follows:

[0287] ;

[0288] ;

[0289] ;

[0290] The communication topology between the virtual leader and five following mobile robots is shown in Figure 2 The virtual leader's trajectory is shown below:

[0291] ;

[0292] In addition, the expected time-varying distances for the five following mobile robots and virtual leaders are given as

[0293] ;

[0294] and

[0295] ;

[0296] let ,set up , , , and Set the initial state of the five mobile robots as follows: , , , , and In the simulation experiment, the design parameters were selected as follows: , , , , , , , , , , , , .

[0297] let , Select as satisfy Next, we will verify the inequality conditions in the control method design process:

[0298] ;

[0299] ;

[0300] ;

[0301] ;

[0302] ;

[0303] The simulation results are shown in Figures 3 to 7 middle. Figure 3 This demonstrates the time-varying formation motion trajectories of five following mobile robots and a virtual leader on a plane. At that moment, the five mobile robots had successfully converged to the desired formation. To pass through the narrow passage, the mobile robots needed to dynamically shorten the distance to the virtual leader. During this period, the formation continuously contracted. After passing through the narrow passage, the formation needed to return to its original form, therefore... During this period, the formation gradually expanded. Figure 4 Show the adaptive parameter curves, where (a) is The estimated L2 norm curve, (b) is The estimated L2 norm curve, Figure 5 for The estimated L2 norm curve shows that this invention, through the design of adaptive parameters, effectively compensates for sensor and actuator failures. Finally, Figures 6 to 7 The diagram shows the weight convergence curves of the actor-critic neural network, where... Figure 6 (a) represents the critic's weight. The estimated L2 curve, (b) represents the actor weights. The estimated L2 norm curve; where, Figure 7 (a) represents the critic's weight. The estimated L2 curve, (b) represents the actor weights. The estimated L2 norm curve; Based on the above analysis, for multi-Euler-Lagrange systems with sensor and actuator failures, the time-varying optimal formation fault-tolerant control method proposed in this invention can enable multiple systems to achieve the desired time-varying formation while ensuring that the control cost is minimized.

[0304] In summary, the key points of this invention are:

[0305] 1. For multi-Euler-Lagrange systems with strong dynamic coupling and high variable dimension, this invention proposes a time-varying optimal formation fault-tolerant control method, which can still achieve high-precision time-varying formation and optimal performance control under harsh operating conditions where both sensors and actuators suffer from bias faults.

[0306] 2. Unlike traditional optimization frameworks, this invention introduces a reinforcement learning strategy, which has significant advantages such as simple structure and easy integration with fault-tolerant mechanisms. While ensuring optimality, it enhances the system's adaptability under abnormal operating conditions.

[0307] 3. The method proposed in this invention no longer relies on the two key assumptions commonly found in traditional optimal control: "the system dynamics are accurately known" and "the continuous excitation conditions are met." This significantly reduces the dependence on the accuracy of the system model and the excitation signal, and greatly improves the engineering applicability and deployment flexibility of the method.

[0308] This invention studies a time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems. Euler-Lagrange systems generally possess characteristics such as strong dynamic coupling and high-dimensional variables, and are applicable to the modeling and control of practical physical systems such as multi-joint robotic arms, UAV swarms, and multi-mobile robots. This invention, based on low-level dynamic modeling of the Euler-Lagrange equations, significantly improves the universality of this control method across different types of intelligent agents. The proposed method enables a following mobile robot to adjust its relative distance to the leader as needed according to environmental changes. By adjusting this distance, dynamic changes in the topology are achieved, thereby enabling obstacle avoidance. Therefore, this method significantly improves the reliability of multi-Euler-Lagrange systems in cooperative operations. By integrating optimal backstepping control and a fault-tolerant mechanism, the proposed method enables the system to maintain high-precision time-varying formation even when both sensors and actuators fail simultaneously. Simultaneously, this method achieves an optimal balance between energy consumption and control cost, significantly improving the robustness and control efficiency of multi-Euler-Lagrange systems in complex dynamic environments.

[0309] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems, characterized in that, include: Step 1: Obtain a virtual leader and N A following mobile robot, building N The communication topology of the following mobile robot is obtained, and then the adjacency matrix is ​​obtained, and the communication weight matrix is ​​obtained; the virtual leader is a preset smooth time-varying reference movement trajectory, and the virtual leader and the following mobile robot have communication interaction; Step 2: Construct the target mathematical model for each Euler-Lagrange system following the mobile robot; Step 3: Based on the target mathematical model, construct a sensor fault model for each following mobile robot, and simultaneously construct an actuator fault model; Step 4: Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, construct the target coordinate transformation equation; Step 5: Derive the optimal virtual control law based on the target coordinate transformation equation. The objective expression for the optimal virtual control law The parameters in the objective expression are updated to obtain the optimal virtual control law. ; Step 5.1: Construct the first optimal performance index function , represented as: ； in, Let be the cost function, s represent the integration variable, and t represent the lower limit of integration. It is a compact set containing the equilibrium point. It is a permissive control set. It is a virtual control law; Indicates the compensation after the first i The tracking error of a mobile robot This represents the optimal virtual control law; Differentiating both sides of the equation for the first optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as: ； in, express Hamiltonian function, Represents the tracking error term. , for The derivative of for The derivative of for The derivative of for The derivative of for The derivative of T Indicates transpose; Indicates the first i The total coupling weight of the following mobile robot This represents the position vector output by the sensor after a fault. It is a bounded bias signal. The estimate, Indicates the first i Time-varying parameters of the formation distance for a following mobile robot This represents the smooth, time-varying reference movement trajectory of the virtual leader. These are the elements in the adjacency matrix; make Solving for it yields the optimal virtual control law. The first expression is represented as: ； right By decomposing the equations, we obtain the first decomposition equation: ； in, This indicates that the control gain is a positive design parameter. Indicates uncertain terms; Will Approximation is obtained through neural networks. The formula is expressed as: ； in, It is the weight matrix of an ideal neural network; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error; Based on the first decomposition equation and The formula for the optimal virtual control law By rearranging the first expression, we obtain the optimal virtual control law. The second expression is represented as: ； A reinforcement learning algorithm using an actor-critic neural network is employed to determine the optimal virtual control law. Asymptotically approximating the second expression yields the optimal virtual control law. The target expression is represented as: ； in, This represents the ideal actor weight estimate; Step 5.2: Construct the first weight update equation, expressed as: ； in, The rules for updating actor weights are indicated. , and All are positive design parameters. It is the identity matrix; This represents an estimate of the ideal critic weights; Construct the first parameter update law, expressed as: ； in, and All are positive design parameters. A positive design parameter vector. express Update rules; Step 5.3: Update the equation using the first weight. To update, update the law using the first parameter. Update it, and the updated version will be available. Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substitute into the optimal virtual control law The objective expression is used to obtain the optimal virtual control law. ; Step 6: Based on the target coordinate transformation equation and the optimal virtual control law The approximate expression for the optimal output signal is derived, and the parameters in the approximate expression for the optimal output signal are updated to obtain the approximate value of the optimal output signal and the approximate value of the optimal input signal of the actuator.

2. The time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems according to claim 1, characterized in that, The communication topology diagram described in step 1 is represented as follows: ,in, Communication topology diagram The point set, ,in, Indicates the first node, the second node N There are 10 nodes, and each node corresponds to a following mobile robot. Communication topology diagram edge set, , Indicates the first i The node and the first j Undirected edges between nodes It is an adjacency matrix. ,in, Let be an element in the adjacency matrix, and if ,but ,like ,but ; The communication weight matrix is ​​obtained in the following way: Obtain the communication weight between the virtual leader and each following mobile robot. , This indicates that the virtual leader communicates directly with the i-th following mobile robot, and all communication weights form a communication weight matrix. .

3. The time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems according to claim 1, characterized in that, Step 2 specifically includes: The initial mathematical model for the i-th Eulerian-Lagrange system following the mobile robot is expressed as: ； in, , Let m represent the position vector and velocity vector, respectively, and m represent the vector dimension. Represents the acceleration vector; The inertia matrix; The resultant force vector of the Coriolis force and the centripetal force; It is the gravity vector; The control force vector that has failed; make The initial mathematical model is transformed to obtain the target mathematical model, which is expressed as: ； ； ； ； in, For coupling nonlinear uncertain vectors, Given that it is symmetric positive definite The control gain matrix.

4. The time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems according to claim 3, characterized in that, The sensor fault model described in step 3 is represented as follows: ； in, , These are the position vector and velocity vector output by the sensor after the fault, respectively. , It is a bounded bias signal; The actuator fault model is represented as follows: ； in, The control force vector that has failed. For the input of the actuator, It is a bounded bias signal.

5. The time-varying optimal formation fault-tolerant control method for multi-Euler-Lagrange systems according to claim 4, characterized in that, Step 4 specifically includes: Based on the communication topology graph, adjacency matrix, communication weight matrix, sensor fault model, and actuator fault model, the initial coordinate transformation equation is constructed as follows: ； ； ； in, Indicates the compensation after the first i The tracking error of a mobile robot , and They are , and The estimate; Indicates the first i Time-varying parameters of the formation distance for a following mobile robot Indicates the connection between the communication topology diagram and the first i The first one directly connected to the mobile robot j The position vector output by the sensor after a malfunction in a mobile robot. express The estimate, Indicates the first j A bounded bias signal that follows the mobile robot. Indicates the first j Time-varying parameters of the formation distance for a following mobile robot This represents the smooth, time-varying reference movement trajectory of the virtual leader. Indicates the compensation after the first i A virtual error following the mobile robot, It is the optimal virtual control law. express Approximate value; Transforming the initial coordinate transformation equation yields the target coordinate transformation equation, expressed as: ； ； ； ； ； ； ； ； in, Indicates the first i The actual tracking error of a mobile robot. For the first i The total coupling weight of the following mobile robot; Indicates the connection between the communication topology diagram and the first i The first one directly connected to the mobile robot j The true position vector of the following mobile robot; Indicates the first i A realistic virtual error that follows the mobile robot. express The estimation error, express The estimation error, This represents the actual formation tracking error. express The estimation error.

6. The time-varying optimal formation fault-tolerant control method for a multi-Euler-Lagrange system according to claim 5, characterized in that, Step 6 specifically includes: Step 6.1: Construct the second optimal performance index function , represented as: ； in, It is the optimal output signal of the actuator. The cost function; Differentiating both sides of the equation for the second optimal performance index function, and combining this with the target coordinate transformation equation, we obtain... The Hamilton-Jacobi-Bellman equation is expressed as: ； in, express Hamiltonian function, for The derivative of Given that it is symmetric positive definite The control gain matrix; make Solve for it to obtain the optimal output signal of the actuator. The first expression is represented as: ； in, Represents the composite gradient term. ; right Decomposing the equation yields the second decomposition equation, which is expressed as: ； in, Positive design parameters Indicates uncertain terms; According to the second decomposition equation, for The first expression, after being rearranged, yields... The second expression is represented as: ； in, Represents the extended state vector. ; Will Approximation is obtained through neural networks. The formula is expressed as: ； in, It is an ideal neural network weight matrix; Let represent the basis function vector, and It refers to the number of neurons; Indicates asymptotic error; according to The formula, The second expression is rearranged to obtain... The third expression is represented as: ； A reinforcement learning algorithm using an actor-critic neural network is employed to... By asymptotically approximating the third expression, we obtain an approximate expression for the optimal output signal, which is expressed as: ； in, This represents the ideal actor weight estimate. for Approximate value; Step 6.2: Construct the second weight update equation, expressed as: ； in, The rules for updating actor weights are indicated. , and All are positive design parameters, and It is the identity matrix; This represents an estimate of the ideal critic weights; Construct the second parameter update law, expressed as: ； ； in, , , , and All are positive design parameters. express Update rules, express Update rules; Step 6.3: Update using the second weight update equation Updated by the second parameter update law The updated version Substituting into the target coordinate transformation equation, we obtain the updated... The updated version And the updated Substituting the approximate expression for the optimal output signal, we obtain an approximate value for the optimal output signal. ; Step 6.4: Update according to the second parameter update law Then, based on the approximation of the optimal output signal... And the updated Calculate the approximate value of the optimal input signal for the actuator. Specifically, this is achieved through the following formula: 。

Images