A Reinforcement Learning-Based Dynamic Event Triggering Control Method for Mobile Robots

Abstract

This application belongs to the field of robot control, specifically relating to a dynamic event-triggered control method for mobile robots based on reinforcement learning. Addressing the problems of unknown wheel slippage effects and communication resource constraints, a kinematic model and an Euler-Lagrange dynamics model of a four-wheeled Mecanum wheel mobile robot are established. The motion deviation caused by four-wheel slippage is characterized as a mismatched disturbance. By combining the tracking error and the desired trajectory and introducing steady-state control, the trajectory tracking problem is transformed into an extended-dimensional system regulation problem constrained by worst-case disturbances. To reduce computational burden and transmission load, a dynamic event triggering mechanism is introduced, and the Critic reinforcement learning algorithm is designed to solve the Hamiltonian-Jacobi-Isax equations, obtaining an approximately optimal controller under dynamic event triggering conditions. Based on Lyapunov theory, the uniform event-bounded stability of the closed-loop system is analyzed, ensuring high-precision trajectory tracking performance of the robot while conserving communication and computational resources.

Description

Technical Field

[0001] This application belongs to the field of robot control, specifically relating to a dynamic event-triggered control method for mobile robots based on reinforcement learning. Background Technology

[0002] Wheeled mobile robots are widely used in industrial logistics, social services, and military reconnaissance due to their flexibility and high mobility. Among them, the four-wheeled Mecanum-Wheel Mobile Robot (FMWMR) possesses stable omnidirectional mobility and has a wide range of applications. However, this type of robot has redundant drive structures and numerous wheel-to-ground contact points, which easily lead to complex coupling and strong nonlinear characteristics during operation. Achieving high-precision trajectory tracking control in complex dynamic environments presents significant technical challenges.

[0003] In practical engineering applications, wheel slippage is one of the core problems faced by FMWMR. Uneven ground friction coefficient, wear of mechanical parts, and load fluctuations inevitably cause wheel slippage, causing the robot's actual kinematic model to deviate from the ideal model, introducing unmatched uncertainty disturbances, and significantly reducing the system's trajectory tracking accuracy and control robustness.

[0004] Existing traditional control methods, such as sliding mode control and backstepping, can mitigate the effects of uncertainty to some extent, but they struggle to achieve optimal trajectory tracking for complex dynamic uncertainties induced by wheel slippage, and their control performance still needs improvement. Meanwhile, modern robot-mounted embedded control systems suffer from limited computing resources and communication bandwidth. Traditional digital control generally employs a periodic triggering mechanism, executing control tasks at fixed time intervals regardless of the system's operating state. This can lead to redundant calculations and invalid data transmissions when the system is in a steady state, resulting in low resource utilization. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a reinforcement learning-based dynamic event-triggered control method for mobile robots. Its aim is to overcome the shortcomings of existing FMWMR trajectory tracking control technology and solve technical challenges such as strong mismatched disturbances caused by wheel slippage, waste of communication and computing resources in traditional time-triggered control, fixed static event triggering thresholds, poor adaptive capability, and difficulty in accurately solving the HJI equation. The technical solution is as follows: A method for dynamic event-triggered control of a mobile robot based on reinforcement learning includes the following steps: S1. Establish a robot system model under slippage disturbance: Establish the kinematic model and Euler-Lagrange dynamics model of FMWMR, and uniformly model the motion deviation caused by four-wheel slippage as a non-matching disturbance to complete the accurate description of the model; S2. Set the desired trajectory of the mobile robot, and introduce steady-state control by splicing the trajectory tracking error with the desired trajectory. Transform the trajectory tracking problem into the optimal adjustment problem of the augmented system, and characterize the slippage effect in the form of the worst perturbation. S3. Design the value function to obtain the optimal control strategy and the worst-case perturbation, and obtain the HJI equation describing the optimal zero-sum game strategy; S4. Design a dynamic event triggering mechanism (DET), introduce auxiliary dynamic variables, and construct the HJI equation under DET; S5. Construct a reinforcement learning solution framework based on Critic neural network, and use a single Critic neural network to approximate the optimal performance index function to solve the HJI equation; S6. Construct the Bellman error, design the Critic network weight update law, and adaptively approximate the optimal control law and the worst disturbance.

[0006] Preferably, the FMWMR forward kinematic model under the influence of sliding in step S1 is: ; Differentiating the above equation, we obtain the following dynamic equation. ; in, Indicates the position and azimuth of FMWMR. Represents a three-dimensional column vector The derivative of for The second derivative, Indicates the radius of the Mecanum wheel. This represents the slip ratio matrix, where slip ratio is... Indicates the degree of wheel slippage. This represents the angular vector representing the rotation of the four wheels. express The derivative of , i.e., the angular velocity vector of the four wheels. express The derivative of ; express The derivative of and These represent the lateral and longitudinal distances from the center of mass to the center of each wheel, respectively. It is an identity matrix with matching dimensions; The inverse kinematic equations of FMWMR under sliding perturbation are: ; in, ; The dynamic model of FMWMR is obtained as follows: ; in, For the control input of the FMWMR trajectory control model, The coefficient of viscous friction is... ; ; ; For the total mass of FMWMR, The moment of inertia of the FMWMR rotation. Let be the moment of inertia of a single Mecanum wheel about its axis of rotation; Substituting the inverse kinematics model and the dynamics model into the above equation, we obtain the following dynamic equations describing the motion of FMWMR: ; make , We can obtain: ; in, ; ; ; .

[0007] Preferably, the expected FMWMR trajectory in step S2 is: ; In the formula, The trajectory tracking error is defined as Its dynamic equations satisfy: ; Introduce the following for the desired trajectory Steady-state control under no external disturbances : ; in, ; Define the state of the augmented system as , The dynamic equation can be expressed as: ; ; ; ; It is the control input of the error augmentation system; By transforming unknown sliding perturbations into mathematical game objects aimed at disrupting system stability. The dynamic equations of the augmented system for robust control design can be reconstructed as follows: .

[0008] Preferably, in step S3: S31. Design the value function to obtain the Hamiltonian function; S32. Based on the Hamiltonian function, the optimal control strategy, worst-case disturbance, and HJI equation are obtained.

[0009] Preferably, in step S31, a discount value index function is defined. ; in, As a discount factor, the state value is defined as The combined value of control and disturbance is defined as: ; and All are positive definite symmetric matrices. Given the disturbance attenuation parameter; The optimal performance-value function can be expressed as the following Min-Max optimization problem: ; in, The optimal performance index function. and These represent the system's permissible control set and disturbance set, respectively. Assumption Since it is continuously differentiable, its derivative along the trajectory of the augmented system can be used to construct the following Hamiltonian function: ; in, Represents the value function in relation to the state The vector of partial derivatives.

[0010] Preferably, in step S32, the extreme value condition of Nash equilibrium in differential game theory is applied, i.e. and The optimal robust control law under ideal conditions can be obtained separately. worst-case disturbance The parsing form: ; ; Substituting the extreme value solution back into the Hamiltonian function yields the HJI equation describing the optimal zero-sum game strategy: .

[0011] Preferably, the system's state sampling measurement error is defined in step S4. The tracking error at the current moment Tracking error compared to the last trigger time Difference: ; At the trigger time Updated discretization event-triggered optimal control law worst-case disturbance They can be represented as: ; in, To trigger at the time The system state is determined by introducing an internal dynamic auxiliary variable. Its evolution over time is governed by the following first-order differential equation: ; in, This is the initial value for the dynamic variable; and The design parameters must be positive and must satisfy the following conditions: ; Based on this dynamic auxiliary variable, the next trigger time... The following DET execution conditions apply: ; In the formula, A positive weighting adjustment parameter; The continuous-time HJI equations are transformed into HJI equations under discrete event triggering: .

[0012] Preferably, the optimal performance index function in step S5 Available with The Critic network with hidden layer neurons is represented as follows: ; in, The ideal constant weight vector of the Critic network; For linearly independent neuron activation basis functions; vectors This represents the approximation error of the neural network. Find the augmented state for both sides of the above equation. From the gradient vector, the partial derivative of the optimal value function can be obtained as follows: ; Obtain the optimal robust control law that depends on the ideal network weights. worst-case disturbance Structural expression: ; The optimal Hamiltonian function is obtained: ; in, The residual term is expressed as follows: ; definition Given the current estimated weights of the Critic network, the approximate control law can be directly obtained using these estimated weights. With approximate perturbation : ; .

[0013] Preferably, in step S6, the estimate of the single Critic network is substituted into the constructed Hamiltonian function, and its actual output is an approximate Hamiltonian function. It can be expanded as follows: ; Define matrix parameters: ; ; Since the ideal optimal Hamiltonian function is always equal to zero, the deviation between the approximate Hamiltonian function and the ideal value is defined as the Bellman error. Based on the above expansion, the Bellman error can be calculated as follows: ; The online training objective of the Critic network is to minimize the squared integral of the Bellman error, i.e. .

[0014] Preferably, a normalized least squares algorithm with an exponential forgetting factor is used to update the estimated weights of the Critic network. : ; in, Defined as the partial derivative of the approximate Hamiltonian function with respect to the weights, i.e.: ; Among them, parameters The learning rate of the neural network; With a positive constant exponential forgetting factor, the Critic learning algorithm drives the weights in real time through the aforementioned update law. Towards the ideal optimal weight Approaching.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: This application establishes a kinematic and dynamic model of FMWMR under wheel slip disturbance, characterizing wheel slip deviation as mismatch disturbance, and transforming the trajectory tracking problem into an adjustment problem under worst-case disturbance constraints through dimension expansion transformation. To reduce the consumption of system communication transmission and computing resources, a DET mechanism is introduced, and a Critic reinforcement learning algorithm is designed to solve the HJI equation, obtaining a robust optimal control strategy under dynamic event triggering conditions, thereby effectively suppressing wheel mismatch slip disturbance. Under the premise of significantly reducing the controller update frequency and saving on-board embedded resources, the high-precision robust trajectory tracking performance of the robot is guaranteed. Attached Figure Description

[0016] Figure 1 To use the control method FMWMR of the present invention in A diagram showing the expected movement trajectory and the actual movement trajectory in the desired direction; Figure 2 To use the control method FMWMR of the present invention in A diagram showing the expected movement trajectory and the actual movement trajectory in the desired direction; Figure 3 A diagram showing the desired and actual orientation angles during the movement process of FMWMR using the control method of this invention; Figure 4 To control the weights of the Critic neural network using the method of this invention The convergence curve; Figure 5 For the approximate optimal discrete control law The evolution curve; Figure 6 For approximate worst-case perturbation The evolution curve; Figure 7 For dynamic auxiliary variables The evolutionary trajectory; Figure 8 For trajectory tracking error under the DET mechanism; Figure 9 For trajectory tracking error under the SET mechanism; Figure 10 The distribution of trigger times under the DET mechanism; Figure 11 The distribution of trigger times under the SET mechanism; Figure 12 The diagram shows the number of triggers under the SET mechanism, the DET mechanism, and the time-triggered reinforcement learning control method. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] A method for dynamic event-triggered control of a mobile robot based on reinforcement learning includes the following steps: S1. Establish a robot system model under slippage disturbance, establish the kinematic model and Euler-Lagrange dynamics model of FMWMR, and uniformly model the motion deviation caused by four-wheel slippage as a non-matching disturbance to complete the accurate description of the model; The forward kinematic model of FMWMR under the influence of sliding in step S1 is as follows: ; Differentiating the above equation, we obtain the following dynamic equation.

[0019] ; in, Indicates the position and azimuth of FMWMR. Represents a three-dimensional column vector The derivative of for The second derivative, Indicates the radius of the Mecanum wheel. This represents the slip ratio matrix, where slip ratio is... Indicates the degree of wheel slippage. This represents the angular vector representing the rotation of the four wheels. express The derivative of , i.e., the angular velocity vector of the four wheels. express The derivative; ; express The derivative of and These represent the lateral and longitudinal distances from the center of mass to the center of each wheel (i.e., half the wheel track and half the wheelbase). It is an identity matrix with matching dimensions.

[0020] The inverse kinematic equations of FMWMR under sliding perturbation are: ; in, .

[0021] The dynamic model of FMWMR is obtained as follows: ; in, For the control input of the FMWMR trajectory control model, It is the coefficient of viscous friction; ; ; , For the total mass of FMWMR, The moment of inertia of the FMWMR rotation. Let be the moment of inertia of a single Mecanum wheel about its axis of rotation.

[0022] Substituting the inverse kinematics model and the dynamics model into the above equation, we obtain the following dynamic equations describing the motion of FMWMR: ; make , We can obtain: ; in, ; ; ; .

[0023] S2. Set the desired trajectory of the mobile robot, and introduce steady-state control by splicing the trajectory tracking error with the desired trajectory. Transform the trajectory tracking problem into the optimal adjustment problem of the augmented system, and characterize the slippage effect in the form of the worst perturbation.

[0024] The expected trajectory of FMWMR in step S2 is: ; In the formula, The trajectory tracking error is defined as Its dynamic equations satisfy: ; Introduce the following for the desired trajectory Steady-state control under no external disturbances : ; in, .

[0025] Define the state of the augmented system as , The dynamic equation can be expressed as: ; in, , , , It is the control input of the error augmentation system.

[0026] By transforming unknown sliding perturbations into mathematical game objects aimed at disrupting system stability. The dynamic equations of the augmented system for robust control design can be reconstructed as follows: .

[0027] S3. Design the value function to obtain the optimal control strategy and the worst-case perturbation, and obtain the HJI equation describing the optimal zero-sum game strategy; S31. Design the value function to obtain the Hamiltonian function; In step S31, the discount value index function is defined: ; in, As a discount factor, the state value is defined as The combined value of control and disturbance is defined as , and All are positive definite symmetric matrices. is the given disturbance attenuation parameter.

[0028] The optimal performance-value function can be expressed as the following Min-Max optimization problem: ; in, The optimal performance index function. and These represent the system's permissible control set and disturbance set, respectively.

[0029] Assumption Since it is continuously differentiable, its derivative along the trajectory of the augmented system can be used to construct the following Hamiltonian function: ; in, Represents the value function in relation to the state The vector of partial derivatives.

[0030] S32. Based on the Hamiltonian function, the optimal control strategy, worst-case disturbance, and HJI equation are obtained.

[0031] In step S32, the extreme value condition of Nash equilibrium in differential game theory is applied, i.e. and The optimal robust control law under ideal conditions can be obtained separately. worst-case disturbance The parsing form: ; Substituting the above extreme value solution back into the Hamiltonian function yields the HJI equation describing the optimal zero-sum game strategy: .

[0032] S4. Design a dynamic event triggering mechanism (DET), introduce auxiliary dynamic variables, and construct the HJI equation under DET; In step S4, the system's state sampling measurement error is defined. The tracking error at the current moment Tracking error compared to the last trigger time Difference: ; At the trigger time Updated discretization event-triggered optimal control law worst-case disturbance They can be represented as: ; To achieve dynamic threshold adjustment, this invention introduces an internal dynamic auxiliary variable based on the event triggering mechanism. Its evolution over time is governed by the following first-order differential equation: ; in, This is the initial value for the dynamic variable; and The design parameters must be positive and must satisfy the following conditions: .

[0033] Based on this dynamic auxiliary variable, the next trigger time of the system Strictly determined by the following DET execution conditions ; In the formula, The weighting adjustment parameter is positive.

[0034] Accordingly, combining the above DET mechanism, the continuous-time HJI equation is transformed into the HJI equation under discrete event triggering: .

[0035] S5. Construct a reinforcement learning solution framework based on Critic neural network, and use a single Critic neural network to approximate the optimal performance index function to solve the HJI equation.

[0036] The optimal performance index function in step S5 Available with The Critic network with hidden layer neurons is represented as follows: ; in, The ideal constant weight vector of the Critic network; For linearly independent neuron activation basis functions; vectors This represents the approximation error of the neural network.

[0037] Find the augmented state for both sides of the above equation. From the gradient vector, the partial derivative of the optimal value function can be obtained as follows: ; Obtain the optimal robust control law that depends on the ideal network weights. worst-case disturbance Structural expression: ; The optimal Hamiltonian function is obtained: ; in, The residual term is expressed as follows: .

[0038] However, in actual online control processes, the ideal network weight... and approximation error All are unknown. Therefore, the definition... These are the estimated weights of the Critic network. Using these estimated weights, the approximate control law can be directly obtained. With approximate perturbation : ; .

[0039] S6. Construct the Bellman error, design the Critic network weight update law, and adaptively approximate the optimal control law and worst-case perturbation. Analyze the uniform final bounded stability of FMWMR, and complete the dynamic event-triggered control of the omnidirectional mobile robot under reinforcement learning sliding perturbation.

[0040] In step S6, the estimators of the single Critic network (i.e., the approximate value function and the approximate control law) are... With approximate perturbation Substituting this into the constructed Hamiltonian function, its actual output approximate Hamiltonian function It can be expanded as follows: ; For ease of writing, matrix parameters are defined. as well as .

[0041] Since the ideal optimal Hamiltonian function is always equal to zero The deviation between the approximate Hamiltonian function and the ideal value is defined as the Bellman error. Combining the above expansion, the Bellman error can be calculated as follows: ; The online training objective of the Critic network is to minimize the squared integral of the Bellman error, i.e. To improve the online convergence speed of the weights and ensure the transient stability of the system, this invention employs a normalized least squares algorithm with an exponential forgetting factor to update the estimated weights of the Critic network. : ; in, Defined as the partial derivative of the approximate Hamiltonian function with respect to the weights, i.e.: ; Among them, parameters The learning rate of the neural network; A positive constant exponential forgetting factor is used to gradually attenuate the influence of historical outdated data on the current network gradient under complex time-varying mismatch slippage conditions, thereby preventing the covariance matrix from diverging and improving the network's dynamic adaptability.

[0042] The proposed Critic learning algorithm drives the weights in real time through the aforementioned update law. Towards the ideal optimal weight Approximation ensures that the trajectory tracking controller can autonomously converge to the optimal game strategy under the scheduling of the DET mechanism.

[0043] Figure 1 , Figure 2 , Figure 3 FMWMR respectively , and The tracking effect of the three state dimensions on the desired trajectory, with the solid line representing the actual tracking trajectory and the dashed line representing the desired tracking trajectory; Figure 4 The weight change curve of the Critic neural network proposed in this invention reflects the training process of the neural network. As the training iterates, the weights tend to stabilize. Figure 5 , Figure 6 , Figure 7 These are approximately optimal discrete control laws. The evolution curve and the approximate worst perturbation Evolution curves and dynamic auxiliary variables The evolutionary trajectory; Figure 8 , Figure 9 The figures represent the trajectory tracking errors under the DET and SET mechanisms, respectively. Thanks to the adaptive adjustment capability of the dynamic threshold, the DET scheme not only has a convergence speed comparable to the SET scheme, but also has better trajectory tracking accuracy.

[0044] Figure 10 , Figure 11 The figures show the trigger point distribution of the DET and SET mechanisms throughout the simulation cycle. Compared with the SET mechanism, the trigger points of the DET mechanism are significantly sparser.

[0045] Figure 11 The cumulative trigger counts for the three mechanisms are shown. The solid blue line represents SET control, the dashed red line represents DET proposed in this invention, and the dotted black line represents the traditional time-triggered mechanism. Within the simulation time of seconds, the traditional time-triggered mechanism needs to issue a command. The next control command, SET control, lowers it to... This invention proposes a DET that only triggers... Next. Compared to time-triggered mechanisms, the SET and DET mechanisms reduce the communication frequency. and Gundam This fully demonstrates the advantages of the DET mechanism proposed in this invention in saving network bandwidth.

[0046] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for dynamic event-triggered control of a mobile robot based on reinforcement learning, characterized in that, Includes the following steps: S1. Establish a robot system model under slippage disturbance: Establish the kinematic model and Euler-Lagrange dynamics model of FMWMR, and uniformly model the motion deviation caused by four-wheel slippage as a non-matching disturbance to complete the accurate description of the model; S2. Set the desired trajectory of the mobile robot, and introduce steady-state control by splicing the trajectory tracking error with the desired trajectory. Transform the trajectory tracking problem into the optimal adjustment problem of the augmented system, and characterize the slippage effect in the form of the worst perturbation. S3. Design the value function to obtain the optimal control strategy and the worst-case perturbation, and obtain the HJI equation describing the optimal zero-sum game strategy; S4. Design a dynamic event triggering mechanism (DET), introduce auxiliary dynamic variables, and construct the HJI equation under DET; S5. Construct a reinforcement learning solution framework based on Critic neural network, and use a single Critic neural network to approximate the optimal performance index function to solve the HJI equation; S6. Construct the Bellman error, design the Critic network weight update law, and adaptively approximate the optimal control law and the worst disturbance.

2. The reinforcement learning-based dynamic event triggering control method for mobile robots according to claim 1, characterized in that, The forward kinematic model of FMWMR under the influence of sliding in step S1 is as follows: ； Differentiating the above equation, we obtain the following dynamic equation. ； in, Indicates the position and azimuth of FMWMR. Represents a three-dimensional column vector The derivative of for The second derivative, Indicates the radius of the Mecanum wheel. This represents the slip ratio matrix, where slip ratio is... Indicates the degree of wheel slippage. This represents the angular vector representing the rotation of the four wheels. express The derivative of , i.e., the angular velocity vector of the four wheels. express The derivative of ； express The derivative of and These represent the lateral and longitudinal distances from the center of mass to the center of each wheel, respectively. It is an identity matrix with matching dimensions; The inverse kinematic equations of FMWMR under sliding perturbation are: ； in, ； The dynamic model of FMWMR is obtained as follows: ； in, For the control input of the FMWMR trajectory control model, It is the coefficient of viscous friction; ； ； ； For the total mass of FMWMR, The moment of inertia of the FMWMR rotation. Let be the moment of inertia of a single Mecanum wheel about its axis of rotation; Substituting the inverse kinematics model and the dynamics model into the above equation, we obtain the following dynamic equations describing the motion of FMWMR: ； make , We can obtain: ； in, ； ； ； 。 3. The mobile robot dynamic event triggering control method based on reinforcement learning according to claim 1, characterized in that, The expected trajectory of FMWMR in step S2 is: ； In the formula, The trajectory tracking error is defined as Its dynamic equations satisfy: ； Introduce the following for the desired trajectory Steady-state control under no external disturbances : ； in, ; Define the state of the augmented system as , The dynamic equation can be expressed as: ； ； ； ； It is the control input of the error augmentation system; By transforming unknown sliding perturbations into mathematical game objects aimed at disrupting system stability. The dynamic equations of the augmented system for robust control design can be reconstructed as follows: 。 4. The mobile robot dynamic event triggering control method based on reinforcement learning according to claim 1, characterized in that, In step S3: S31. Design the value function to obtain the Hamiltonian function; S32. Based on the Hamiltonian function, the optimal control strategy, worst-case disturbance, and HJI equation are obtained.

5. The reinforcement learning-based dynamic event triggering control method for mobile robots according to claim 4, characterized in that, In step S31, the discount value index function is defined: ； in, As a discount factor, the state value is defined as The combined value of control and disturbance is defined as: ； and All are positive definite symmetric matrices. Given the disturbance attenuation parameter; The optimal performance-value function can be expressed as the following Min-Max optimization problem: ； in, The optimal performance index function. and These represent the system's permissible control set and disturbance set, respectively. Assumption Since it is continuously differentiable, its derivative along the trajectory of the augmented system can be used to construct the following Hamiltonian function: ； in, Represents the value function in relation to the state The vector of partial derivatives.

6. The reinforcement learning-based dynamic event triggering control method for mobile robots according to claim 4, characterized in that, In step S32, the extreme value condition of Nash equilibrium in differential game theory is applied, i.e. and The optimal robust control law under ideal conditions can be obtained separately. worst-case disturbance The parsing form: ； ； Substituting the above extreme value solution back into the Hamiltonian function yields the HJI equation describing the optimal zero-sum game strategy: 。 7. The mobile robot dynamic event triggering control method based on reinforcement learning according to claim 1, characterized in that, In step S4, the system's state sampling measurement error is defined. The tracking error at the current moment Tracking error compared to the last trigger time Difference: ； At the trigger time Updated discretization event-triggered optimal control law worst-case disturbance They can be represented as: ； in, To trigger at the time The system state is determined by introducing an internal dynamic auxiliary variable. Its evolution over time is governed by the following first-order differential equation: ； in, This is the initial value for the dynamic variable; and The design parameters must be positive and must satisfy the following conditions: ; Based on this dynamic auxiliary variable, the next trigger time... The execution conditions of DET are determined as follows: ； In the formula, A positive weighting adjustment parameter; The continuous-time HJI equations are transformed into HJI equations under discrete event triggering: 。 8. The reinforcement learning-based dynamic event triggering control method for mobile robots according to claim 1, characterized in that, The optimal performance index function in step S5 Available with The Critic network with hidden layer neurons is represented as follows: ； in, The ideal constant weight vector of the Critic network; For linearly independent neuron activation basis functions; vectors This represents the approximation error of the neural network. Find the augmented state for both sides of the above equation. From the gradient vector, the partial derivative of the optimal value function can be obtained as follows: ； Obtain the optimal robust control law that depends on the ideal network weights. worst-case disturbance Structural expression: ； The optimal Hamiltonian function is obtained: ； in, The residual term is expressed as follows: ； definition Given the current estimated weights of the Critic network, the approximate control law can be directly obtained using these estimated weights. With approximate perturbation : ； 。 9. The mobile robot dynamic event triggering control method based on reinforcement learning according to claim 1, characterized in that, In step S6, the estimator of the single Critic network is substituted into the constructed Hamiltonian function, and its actual output is the approximate Hamiltonian function. It can be expanded as follows: ； Define matrix parameters: ； ； Since the ideal optimal Hamiltonian function is always equal to zero, the deviation between the approximate Hamiltonian function and the ideal value is defined as the Bellman error. The Bellman error is calculated as follows: ； The online training objective of the Critic network is to minimize the squared integral of the Bellman error, i.e. .

10. The reinforcement learning-based dynamic event triggering control method for mobile robots according to claim 9, characterized in that, The estimated weights of the Critic network are updated using a normalized least squares algorithm with an exponential forgetting factor. : ； in, Defined as the partial derivative of the approximate Hamiltonian function with respect to the weights, i.e.: ； Among them, parameters The learning rate of the neural network; With a positive constant exponential forgetting factor, the Critic learning algorithm drives the weights in real time through the aforementioned update law. Towards the ideal optimal weight Approaching.

Images