Event-triggered control method for unmanned ship heading tracking based on adaptive dynamic programming
By combining adaptive dynamic programming and event-triggered mechanisms with the Actor-Critic algorithm and fuzzy logic system, a course tracking controller for unmanned surface vessels (USVs) was designed. This design solved the problems of servo wear and increased energy consumption of USVs in harsh sea conditions, and achieved efficient and energy-saving course tracking control.
Patent Information
- Application Number
- CN202411048661.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-01
AI Technical Summary
When unmanned vessels navigate in rough sea conditions, the frequent execution of control commands by the rudder motors leads to increased mechanical wear and energy consumption, which existing control strategies cannot effectively address.
An adaptive dynamic programming method is adopted, combined with the Actor-Critic algorithm and fuzzy logic system, to design an event triggering mechanism, obtain the optimal heading feedback controller, and realize the heading tracking control of the unmanned vessel through the weight update law of the fuzzy logic system.
It reduces mechanical wear and energy consumption of actuators, improves control efficiency, lowers system energy consumption, and the control effect is more in line with the needs of maritime practice.
Smart Images

Figure CN118567365B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an unmanned ship heading tracking event triggering control method based on adaptive dynamic programming. Background Art
[0002] With the development of automatic control technology in the field of marine engineering, unmanned surface vehicles (USV) have undoubtedly become a hot topic of concern for many scholars. Unmanned ships can replace humans to perform some dangerous and boring special operations at sea, and have extremely high research value in protecting the safety of human lives at sea. At present, they have been widely used in resource exploration, coastal surveillance, military patrol and reconnaissance, and other fields.
[0003] Due to the influence of complex sea conditions such as wind, waves and ocean currents, the heading tracking control performance of unmanned ships has been severely challenged. Therefore, in order to ensure the safety and efficiency of navigation, it is necessary to improve the heading tracking control of unmanned ships. As an advanced control concept that pursues the maximum reward at the lowest cost, the optimal control strategy plays an important role in the control design and construction of intelligent systems based on artificial intelligence. What we hope to achieve the best control effect with the smallest error is the adaptive dynamic programming method. The adaptive dynamic programming method can help achieve the optimal control of the unmanned ship tracking control system. However, when the ship is sailing in harsh sea conditions, the ship's steering gear often needs to execute control commands frequently, which will lead to mechanical wear and increased energy consumption of the actuator, and even cause actuator failure. Summary of the invention
[0004] The present invention provides an unmanned ship heading tracking event triggering control method based on adaptive dynamic programming to overcome the above technical problems.
[0005] In order to achieve the above object, the technical solution of the present invention is:
[0006] An unmanned ship heading tracking event triggering control method based on adaptive dynamic programming includes the following steps:
[0007] S1: Establishing a mathematical model for the heading control of an unmanned ship considering the uncertain interference conditions faced by the unmanned ship at sea during navigation, so as to convert the mathematical model for heading control into a second-order state space equation and obtain a mathematical model for heading tracking control;
[0008] S2: according to the tracking error of the unmanned ship heading, a dynamic nominal system about the error and the reference signal is obtained; according to the heading tracking control mathematical model, an optimal feedback controller of the unmanned ship heading is obtained;
[0009] S3: Based on the Actor-Critic algorithm and the unmanned ship heading optimal feedback controller, a weight update law of the Actor fuzzy logic system and a weight update law of the Critic fuzzy logic system are obtained;
[0010] S4: According to the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system, the optimal feedback controller of the unmanned ship heading under event triggering, the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are obtained to realize the event-triggered adaptive dynamic programming heading tracking control of the unmanned ship.
[0011] Beneficial effect: The present invention provides an event-triggered control method for unmanned ship heading tracking based on adaptive dynamic programming, establishes an unmanned ship heading control mathematical model taking into account the uncertain interference conditions faced by the unmanned ship at sea during navigation, and transforms it into a heading tracking control mathematical model; then, according to the dynamic nominal system of errors and reference signals, obtains the unmanned ship heading optimal feedback controller; based on the Actor-Critic algorithm, obtains the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system; finally, obtains the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering, and realizes the event-triggered adaptive dynamic programming heading tracking control of the unmanned ship.
[0012] The present invention takes into account the problem that when a ship is sailing in severe sea conditions, the ship's steering gear frequently executes control commands, which leads to mechanical wear and increased energy consumption of the actuator. The event triggering mechanism is applied to the control algorithm, and the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are obtained, thereby achieving the purpose of reducing wear and reducing energy consumption. Based on reinforcement learning and adaptive dynamic programming methods, the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system are obtained by using Actor-Critic; and finally the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are obtained, so as to realize the event-triggered adaptive dynamic programming heading tracking control of the unmanned ship. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0014] Figure 1 It is a flow chart of the unmanned ship heading tracking event triggering optimization control method based on adaptive dynamic programming of the present invention;
[0015] Figure 2 This is a simulation diagram of the ship heading tracking error result provided in this embodiment;
[0016] Figure 3 It is a trajectory simulation diagram of the ship heading and the reference heading provided in this embodiment;
[0017] Figure 4 This is a simulation diagram of the convergence result of the fuzzy logic value function weights provided in this embodiment;
[0018] Figure 5 This is a simulation diagram of the convergence result of the fuzzy logic strategy function provided in this embodiment;
[0019] Figure 6 This is a control input simulation diagram provided as an example in this embodiment;
[0020] Figure 7 This is a control input simulation diagram under event triggering provided in the example of this embodiment;
[0021] Figure 8 This is a diagram illustrating the number of trigger events and the interval time provided in this embodiment. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] This embodiment provides an unmanned ship heading tracking event triggering control method based on adaptive dynamic programming, such as Figure 1 As shown, the following steps are included:
[0024] S1: Establishing a mathematical model for the heading control of an unmanned ship under uncertain interference conditions at sea, such as wind, waves, and currents, so as to convert the mathematical model for heading control into a second-order state space equation, and obtaining a mathematical model for heading tracking control, which is used as a continuous-time nonlinear dynamic system of this embodiment, specifically includes the following steps:
[0025] Preferably, the sea condition information around the unmanned ship at sea is taken into consideration to obtain a mathematical model for the unmanned ship's heading control with uncertain interference items at sea such as wind, waves, and currents. The expression of the mathematical model for the unmanned ship's heading control is:
[0026]
[0027] Where: φ represents the heading angle of the unmanned ship; represents the first-order derivative of φ, i.e., the angular velocity of the unmanned ship; represents the second-order derivative of φ, i.e., the angular acceleration of the unmanned ship; T1 represents the time constant of the ship's maneuverability; represents a nonlinear function of the ship's bow angular velocity, where α, β are both nonlinear coefficients of the ship's bow angular velocity; K is the maneuverability index of the unmanned ship's rudder gain; δ is the control input of the unmanned ship's rudder angle and heading control system; δ w represents the equivalent rudder angle caused by uncertain environmental disturbance, δ w =α r +β r ,α r and β r Represent the wind pressure difference angle and flow pressure difference angle respectively;
[0028] Preferably, the unmanned ship heading control mathematical model is converted into a second-order state space equation and used as a heading tracking control mathematical model. The expression of the heading tracking control mathematical model is:
[0029]
[0030] in,
[0031]
[0032] x1=φ
[0033]
[0034]
[0035] Where: x(t) represents the state vector of the unmanned ship, where Ω represents R 2 is a subset of and is a compact set; R represents the set of all real numbers; f(x(t)) represents a differentiable function about x(t); g(x(t)) represents a differentiable bounded function about x(t); u(t) represents the control input vector; represents uncertain environmental interference, where d * Indicates uncertainty about environmental interference The upper bound of x1 represents the first component of the state vector, the unmanned ship heading angle φ; x2 represents the second component of the state vector, the unmanned ship heading angular velocity T represents transpose; t represents the time variable;
[0036] S2: According to the heading tracking control mathematical model, based on the existing reinforcement learning and adaptive dynamic programming methods, the optimal feedback controller of the unmanned ship heading is obtained;
[0037] Preferably, the unmanned ship heading optimal feedback controller is obtained as follows:
[0038] S21: define the tracking error of the unmanned ship heading, establish a dynamic system considering the tracking error of the unmanned ship heading and the reference signal, so as to obtain a dynamic nominal system about the error and the reference signal;
[0039] S211: Define the tracking error of the unmanned ship’s heading as follows:
[0040] e(t)=x(t)-r(t)
[0041] Where: r(t) is the heading tracking signal; e(t) is the tracking error of the unmanned ship's heading; x(t) represents the state vector of the unmanned ship;
[0042] According to the heading tracking control mathematical model, the derivative of the tracking error of the unmanned ship heading is obtained as follows:
[0043]
[0044] So, substituting x(t)=e(t)+r(t) into the above formula, we get:
[0045]
[0046] S212: Define the augmented state vector ξ(t) = [e T (t),r T (t)] T ∈R 4 ,
[0047] where ξ(0)=[e0 T ,r0 T ] T ;
[0048] Where ξ(t) represents the augmented state vector; ξ(0) represents the initial value of the augmented state vector ξ(t); e0 represents the initial value of the tracking error e(t) of the unmanned ship’s heading; r0 represents the initial value of the heading tracking signal r(t); R 4 Represents the 4-dimensional Euclidean space of all real vectors; represents the first-order derivative of r(t);
[0049] Then, the dynamic system expression of the error and reference signals is as follows:
[0050]
[0051] in,
[0052]
[0053] f(e(t)+r(t))=f(x(t))
[0054]
[0055] g(e(t)+r(t))=g(x(t))
[0056]
[0057]
[0058] Where: is the first-order derivative of ξ(t); F(ξ(t)) represents the nonlinear system matrix about ξ(t); G(ξ(t)) represents the bounded system matrix about ξ(t); ΔF(ξ(t)) represents the uncertainty term; ‖·‖ represents the second norm of the vector;
[0059] Specifically, in order to achieve the optimal tracking control of the unmanned ship heading tracking control system, a feedback controller u(ξ(t)) will be designed. Under the action of this controller, the closed-loop system is asymptotically stable for the uncertainty term ΔF(ξ(t)). Therefore, the optimal controller of its nominal system can be designed by considering a suitable cost function.
[0060] S213: Obtain the dynamic nominal system regarding the error and reference signal as follows:
[0061]
[0062] S22: In order to solve the heading tracking optimization control problem, the cost function is established as follows:
[0063]
[0064] in,
[0065]
[0066] Where: L(ξ(t)) represents the cost function; Q(ξ(τ)) and u(τ) represent the basic parts of the utility function; τ represents the integral variable; u(τ) represents the integrand related to the control input; ξ(τ) represents the integrand related to the augmented state; and All are positive definite matrices; t represents the time variable;
[0067] S23: According to the dynamic nominal system and cost function about the error and the reference signal, the Hamilton-Jaccobin-Bellman (HJB) equation is obtained:
[0068]
[0069] Where: H(ξ(t),u(ξ(t))) represents the Hamiltonian function; u(ξ(t)) represents the feedback controller; ξ(t) represents the augmented state vector; Q(ξ(t)) represents the semi-positive penalty function; It means to find the partial derivative of the function;
[0070] S24: The controller that minimizes the cost function L(ξ) is the optimal controller. According to the Hamilton-Jaccobin-Bellman (HJB) equation, the control input u(ξ(t)) is derived to obtain the optimal feedback controller for the unmanned ship heading;
[0071] Preferably, the unmanned ship heading optimal feedback controller is obtained as follows:
[0072] Specifically, the minimum value of the cost function L(ξ) is the solution of the Hamilton-Jacobi-Bellman equation (i.e., the HJB equation), that is, when the cost function L(ξ) takes the minimum value When
[0073]
[0074] Where: u * (ξ(t)) represents the optimal feedback controller for the unmanned ship’s heading; L * (ξ(t)) is the optimal cost function, that is, the minimum value of the cost function;
[0075] S3: Based on the Actor-Critic algorithm (execution-evaluation system), the fuzzy logic system approximation value function and the strategy function, and according to the unmanned ship heading optimal feedback controller, the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system are obtained to obtain the weights of the Critic fuzzy logic system and the weights of the Actor fuzzy logic system;
[0076] S31: using an approximation method of a fuzzy logic system function to obtain an optimal cost function expressed by fuzzy logic;
[0077] Specifically, the approximation method of the fuzzy logic system function can be used to express the optimal cost function L * (ξ(t))
[0078]
[0079] Where: ω is the adaptive evaluation weight of fuzzy logic; is the fuzzy basis function of the fuzzy logic system; ε(ξ(t)) is the unknown estimation error; L * ′(ξ(t)) represents the optimal cost function expressed by fuzzy logic;
[0080] S32: according to the optimal cost function expressed by fuzzy logic and the unmanned ship heading optimal feedback controller, obtaining an optimal feedback controller estimated by fuzzy logic;
[0081] Therefore, substituting the optimal cost function in S31 into the optimal feedback controller in S24, the optimal feedback controller expressed by fuzzy logic can be obtained.
[0082]
[0083] Where: u * ′(ξ(t)) represents the optimal feedback controller expressed by fuzzy logic;
[0084] S33: Based on the Actor-Critic algorithm, obtain the estimated value of the optimal cost function and the estimated value of the strategy function expressed by fuzzy logic;
[0085] Specifically, we use the dual network architecture of the existing Actor-critic algorithm (execution-evaluation system) to approximate the optimal cost function L expressed in fuzzy logic. * ′(ξ(t)) and the optimal feedback controller u for the unmanned ship heading * (ξ(t)), we can get
[0086]
[0087]
[0088] Where: Represents the estimated value of the optimal cost function expressed by fuzzy logic; is the weight of the Critic fuzzy logic system; represents the estimated value of the policy function, that is, the estimated value of the optimal feedback controller estimated by fuzzy logic; is the weight of the Actor fuzzy logic system;
[0089] S34: acquiring a Hamiltonian function expressed by fuzzy logic according to the optimal cost function expressed by fuzzy logic and the optimal feedback controller estimated by fuzzy logic;
[0090]
[0091] For ease of analysis, define a non-negative matrix
[0092]
[0093] Where: H(ξ(t),u * ′(ξ(t)),L * ′(ξ(t))) represents the Hamiltonian function expressed by fuzzy logic; A(ξ(t)) represents the defined non-negative matrix; ε HJB Residual represents the error in approximating the function;
[0094]
[0095] S35: Obtaining an estimated value of a Hamiltonian function expressed by fuzzy logic according to the estimated value of the optimal cost function and the estimated value of the strategy function;
[0096]
[0097] Where: represents the estimated value of the Hamiltonian function expressed by fuzzy logic;
[0098] S36: Obtain the weight update law of two networks of the fuzzy logic system according to the Hamiltonian function expressed by fuzzy logic and the estimated value of the Hamiltonian function expressed by fuzzy logic.
[0099] The acquisition method is as follows:
[0100] Considering the dynamic nominal system and the cost function L(ξ(t)), we define a continuously differentiable Lyapunov function L s (ξ(t)), where L s (ξ(t)) satisfies:
[0101]
[0102] And, there exists a 4-dimensional positive definite matrix Make
[0103]
[0104] Next, select As the objective function, According to the Hamiltonian function expressed by fuzzy logic and the estimated value of the Hamiltonian function expressed by fuzzy logic, the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system are constructed by using the gradient descent method.
[0105] E c Respectively and Taking partial derivatives we get:
[0106]
[0107]
[0108] In order to strengthen the learning process of the weight update law, an additional term is added to the construction of the weight update law Therefore, the weight update law is constructed as:
[0109]
[0110]
[0111] Where: α a is the basic learning rate of the weights of the Actor fuzzy logic system; α c is the basic learning rate of the weights of the Critic fuzzy logic system, α s is the adjustment rate of the additional item; L s (ξ(t)) represents a continuously differentiable Lyapunov function; Represents the weight update law of the Actor fuzzy logic system; Represents the weight update law of the Critic fuzzy logic system;
[0112] S4: Define the event trigger mechanism, and obtain the optimal feedback controller of the unmanned ship heading under event triggering according to the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system, and realize the event-triggered adaptive dynamic programming heading tracking control of the unmanned ship.
[0113] The following steps are involved:
[0114] S41: Define the event triggering mechanism as follows:
[0115]
[0116] Where: Indicates the control strategy when the event is triggered; u * (ξ(td)) represents t dThe optimal feedback controller at time t d represents the controller update time, d represents the serial number of the controller update time, d∈Z + , Z + represents a set of positive integers; inf{} represents the infimum; Γ(t) represents the event trigger error, Δ and M are design parameters, and they are both positive constants, where 0<Δ<1, M>0; t1 represents the initial event trigger time;
[0117] S42: When time t∈[t d ,t d+1 ], the control strategy is u * (ξ(t d )), and when the above mechanism is triggered, that is, when t = t d+1 , the control strategy is updated and marked as Therefore, there exist two continuously time-varying parameters ρ1(t) and ρ2(t) such that Among them, |ρ1(t)|≤1, |ρ2(t)|≤1. Then the optimal feedback controller of the unmanned ship heading under event triggering is obtained;
[0118]
[0119] S43: According to the unmanned ship heading optimal feedback controller triggered by the event, the optimal feedback controller expressed by fuzzy logic under event triggering is obtained:
[0120]
[0121] In the above formula, for the convenience of expression, we define the intermediate calculation parameters ρ = 1 + ρ1(t)Δ and
[0122] Combined with S23, the optimal feedback controller constructed by fuzzy logic under event triggering is obtained: The Hamiltonian function is expressed as,
[0123]
[0124] Among them, the residual of the function approximation error under event triggering is expressed as
[0125]
[0126] S44: Obtaining the optimal feedback controller described by fuzzy logic under event triggering An estimated value of
[0127]
[0128] in: represents the estimated value of the optimal feedback controller described by fuzzy logic under event triggering;
[0129] S45: According to the estimated value of the optimal cost function expressed by fuzzy logic Optimal feedback controller estimated by fuzzy logic under event triggering The estimated value of the Hamiltonian function described by fuzzy logic under event triggering can be obtained:
[0130]
[0131] S46: The method of constructing the weight update law is the same as that in S36. Select As the target function under event triggering,
[0132] According to the estimated value of the Hamiltonian function expressed by fuzzy logic under event triggering, the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are constructed by using the gradient descent method:
[0133] E ce Respectively and Taking partial derivatives we get:
[0134]
[0135]
[0136] Where: E ce represents the gradient function triggered by the event; e ce Indicates intermediate calculation parameters;
[0137] In order to strengthen the learning process of the weight update law, an additional term is added to the construction of the weight update law Therefore, the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are constructed as follows:
[0138]
[0139]
[0140] Where: α a is the basic learning rate of the weights of the Actor fuzzy logic system; α c is the basic learning rate of the weights of the Critic fuzzy logic system, α s is the adjustment rate of the additional item; L s (ξ(t)) represents a continuously differentiable Lyapunov function; E cerepresents the gradient function triggered by the event; e ce Indicates intermediate calculation parameters; Represents the weight update law of the Actor fuzzy logic system triggered by events; Represents the weight update law of the Critic fuzzy logic system triggered by events.
[0141] Specifically, this embodiment also proves through step S5 that S6 is used to prove the stability of the closed-loop system under the action of The stability of the closed-loop system under the action of and the proof that the event triggering mechanism considered in this embodiment does not have the Zeno phenomenon. Specifically, in order to more smoothly prove the stability, the following assumptions are made here:
[0142] Assumption 2: ‖G(.)‖≤λ g , ‖ξ * ‖≤λ ξ Where: g , λ ε and λ ξ are all positive constants. G(.) represents the function matrix; Indicates deviation;
[0143] Preferably, the method further comprises step S5: according to Lyapunov theory, proving that the optimal heading controller u of the unmanned ship * (ξ), the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system can ensure that all signals in the closed-loop system of unmanned ship heading tracking are ultimately bounded: the method is as follows:
[0144] S51: According to the Hamiltonian function expressed by fuzzy logic, after deformation, we obtain
[0145]
[0146] S52: The two weight update laws in S37 and get
[0147]
[0148]
[0149]
[0150] Where: Represents the weight error of the Actor fuzzy logic system The derivative of Represents the derivative of the weight error of the Critic fuzzy logic system;
[0151] S53: Select the following Lyapunov function
[0152]
[0153] in, Represents the weight error of the Actor fuzzy logic system; Represents the weight error of the Critic fuzzy logic system;
[0154] S54: Its time derivative is
[0155]
[0156] Where: Represents the estimated value of the optimal controller constructed by the Actor fuzzy logic system Dynamic systems represented;
[0157] S55: The estimated value of the optimal controller constructed by the Actor fuzzy logic system in S33 Substituting into the dynamic nominal system, we can get
[0158]
[0159] Deformation
[0160]
[0161] S56: Substitute the above formula into In the
[0162]
[0163] S57: Then we can get
[0164]
[0165] Where: represents the optimal controller u constructed by the fuzzy logic system * ′(ξ(t)) represents a dynamic system; where
[0166] S58: Due to and is bounded, so It also has an upper bound, combined with Assumption 2, we can get
[0167]
[0168] Where: λ1,λ2,λ3 are all positive constants.
[0169] S59: can further obtain
[0170]
[0171] Where: represents a positive constant, where λ g represents the upper bound of the function matrix mentioned in Assumption 2; ε represents the upper bound of the deviation; λ min Represents the smallest eigenvalue of a matrix; represents a 4-dimensional positive definite matrix; represents a 4-dimensional positive definite matrix The minimum eigenvalue of .
[0172] Therefore, if:
[0173] or or
[0174] So That is to say, the closed-loop system is in the controller Under the action of , all signals can be made eventually uniformly bounded.
[0175] Preferably, it also includes S6: using Lyapunov theory to prove the optimal feedback controller triggered by events expressed by fuzzy logic and proves that the event triggering mechanism considered in this embodiment does not have the Zeno phenomenon.
[0176] S61: Similar to S53, select the Lyapunov function triggered by the following event
[0177]
[0178] Obviously, the difference between the weight update laws under the two control strategies lies only in the positive constants ρ and ρ 2 , so the proof steps are the same as in S5, and the optimal feedback controller of the closed-loop system triggered by events expressed by fuzzy logic can also be proved Under the action of , all signals can be made uniformly bounded in the end.
[0179] S62: Next, it is proved that the event triggering mechanism considered in this embodiment does not have the Zeno phenomenon.
[0180] Consider the derivative of the event-triggered error function in S41
[0181]
[0182] in, It means to find the derivative of the function with respect to t.
[0183] From S61, we can know that the optimal feedback controller of the closed-loop system under the event triggering expressed by fuzzy logic is Under the action of , all signals can be bounded in the end, so there exists a positive constant k such that Based on the event triggering mechanism in S41, we can get Γ(t d )=0 and Then the minimum trigger time interval t * =t d+1 -t d satisfy This ensures that the Zeno phenomenon will not occur.
[0184] This embodiment has the following beneficial effects:
[0185] 1. The unmanned ship heading tracking event-triggered control method based on adaptive dynamic programming provided in this embodiment takes into account the event-triggered control algorithm. By considering the problem of mechanical wear and increased energy consumption of the actuator caused by the frequent execution of control commands by the ship's rudder when the ship is sailing in severe sea conditions, the event trigger mechanism is applied to the control algorithm, thereby achieving the purpose of reducing wear and reducing energy consumption.
[0186] 2. This embodiment is based on reinforcement learning and adaptive dynamic programming methods, uses the Actor-Critic logic system to approximate the value function and the policy function to design the unmanned ship heading optimal feedback controller, constructs the fuzzy logic weight update law, and implements the unmanned ship event-triggered adaptive dynamic programming heading tracking control according to the heading controller and the fuzzy logic weight update law. By considering the event triggering mechanism, the event triggering conditions and the unmanned ship heading optimal feedback controller under the event triggering conditions are obtained to realize the unmanned ship heading tracking control considering the event triggering mechanism.
[0187] 3. This embodiment carried out a simulation experiment based on the Matlab platform, took the event trigger mechanism into consideration, and designed an optimized controller under the event trigger mechanism to achieve tracking of the unmanned ship's heading. After considering the event trigger mechanism, the system controller is more in line with navigation practice, further verifying the effectiveness and rationality of the unmanned ship heading tracking event trigger control method based on adaptive dynamic programming provided in this embodiment.
[0188] A specific embodiment of the present invention is as follows:
[0189] In order to verify the effectiveness of the unmanned ship heading tracking event triggering control method based on adaptive dynamic programming in this embodiment, this embodiment uses Matlab for computer simulation research. The parameters are set as follows:
[0190] The simulation object is the Dalian Maritime University's "Yukun" ship, the ship parameters are: the length between the two columns is 105m, the ship width is 18m, the rudder blade area is 11.46m 2 , full load speed 16.7kn, full load midship draft 5.2m, full load displacement 5735.5m 3 , square coefficient 0.5595. Adopt Norrbin motion model, take unmanned ship rudder gain maneuverability index K = 0.32, ship maneuverability time constant T1 = 62.38, take ship bow angular velocity nonlinear coefficient α = -100, β = -50. Set the marine environment interference parameters as: wind direction ψ wind =30°, wind level S=5, flow direction ψ current =30°, flow velocity v current =5kn. Set the ideal course of the unmanned ship to Choosing the parameter α a =0.001,α c =1,α s =10000, Δ=0.39,M=0.001.
[0191] in, and are all positive definite matrices; I represents the identity matrix; α a Represents the basic learning rate of the weights of the Actor fuzzy logic system; α c Represents the basic learning rate of the weights of the Critic fuzzy logic system; α s It represents the regulation rate of the additional item; Δ and M are both design parameters;
[0192] The simulation experiment results are shown in the figure: Figure 2 It is the trajectory curve of the tracking error. It can be seen that the tracking error is bounded and stable under the action of the designed controller. Figure 3 The tracking result of the ship heading angle to the reference signal shows that the designed controller has a good control effect on the ship and can achieve ship heading tracking within 10 seconds; Figure 4 and Figure 5 They are the convergence of the value function weights and the strategy function weights. According to the simulation results, the fuzzy logic weights all converge. Figure 6 is the trajectory diagram of the control input without considering the event trigger mechanism, Figure 7 : is a trajectory diagram of the control input considering the event trigger mechanism. It can be seen that under the same conditions, the event trigger controller of this embodiment is better than the ordinary controller. Figure 8 Describes the relevant event triggering time.
[0193] The simulation results show that the controller designed based on the unmanned ship heading tracking event-triggered optimization control method based on adaptive dynamic programming in this embodiment can well realize ship heading tracking, and the tracking error can quickly converge to a smaller residual set. After considering the event trigger mechanism, the control effect is more energy-efficient and efficient, and the system control input curve is more in line with navigation practice. This further verifies the effectiveness and rationality of the unmanned ship heading optimization control method considering the input dead zone proposed in this embodiment.
[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An unmanned ship heading tracking event triggering control method based on adaptive dynamic programming, characterized in that: The following steps are involved: S1: Establishing a mathematical model for the heading control of an unmanned ship considering the uncertain interference conditions faced by the unmanned ship at sea during navigation, so as to convert the mathematical model for heading control into a second-order state space equation and obtain a mathematical model for heading tracking control; S2: according to the tracking error of the unmanned ship heading, a dynamic nominal system about the error and the reference signal is obtained; according to the heading tracking control mathematical model, an optimal feedback controller of the unmanned ship heading is obtained; S3: Based on the Actor-Critic algorithm and the unmanned ship heading optimal feedback controller, a weight update law of the Actor fuzzy logic system and a weight update law of the Critic fuzzy logic system are obtained; S4: According to the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system, the optimal feedback controller of the unmanned ship heading under event triggering, the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are obtained to realize the event-triggered adaptive dynamic programming heading tracking control of the unmanned ship.
2. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 1 is characterized in that: The methods used to obtain the optimal feedback controller of the unmanned ship heading under event triggering, the weight update law of the Actor fuzzy logic system under event triggering, and the weight update law of the Critic fuzzy logic system under event triggering are as follows: S41: Define the event triggering mechanism as follows: Where: Indicates the control strategy when the event is triggered; u * (ξ(t d )) means t d The optimal feedback controller at time t d represents the controller update time, d represents the serial number of the controller update time, d∈Z + , Z + represents a set of positive integers; inf{} represents the infimum; Γ(t) represents the event trigger error, Δ and M are both design parameters, where 0<Δ<1, M>0; t1 represents the initial event trigger time; t represents the time; R represents a set of real numbers; S42: Obtaining the optimal feedback controller of the unmanned ship heading under event triggering; Where: u * (ξ(t)) represents the optimal feedback controller of the unmanned ship; ρ2(t) represents the continuous time-varying parameter; ρ1(t) represents the continuous time-varying design parameter; S43: According to the unmanned ship heading optimal feedback controller triggered by the event, the optimal feedback controller expressed by fuzzy logic under event triggering is obtained: ρ=1+ρ1(t)Δ Where: ρ1(t) represents the continuous time-varying design parameter; ρ2(t) represents the continuous time-varying parameter; ρ is the intermediate calculation parameter; represents the optimal feedback controller triggered by events expressed by fuzzy logic; represents a positive definite matrix; G(ξ(t)) represents a bounded system matrix with respect to ξ(t); represents the fuzzy basis function of the fuzzy logic system; ω represents the weight of the fuzzy logic; represents the partial derivative of the function; ε(ξ(t)) represents the unknown estimated deviation; ρ represents the intermediate calculation parameter; ε e (ξ(t)) represents the unknown estimation deviation under event triggering; Then we get the Hamiltonian function expressed by the optimal feedback controller expressed by fuzzy logic under event triggering: Where: L * ′(ξ(t)) represents the optimal cost function expressed by fuzzy logic; Q(ξ(t)) represents the semi-positive penalty function; F(ξ(t)) represents the nonlinear system matrix about ξ(t); A(ξ(t)) represents the defined non-negative matrix; ε HJBe Residual of the function approximation error under event triggering; Among them, the residual expression of the function approximation error under event triggering is: S44: obtaining an estimated value of an optimal feedback controller expressed by fuzzy logic under event triggering; in: represents the estimated value of the optimal feedback controller described by fuzzy logic under event triggering; S45: Estimated value of the optimal cost function expressed by fuzzy logic The estimated value of the optimal feedback controller described by fuzzy logic under event triggering is obtained by: Where: Represents the weight estimate of the Actor fuzzy logic system; Represents the weight estimate of the Critic fuzzy logic system; S46: According to the estimated value of the Hamiltonian function expressed by the fuzzy logic under event triggering, the weight update law of the Actor fuzzy logic system under event triggering and the weight update law of the Critic fuzzy logic system under event triggering are obtained: Where: α a is the basic learning rate of the weights of the Actor fuzzy logic system; α c is the basic learning rate of the weights of the Critic fuzzy logic system, α s is the adjustment rate of the additional item; L s (ξ(t)) represents a continuously differentiable Lyapunov function; E ce represents the target function under event triggering; e ce Indicates intermediate calculation parameters; Represents the weight update law of the Actor fuzzy logic system triggered by events; Represents the weight update law of the Critic fuzzy logic system triggered by events.
3. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 1, characterized in that: The mathematical model of the unmanned ship heading control is expressed as follows: Where: φ represents the heading angle of the unmanned ship; represents the first-order derivative of φ, i.e., the angular velocity of the unmanned ship; represents the second-order derivative of φ, i.e., the angular acceleration of the unmanned ship; T1 represents the time constant of the ship's maneuverability; represents a nonlinear function of the ship's bow angular velocity, where All represent the nonlinear coefficient of the ship's bow angular velocity; K represents the unmanned ship rudder gain maneuverability index; δ represents the control input of the unmanned ship rudder angle and heading control system; δ w represents the equivalent rudder angle caused by uncertain environmental disturbance, δ w =α r +β r ,α r and β r They represent the wind pressure difference angle and flow pressure difference angle respectively.
4. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 1 is characterized in that: The mathematical model of heading tracking control is expressed as follows: in, x1=φ Where: x(t) represents the state vector of the unmanned ship, where Ω represents R 2 is a subset of and is a compact set; R represents the set of all real numbers; f(x(t)) represents a differentiable function about x(t); g(x(t)) represents a differentiable bounded function about x(t); u(t) represents the control input vector; represents uncertain environmental interference, where d * Indicates uncertainty about environmental interference The upper bound of x1 represents the first component of the state vector, the unmanned ship heading angle φ; x2 represents the second component of the state vector, the unmanned ship heading angular velocity T represents transpose; t represents the time variable.
5. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 1, characterized in that: In S2, the unmanned ship heading optimal feedback controller is obtained as follows: S21: define the tracking error of the unmanned ship heading, establish a dynamic system considering the tracking error of the unmanned ship heading and the reference signal, so as to obtain a dynamic nominal system about the error and the reference signal; S22: Establish the cost function as follows: Where: L(ξ(t)) represents the cost function; Q(ξ(τ)) represents the basic part of the utility function; τ represents the integral variable; u(τ) represents the integrand related to the control input; ξ(τ) represents the integrand related to the augmented state; and All are positive definite matrices; t represents the time variable; S23: According to the dynamic nominal system and cost function about the error and the reference signal, obtain the Hamilton-Jacobi-Bellman equation: Where: H(ξ(t),u(ξ(t))) represents the Hamiltonian function; u(ξ(t)) represents the feedback controller; ξ(t) represents the augmented state vector; Q(ξ(t)) represents the semi-positive penalty function; It means to find the partial derivative of the function; S24: According to the Hamilton-Jacobi-Bellman equation, an optimal feedback controller for the unmanned ship heading is obtained.
6. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 5 is characterized in that: In S21, a method for obtaining a dynamic nominal system regarding an error and a reference signal is as follows: S211: Define the tracking error of the unmanned ship’s heading as follows: e(t)=x(t)-r(t) Where: r(t) is the heading tracking signal; e(t) is the tracking error of the unmanned ship's heading; x(t) represents the state vector of the unmanned ship; According to the heading tracking control mathematical model, the derivative of the tracking error of the unmanned ship heading is obtained as follows: So, we get: S212: Define the augmented state vector ξ(t) = [e T (t),r T (t)] T ∈R 4 , where ξ(0)=[e0 T ,r0 T ] T ; Where ξ(t) represents the augmented state vector; ξ(0) represents the initial value of the augmented state vector ξ(t); e0 represents the initial value of the tracking error e(t) of the unmanned ship’s heading; r0 represents the initial value of the heading tracking signal r(t); R 4 Represents the 4-dimensional Euclidean space of all real vectors; represents the first-order derivative of r(t); but, in, f(e(t)+r(t))=f(x(t)) g(e(t)+r(t))=g(x(t)) Where: is the first-order derivative of ξ(t); F(ξ(t)) represents the nonlinear system matrix about ξ(t); G(ξ(t)) represents the bounded system matrix about ξ(t); ΔF(ξ(t)) represents the uncertainty term; ‖·‖ represents the second norm of the vector; S213: Obtain the dynamic nominal system regarding the error and reference signal as follows:
7. The unmanned ship heading tracking event triggering control method based on adaptive dynamic programming according to claim 5 is characterized in that: In S24, the unmanned ship heading optimal feedback controller is obtained as follows: Where: u * (ξ(t)) represents the optimal feedback controller for the unmanned ship’s heading; L * (ξ(t)) is the optimal cost function, that is, the minimum value of the cost function.
8. The method for controlling the heading tracking of an unmanned ship based on adaptive dynamic programming according to claim 1 is characterized in that: The method used to obtain the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system is as follows: S31: using an approximation method of a fuzzy logic system function to obtain an optimal cost function expressed by fuzzy logic; Where: ω is the weight of fuzzy logic; is the fuzzy basis function of the fuzzy logic system; ε(ξ(t)) is the unknown estimation error; L * ′(ξ(t)) represents the optimal cost function expressed by fuzzy logic; S32: according to the optimal cost function expressed by fuzzy logic and the unmanned ship heading optimal feedback controller, obtaining an optimal feedback controller expressed by fuzzy logic; Where: u * ′(ξ(t)) represents the optimal feedback controller expressed by fuzzy logic; S33: Based on the Actor-Critic algorithm, obtain the estimated value of the optimal cost function and the estimated value of the strategy function expressed by fuzzy logic; Where: Represents the estimated value of the optimal cost function expressed by fuzzy logic; It is the weight estimation of the Critic fuzzy logic system; represents the estimated value of the policy function, that is, the estimated value of the optimal feedback controller expressed by fuzzy logic; It is the weight estimation of Actor fuzzy logic system; S34: acquiring a Hamiltonian function expressed by fuzzy logic according to the optimal cost function expressed by fuzzy logic and the optimal feedback controller expressed by fuzzy logic; Where: H(ξ(t),u * ′(ξ(t)),L * ′(ξ(t))) represents the Hamiltonian function expressed by fuzzy logic; A(ξ(t)) represents the defined non-negative matrix; ε HJB Residual represents the error in approximating the function; in, S35: Obtaining an estimated value of a Hamiltonian function expressed by fuzzy logic according to the estimated value of the optimal cost function and the estimated value of the strategy function; Where: represents the estimated value of the Hamiltonian function expressed by fuzzy logic; S36: According to the Hamiltonian function expressed by fuzzy logic and the estimated value of the Hamiltonian function expressed by fuzzy logic, a weight update law of the fuzzy logic system is obtained: Where: α a is the basic learning rate of the weights of the Actor fuzzy logic system; α c is the basic learning rate of the weights of the Critic fuzzy logic system, α s is the adjustment rate of the additional item; L s (ξ(t)) represents a continuously differentiable Lyapunov function; Represents the weight update law of the Actor fuzzy logic system; Represents the weight update law of the Critic fuzzy logic system.
9. The method for controlling the heading tracking event of an unmanned ship based on adaptive dynamic programming according to claim 1, characterized in that: It also includes S5: According to Lyapunov theory, it is proved that according to the unmanned ship optimal heading controller, the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system, it can ensure that all signals in the closed-loop system of the unmanned ship heading tracking are ultimately bounded, and the method is as follows: S51: According to the Hamiltonian function expressed by fuzzy logic, we get S52: According to the weight update law of the Actor fuzzy logic system and the weight update law of the Critic fuzzy logic system, and get Where: Represents the weight error of the Actor fuzzy logic system The derivative of Represents the derivative of the weight error of the Critic fuzzy logic system; S53: Select the following Lyapunov function in, Represents the weight error of the Actor fuzzy logic system; Represents the weight error of the Critic fuzzy logic system; Taking the derivative of S(t), we get Where: Represents the estimated value of the optimal controller constructed by the Actor fuzzy logic system Dynamic systems represented; S55: Estimated value of the optimal feedback controller for the unmanned ship heading Substituting into the dynamic nominal system, we can get Deformation Further can get then, Where: represents the optimal controller u constructed by the fuzzy logic system * ′(ξ(t)) represents a dynamic system; where because and is bounded, so It also has an upper bound, so we can get: Where: λ1,λ2,λ3 are all positive constants; Furthermore, Where: represents a positive constant, where λ g represents the upper bound of the function matrix; λ ε represents the upper bound of the deviation; λ min represents the smallest eigenvalue of a matrix; represents a 4-dimensional positive definite matrix; Represents a 4-dimensional positive definite matrix The minimum eigenvalue of ; Therefore, if: or or but That is to say, the closed-loop system is in the controller Under the action of , all signals can be made eventually uniformly bounded.
