Event-triggered optimal visual range formation control method and system for underactuated unmanned surface vehicle under unknown disturbance

Abstract

The application relates to an event-triggered optimal line-of-sight (LOS) formation control method and system for an underactuated unmanned surface vehicle (USV) under unknown disturbance, and relates to the technical field of USV control. The application aims to solve the technical problem of realizing event-triggered optimal control while considering actuator saturation by establishing an RL-driven LOS guidance scheme, so as to achieve optimal state regulation and tracking control. The technical points are as follows: the application establishes an RL-driven LOS guidance scheme, decouples the trigger decision and the actuator allocation, and introduces a new event-triggered mechanism. The overall structure of the ETO-LOS control scheme proposed by the application is as follows: firstly, the expected trajectory and external disturbance are processed in the LOS guidance layer to generate a virtual control signal. Then, optimal control is realized through a DSC and a two-stage IAC network, and then an event-triggered mechanism is constructed in the actuator layer. The ETO-LOS scheme proposed by the application realizes stable learning and closed-loop performance while significantly reducing the control update frequency.

Description

Technical Field

[0001] This invention relates to the field of unmanned surface vessel (USV) control technology, specifically to an event-triggered optimal line-of-sight formation control method and system for under-actuated USVs under unknown interference. Background Technology

[0002] In recent years, unmanned surface vessels (USVs) have become a core research focus in the field of control. Compared to a single USV, multiple USVs often exhibit superior performance when operating in coordinated formations, such as in environmental monitoring. [1] Resource exploration [2] and coastal patrol [3] Existing unmanned surface vessel (USV) formation control methods can generally be classified into three categories based on the type of reference signal: trajectory tracking control. [4][5] Path following control [6][7] and target following control [8][9] Among them, path following control, which relies only on geometric guidance information and does not require time-parameterized trajectory, has superior engineering practicality.

[0003] Existing patent literature concerning multi-UAV cooperative formation control includes: CN112835368A, which discloses a method and system for multi-UAV cooperative formation control, relating to the field of marine intelligent UAV cooperative operations. This includes determining the formation shape generation model and formation shape maintenance model for each UAV based on the current task requirements of the UAV team; it solves the problems of poor real-time performance, instability, and low efficiency in existing multi-UAV cooperative formation systems. CN121349100A discloses a disturbance-resistant adaptive sliding mode control method for multi-UAV cooperative formation. This method aims to solve problems such as abrupt changes in initial control input, uncertain error convergence time, and control chattering in existing formation control. It includes: first, estimating the state of the leader UAV through a distributed state observer; then, continuously monitoring the observer's estimation error; when the error is less than a preset threshold, determining that the observer has converged and activating a preset time error correction mechanism; this mechanism converts the original tracking error into a corrected tracking error; finally, the adaptive dynamic sliding mode controller calculates the control input based on the corrected error. By binding the controller startup to the observer convergence state and through the dynamic correlation design of the adaptive rate, the tracking error is ensured to converge within a preset time, which significantly improves the robustness and performance of formation control.

[0004] As a key control strategy, cooperative path tracking aims to guide the system to track virtual targets on a path. In this context, model linearization is used as a basis for...

[10]

[11] Vector fields

[12]

[13] and line of sight

[14]

[15] The method has become the mainstream approach, among which the LOS guidance scheme is widely used due to its intuitiveness and low computational resource requirements. LOS-based path tracking methods typically define a forward look-ahead distance in the tangential direction of the path and generate the desired heading angle based on the lateral tracking error, including the original LOS guidance scheme.

[16]

[17] Adaptive LOS guidance scheme

[18]

[19] Integral LOS guidance scheme

[20]

[21] To enhance robustness to model uncertainties and environmental disturbances, various observer-based strategies have been developed to estimate lumped disturbances and unmeasured states, such as extended state observers.

[22] and high-gain observers

[23] A joint uncertainty and disturbance estimator is then developed. Subsequently, a neural network control strategy is implemented to approximate the unknown dynamics and / or control effects of the unmanned underwater vehicle, thereby promoting various neural network learning-based methods.

[24]

[25] However, the design goals of the above methods are still focused on ensuring the convergence and boundedness of the tracking error, with less attention paid to the optimality of the control strategy.

[0005] By learning solutions to approximate Hamilton-Jacobi-Bellman (HJB) equations, optimal control problems can be solved without incurring the burden of high-dimensional computation.

[26] Therefore, reinforcement learning (RL) methods for optimal control have attracted considerable research interest in recent years.

[27]

[28]

[29] Based on the actor-critic architecture, Vamvoudakis, etc.

[30] A policy iteration framework is proposed to obtain continuous-time optimal solutions with an infinite viewpoint performance metric. Furthermore, Wen et al.

[31] An enhanced backstepping-based strategy was also introduced to design the optimal tracking controller for nonlinear systems. (Wang et al.)

[32] RL is combined with finite-time observers for lumped uncertainty compensation. To enhance robustness to worst-case disturbances, the optimal control problem is also described as a zero-sum game, using an identifier-actor-critic (IAC) RL framework to solve for saddle point solutions.

[33] Although the above optimal control strategy focuses on minimizing the mathematical cost function, further optimization layers can be considered at the physical level by reducing the operating cost and wear of the actuators.

[0006] Furthermore, considering the significant consumption of communication bandwidth and airborne energy by traditional control strategies, the event-triggered method only performs calculations when the system state meets the triggering conditions, thus providing a more resource-efficient solution.

[34] This principle has been effectively applied in USV environments, where event-triggered mechanisms are integrated with various observers to handle unknown velocities and disturbances.

[35]

[36] Meanwhile, in pursuit of optimality, event-triggered programming has been combined with adaptive dynamic programming (ADP) to address issues such as optimal state regulation.

[37] Tracking and control

[38] and systems with matching uncertainties

[39] Robust control and other issues remain. However, simultaneously considering actuator saturation within such an event-triggered optimal control framework is still an area that requires further research.

[0007] Inspired by the above analysis, the technical problem to be solved by this invention is how to establish an RL-driven LOS guidance scheme to achieve optimal control of event triggering. Summary of the Invention

[0008] The technical problem to be solved by this invention is:

[0009] The purpose of this invention is to provide an event-triggered optimal line-of-sight formation control method and system for under-actuated unmanned surface vessels under unknown interference. By establishing an RL-driven LOS guidance scheme, optimal control triggered by events is achieved, thereby achieving optimal state adjustment and tracking control.

[0010] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0011] An event-triggered optimal line-of-sight formation control method for underactuated unmanned surface vessels (USVs) under unknown interference is proposed. The method involves: firstly, obtaining the desired trajectory of multiple USVs in a cooperative formation through a LOS guidance layer, generating a virtual control signal, and achieving optimal control through dynamic surface control (DSC) and a two-level inter-actuated control (IAC) network; then, constructing an event-triggered mechanism in the actuators; embedding optimal strategy learning simultaneously into both the LOS guidance layer and the underactuated dynamics layer; utilizing the two-level IAC network (architecture) to compensate for model uncertainties and suppress external interference in real time; designing an event-triggered mechanism (ETM) based on the overall control vector to eliminate independent monitoring and triggering of thrust and torque channels; the ETM operates on the norm of the entire control vector, avoiding complex actuator allocation problems between propeller speed and rudder angle at the actuator level, and decoupling triggering decisions from control allocation logic; incorporating an inertial matrix norm normalization term into the reference signal design of the ETM to constrain the amplitude of the control input, which is a two-dimensional vector composed of forces and torques generated by the servo motor and propeller, ensuring the physical feasibility of generating the control input, and ultimately achieving optimal line-of-sight formation control for USVs.

[0012] The present invention has the following beneficial technical effects:

[0013] This invention establishes an RL-driven LOS guidance scheme, decoupling triggering decisions from actuator allocation and introducing a novel event-triggered mechanism. The overall structure of the proposed ETO-LOS control scheme is as follows: First, the desired trajectory of the formation follower is generated at the LOS guidance layer, producing a virtual control signal. Then, optimal control is achieved through Dynamic Surface Control (DSC) and a two-level IAC network. Finally, an event-triggered mechanism (ETM) is constructed at the actuator level. The main contributions of this invention are summarized below:

[0014] 1. A unified control framework based on reinforcement learning is proposed, in which optimal policy learning is simultaneously embedded in both the LOS guiding layer and the underactuated dynamics layer. The two-level IAC architecture can compensate for model uncertainties in real time and suppress external disturbances.

[0015] 2. A novel event-triggered mechanism (ETM) based on the global control vector norm was designed, eliminating the need for independent monitoring and triggering of the thrust and torque channels. This design avoids the chaotic propeller and rudder distribution caused by independent triggering of the force and torque channels, making it more consistent with physical reality.

[0016] 3. To avoid the control signal amplification phenomenon commonly found in traditional relative threshold ETM, an inertial matrix norm normalization term was added to the reference signal design, which constrained the growth of the control amplitude and ensured the physical feasibility of generating the control input. Attached Figure Description

[0017] Figure 1 ETO-LOS control scheme diagram; Figure 2 This is a schematic diagram of the ETO-LOS formation structure; Figure 3 Photo of the "Dolphin 2" USV experimental platform; Figure 4 The trajectory tracking curves are shown under ETO-LOS and Backstepping-LOS. Figure 5 6 shows the state error diagram of follower 1 under ETO-LOS and Backstepping-LOS, and 6 shows the state tracking error diagram of follower 2 under ETO-LOS and Backstepping-LOS. Figure 7 Trajectory tracking curves for event-triggered (ET) and continuous-triggered (no ET) scenarios; Figure 8 The state error diagram for follower 1 under event-triggered (ET) and continuous-triggered (no ET) conditions is shown. Figure 9 The state error diagram for follower 2 under event-triggered (ET) and continuous-triggered (no ET) conditions; Figure 10 This is the weight graph of the first-level ACI network. Figure 11 The weight norm of the second-order ACI network; Figure 12 This is a norm graph of the value function. Figure 13Let be the norm of the weights in the Identifier network; Figure 14 This is a graph showing the time intervals between events for the event-triggered controller. Detailed Implementation

[0018] Combined with appendix Figure 1-14 The implementation of the event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference described in this invention is explained as follows:

[0019] 1. This invention establishes an RL-driven LOS guidance scheme, decoupling triggering decisions from actuator allocation and introducing a new event triggering mechanism. Figure 1 The overall structure of the proposed ETO-LOS control scheme is as follows: First, a virtual control signal is generated at the LOS guidance layer based on the desired trajectory. Then, optimal control is achieved through DSC and a two-level IAC network, and finally, an event-triggered mechanism is constructed in the actuator.

[0020] 2. The technical means and related principles of this invention are explained:

[0021] 2.1 Key Lemma

[0022] Lemma 1

[31] Consider the following system:

[0023] (1)

[0024] in, It is a state vector, if there exist two constants. and Make For all If the solution holds, then its solution is called a semi-globally consistent eventually bounded (SGUUB), where... It is compact.

[0025] Lemma 2

[40] If a continuous function satisfy ,in and yes

[0026] If two constants are given, then the following inequalities hold:

[0027] (2)

[0028] Lemma 3

[41] For any given nonlinear function Defined in compact set ,

[0029] There exists a neural network (NN). Such that for any given positive scalar ,

[0030] (3)

[0031] in, and The weight vector and basis function vector are respectively, and satisfy the following conditions: and ,in and It is a constant. Number of nodes. Basis functions. It can be selected as a Gaussian function, that is,

[0032] (4)

[0033] make and The center of the receptive field and The width.

[0034] 2.2 USV Dynamics

[0035] Consider a USV operating with three degrees of freedom (DOF), described by the following kinematics:

[0036] (5)

[0037] And dynamics:

[0038] (6)

[0039] in The position and heading angle in the Earth's fixed coordinate system. These represent the sway, roll, and bow angular velocities in the ship's coordinate system, respectively. It is the inertia matrix; These represent the Coriolis force and centripetal force matrix, and the damping matrix, respectively. It is a control vector composed of control force and torque; This indicates environmental interference.

[0040] By choosing an appropriate origin for the reference frame in the body coordinate system, the coupling terms in the inertial matrix can be made zero. Therefore, the dynamics in (6) can be rewritten as:

[0041] (7)

[0042] in, ,matrix , , , Generated by a second-order Markov process, where

[0043] (8)

[0044] Among them, matrix These are the reduced-order inertia matrix, the Coriolis centripetal matrix, and the damping matrix, respectively.

[0045] 2.3 Grouping based on LOS

[0046] Consider a set of USVs arranged in a leader-follower structure. Figure 2 The geometric relationship between follower i and leader j is given. The distance is measured via LOS. and relative angle Defined as:

[0047] (9)

[0048] (10)

[0049] in express The tangent value of is within the range of Since this paper controls the formation structure by measuring the relative distance and angle between the navigator and the followers, the formation output vector is represented as... The desired formation output vector is given by Given, among which .

[0050] 3. The RL-driven event-triggered LOS grouping control scheme proposed in this invention is as follows:

[0051] Before designing the controller, based on the relative dynamics between the leader and the follower derived in (5), we differentiate (9) and (10) and substitute them into (5) to obtain the following relative dynamic relationship:

[0052] (11)

[0053] in Given by the following formula:

[0054] (12)

[0055] (13)

[0056] The navigator's status here For followers Unavailable, therefore can It is considered as an unmodeled dynamic of the relative dynamics between the leader and the follower.

[0057] 3.1 Optimal Virtual Control Design Based on Reinforcement Learning

[0058] The goal of this section is to make Tracking expected formation structure Therefore, the tracking error vector is defined. Its time derivative satisfies:

[0059] (14)

[0060] in It is an intermediate control signal.

[0061] make For virtual control, For optimal virtual control, the optimal performance index function is defined as:

[0062] (15)

[0063] in It is a signal that includes the origin and the reference signal. The predefined compact set, Let this be the instantaneous cost function. Consider... For optimal virtual control Taking the time derivative of the optimal performance index function (15), the HJB equation associated with (14) is obtained as follows:

[0064] (16)

[0065] By solving The optimal virtual control is obtained as follows:

[0066] (17)

[0067] make In order to estimate the nonlinear terms in the relative dynamic equations, Divided into three parts:

[0068] (18)

[0069] in If the parameter matrix is ​​positive definite, then the optimal virtual control can be expressed as:

[0070] (19)

[0071] because and Two unknown but continuous elements are used, therefore a neural network is employed in compact sets. Approximating it from above, making

[0072] (20)

[0073] in and It is the ideal weight. and It is a basis function vector. and It is the approximation error.

[0074] By inserting (20) into (18) and (19), we can obtain

[0075] (twenty one)

[0076] (twenty two)

[0077] in .

[0078] However, optimal control (22) cannot be achieved because of the two ideal weights. and Unknown. To obtain an effective optimal virtual controller, reinforcement learning is used to implement usable optimized control by building an IAC architecture.

[0079] (twenty three)

[0080] (twenty four)

[0081] (25)

[0082] in and They are and The estimated value, , and These are the network weights for identifier, critic, and actor, respectively.

[0083] Weight vector of the identifier-actor-criitic network , and The update law is designed as follows:

[0084] (26)

[0085] (27)

[0086] (28)

[0087] in It is a design constant. , and These are design parameters that must be met.

[0088] (29)

[0089] Note 1: The following is a detailed derivation of the critic and actor update laws in (27) and (28).

[0090] because , The estimate can be expressed as:

[0091] (30)

[0092] Substituting (25) and (30) into (17), the approximation of the HJB equation is:

[0093] (31)

[0094] Define Bellman residual for:

[0095] (32)

[0096] Should meet ,like If a condition is true and has a unique solution, then it is equivalent to the following equation holding true:

[0097] (33)

[0098] To ensure (33), the following positive definite function is established:

[0099] (34)

[0100] in It is a trace operator, obviously. Equivalent to (33), as can be seen from the above equation. Therefore, for Differentiating along (27) and (28), we get:

[0101] (35)

[0102] Inequality (35) describes how using the renewal laws (27) and (28) can guarantee .

[0103] 3.2 Implementation process of Dynamic Surface Control (DSC)

[0104] This section stabilizes the yaw tracking error by designing a dynamic surface for the virtual control law derived in Section 3.1. .make Inversely decode the virtual control signal and :

[0105] (36)

[0106] Introducing two new state variables and and let and Through two time constants and To obtain a first-order filter and .

[0107] (37)

[0108] in , Then the error between the filtered signal and the true value, as well as the filtering error, are:

[0109] (38)

[0110] let along Differential yield The yaw control law is designed as follows:

[0111] (39)

[0112] Introducing new state variables ,let With a time constant To obtain a first-order filter :

[0113] (40)

[0114] Note 2: Residual Error Items It can be transformed using the mean value theorem. and The function.

[0115] make Then there is

[0116] (41)

[0117] According to the Mean Value Theorem for Multivariable Functions, for from arrive The change exists as a constant. , so that:

[0118] (42)

[0119] where constant Depends on function The upper bound of the gradient, since It contains only bounded trigonometric functions and state variables, and this upper bound always exists.

[0120] Based on (38), squaring both sides of (42) and applying Young's inequality, we can obtain:

[0121] (43)

[0122] 3.3 Event-Triggered Optimal Controller Design

[0123] By combining the relative threshold event triggering mechanism with the reinforcement learning-optimized backstepping control law, the final control law can be obtained in this subsection.

[0124] First, define the tracking error as... The desired reference signal is From equations (3), (38), and (40), it can be seen that... The derivative with respect to time is:

[0125] (44)

[0126] make ,but:

[0127] (45)

[0128] set up For optimal control, the optimal performance index function of the subsystem is defined as:

[0129] (46)

[0130] in Let be the cost function.

[0131] The HJB equation derived from (45) is:

[0132] (47)

[0133] Similar to Section 3.1, by solving Achieving optimal control:

[0134] (48)

[0135] make In order to estimate the nonlinear terms in the dynamic equations, The following split is performed:

[0136] (49)

[0137] in It is a positive definite parameter matrix. .because and For an unknown and continuous set, approximate it using an RBF neural network on a given compact set:

[0138] (52)

[0139] in and It is the ideal weight. and It is a basis function vector. and It is the approximation error.

[0140] Substituting (50) into (48) and (49) yields:

[0141] (51)

[0142] (52)

[0143] in .

[0144] Due to ideal weights and Since the two weights are unknown constants, an optimal controller cannot be used. To obtain effective optimal control, the following is constructed: Network and network:

[0145] (53)

[0146] (54)

[0147] (55)

[0148] in, , They are and The estimated value, , and They are , and The weights of the neural network.

[0149] As is consistent with Section 3.1, , and The neural network weight update rule is designed as follows:

[0150] (56)

[0151] (57)

[0152] (58)

[0153] in, It is a design constant. , and These are design parameters that must be met.

[0154] (59)

[0155] In order to achieve the system control signal When the amplitude is high, a relatively large measurement error can be tolerated, thus obtaining a longer update interval; while when the control signal tends to the equilibrium point with the system state, more precise control can be obtained through a smaller threshold, thereby improving system performance. We propose the following relative threshold control strategy:

[0156] To balance control performance and update frequency, a relative threshold strategy is introduced, where the trigger condition adapts to the control signal. The magnitude. The strategy is formulated as follows:

[0157] (60)

[0158] Then the triggering mechanism is defined as:

[0159] (61)

[0160] (62)

[0161] in, To account for measurement error, as well as For design parameters, The controller updates the time, that is, whenever (62) is triggered, the time will be marked as Then control the amount It will be applied to the system. In time... Inside, the control signal remains constant, that is .

[0162] From equation (86), we can obtain that in the interval It contains: ,in, As a unit array, , , It is to satisfy The time-varying parameters. Therefore, we obtain:

[0163] (63)

[0164] Note 3: Unlike traditional ETMs that independently monitor thrust and torque channels, the ETM proposed in this study operates on the norm of the entire control vector. The unified triggering strategy avoids the complex actuator allocation problem between propeller speed and rudder angle at the actuator level. It decouples triggering decisions from control allocation logic, making it more consistent with physical reality.

[0165] 4. Stability Analysis

[0166] Theorem: Consider An unmodeled dynamic USV formation. Under the proposed ETO-LOS method, the optimized control law ensures the formation output... Track the reference trajectory with the desired accuracy In the closed-loop system, all signals are SGUUBs, and the event-triggered mechanism effectively avoids Zeno behavior.

[0167] prove.

[0168] To prove that all closed-loop signals are SGUUB, the first Lyapunov function... Designed as follows:

[0169] (64)

[0170] in , , These are the Identifier, Critic, and Actor neural network weight errors, respectively.

[0171] Calculate along (14) and (26)-(28) The time derivatives are:

[0172] (65)

[0173] Interaction terms involving the product of estimated weights and ideal weights (e.g.) This can be simplified using the standard trace identity:

[0174] (66)

[0175] This also applies to the weight error term in other neural networks (i.e. and ).

[0176] set up The smallest eigenvalue is ,set up The largest eigenvalue is By repeatedly applying Young's inequality to the cross product term in (65) (e.g.) Substituting this into the trace identity above, we can obtain the derivative. It is bounded. Given the conditions in equation (29), the inequality can be restated as:

[0177] (67)

[0178] in .because All terms in the set are bounded, therefore there exists a constant. > 0, where .make Then the above equation (67) can be rewritten as:

[0179] (68)

[0180] In the second step of the backstep, the Lyapunov function Designed as follows:

[0181] (69)

[0182] Taking the derivative with respect to time along formula (69), we get:

[0183] (70)

[0184] The filter error dynamics, the limits in Note 2, and Substituting the result (68) into (70), and then repeatedly applying Young's inequality to all the resulting cross terms, we obtain:

[0185] (71)

[0186] in Considering Composed of bounded terms, it can introduce positive constants. To satisfy .make , Then the above expression can be rewritten as:

[0187] (72)

[0188] For the final backstep design incorporating a relative threshold event triggering mechanism, the Lyapunov function... Designed as follows:

[0189] (73)

[0190] in , , .

[0191] Calculate along (45), (56)-(58) The time derivative, when substituted into (50) and (55), yields:

[0192] (74)

[0193] Inverse matrix terms generated by the event-triggered mechanism Satisfying the matrix norm inequality This ensures that all matters involving and The terms remain bounded. Furthermore, using traces... The cyclic property allows for the rearrangement of trace terms to facilitate the application of Young's inequality and Cauchy-Schwarz inequality.

[0194] set up The smallest eigenvalue is , The largest eigenvalue is By applying these boundaries and repeatedly using the above inequalities, we can obtain:

[0195] (75)

[0196] All remaining cross products and bounded constant terms are set to Since each term is bounded, there exists a constant. , making .make , Then equation (75) can be restated as:

[0197] (76)

[0198] Applying Lemma 2 to (76), we get:

[0199] (77)

[0200] From the above inequalities, it can be seen that all error signals Dynamic surface filtering error And the weight estimation errors of all Identifier, Critic, and Actor neural networks. All are SGUUB.

[0201] To prove that the Zeno phenomenon is avoidable, we need to verify that there is a strictly positive minimum time interval between two consecutive event triggers.

[0202] For any trigger interval The measurement error norm is defined as .because The error derivative is constant within this interval, therefore it is... Since all system signals are SGUUB, this means It is bounded. Therefore, there exist positive constants. Make .

[0203] from arrive Integrate both sides and take the norm:

[0204] (78)

[0205] At the trigger time The error norm reaches the threshold ,lead to:

[0206] (79)

[0207] make This allows us to obtain the lower bound of the event interval:

[0208] (80)

[0209] Since all signals are SGUUB, therefore It is bounded. Assume and If , then the right side of the inequality is strictly positive. Therefore, there exists such that Minimum time between events This ensures that Zeno behavior does not exist under the proposed ETO-LOS scheme.

[0210] 5. Simulation Results

[0211] In this section, simulation studies are used to verify the effectiveness and robustness of the proposed ETO-LOS control strategy. Experiments were conducted on a formation of three USVs configured as a leader-follower structure with one leader and two followers. The dynamic models of each USV were built based on the Dolphin 2 platform, such as... Figure 3 As shown in Table 1, the parameters for "Dolphin 2" are listed.

[0212] Table 1. Parameter Matrix of "Dolphin 2" USV System

[0213] parameter value

[0214] The required reference trajectory input for USV formations is designed as follows: The external ocean environmental disturbance was designed as the sum of a second-order Markov process and a deterministic time-varying sinusoidal component, by... The initial positions and velocities of the USV swarm are given in Table 2.

[0215] Table 2 Initial State of USAs

[0216] state initial value

[0217] The two-stage IAC neural network is designed using Gaussian radial basis functions (RBF). For the first-stage motion controller, both the actor-critic network and the identifier network consist of 100 nodes. The basis function vectors are... ,in ,center exist Uniformly distributed within the range, with all centers having a width of [missing information]. For the second-stage dynamics controller, both the actor-critic and the recognizer networks consist of 400 nodes. The basis function vector is defined as follows: The activation function is .center exist Uniformly distributed within the range, with the width of all neurons set to... The learning rate and related parameter settings for the network update pattern are as follows: The controller and simulation parameters are selected as follows. The feedback gain matrix is ​​set to... and The time constant of the dynamic surface filter is set to... yaw rate controller gain For the event triggering mechanism, the parameters are: .

[0218] The ETO-LOS scheme implemented on the "Dolphin 2" experimental platform based on the simulation results of formation control is as follows: Figure 4-14 As shown. To verify the superiority and effectiveness of the proposed strategy, two comparative studies were conducted.

[0219] Figure 4-6 A comparative study is presented between the proposed ETO-LOS scheme and the traditional Backstepping-LOS controller (without IAC architecture). Figure 4 As shown, both controllers can follow the desired sinusoidal trajectory, but Figure 5 and 6 The error plot shows a significant performance gap. Under unknown dynamic conditions, the steady-state position error of the Backstepping-LOS controller is relatively large ( The ETO-LOS controller, through online learning of the optimal strategy, enables all tracking errors to converge rapidly to near zero. These results demonstrate the superiority of reinforcement learning-based optimal control strategies in handling uncertainty and improving tracking accuracy.

[0220] To evaluate the effectiveness of the proposed ETM, the ETO-LOS scheme is compared with continuous update schemes, such as... Figure 7 As shown in Figure 9, the event-triggered controller exhibits nearly the same convergence and tracking performance as the continuous version, while significantly reducing update frequency and resource consumption. The results demonstrate that this ETM algorithm maintains formation tracking accuracy while improving computational efficiency.

[0221] The internal convergence of the ETO-LOS algorithm proposed in this invention is as follows: Figure 10 - As shown in Figure 14. Weight norms of first- and second-level Actor-Critic networks ( Figure 10 , 11 ) and value function ( Figure 12 The simulation consistently converged to the bounded region, verifying that the optimal control strategy is effectively learned online. The identifier network weights (Figure 13) remained bounded and adaptive throughout the simulation, ensuring compensation for model uncertainties. Furthermore, the event intervals of the event-triggered controller (…) Figure 14 By strictly maintaining a positive value and exceeding the sampling period, Zeno behavior was confirmed to be absent. Overall, the ETO-LOS scheme achieved stable learning and closed-loop performance while significantly reducing the control update frequency.

[0222] This invention proposes an Event-Triggered Optimal Line-of-Sight (ETO-LOS) control scheme for underactuated USV formations facing external disturbances and model uncertainties. This framework integrates an identifier-actor-critic reinforcement learning architecture with dynamic surface control to achieve efficient online optimal control while reducing computational complexity. Simulation results on the "Dolphin 2" USV high-fidelity platform verify that the proposed ETO-LOS method, compared to traditional continuous backstepping-LOS control, achieves faster convergence, smaller tracking error, and a significantly reduced control update frequency, while maintaining comparable tracking accuracy. Future work will extend the proposed scheme to dynamic obstacle avoidance and include high-fidelity simulations and real-world multi-USV experiments for practical validation.

[0223] The following is a list of references cited in this invention:

[0224] [1]TOYOMOTO Y, OSHIMA T, OISHI K, et al. Constraint-Driven Multi-USVCoverage Path Generation for Aquatic Environmental Monitoring[J / OL]. IEEETransactions on Control Systems Technology, 2025: 1-15. DOI:10.1109 / TCST.2025.3572776.

[0225] [2]LIU Y, CHEN C, QU D, et al. Multi-USV System AntidisturbanceCooperative Searching Based on the Reinforcement Learning Method[J / OL]. IEEEJournal of Oceanic Engineering, 2023, 48(4): 1019-1047. DOI:10.1109 / JOE.2023.3281630.

[0226] [3]ENNONG T, YE L, TENG M, et al. Design and experiment of a sea-air heterogeneous unmanned collaborative system for rapid inspection tasks at sea[J / OL]. Applied Ocean Research, 2024, 143: 103856. DOI:10.1016 / j.apor.2023.103856.

[0227] [4]FAN Y, QIU B, LIU L, et al. Global fixed-time trajectory tracking control of underactuated USV based on fixed-time extended state observer[J / OL]. ISA Transactions, 2023, 132: 267-277. DOI:10.1016 / j.isatra.2022.06.011.

[0228] [5]TANG C, ZHANG H T, CAO H, et al. Time-Varying Formation Control of Autonomous Surface Vehicles Based on Affine Observer[J / OL]. IEEE Transactions on Industrial Electronics, 2024, 71(10): 12952-12963. DOI:10.1109 / TIE.2024.3355505.

[0229] [6]SONG S, LIU Z, YUAN S, et al. A finite-time path following scheme of unmanned surface vessels with an optimization strategy[J / OL]. ISA Transactions, 2024, 146: 61-74. DOI:10.1016 / j.isatra.2024.01.016.

[0230] [7] GU N, WANG D, PENG Z, et al. Adaptive bounded neural network control for coordinated path-following of networked underactuated autonomous surface vehicles under time-varying state-dependent cyber-attack[J / OL]. ISA Transactions, 2020, 104: 212-221. DOI:10.1016 / j.isatra.2018.12.051.

[0231] [8] LIN B, XIE W, SHI Y, et al. Robust Target Interception Strategy for a USV With Experimental Validation[J / OL]. IEEE Robotics and Automation Letters, 2023, 8(11): 7042-7049. DOI:10.1109 / LRA.2023.3300235.

[0232] [9] KIM J. Target Following and Close Monitoring Using an Unmanned Surface Vehicle[J / OL]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(11): 4233-4242. DOI:10.1109 / TSMC.2018.2846602.

[0233]

[10] GHOMMAM J, MNIF F. Coordinated Path-Following Control for a Group of Underactuated Surface Vessels[J / OL]. IEEE Transactions on Industrial Electronics, 2009, 56(10): 3951-3963. DOI:10.1109 / TIE.2009.2028362.

[0234]

[11] PALIOTTA C, LEFEBER E, PETTERSEN K Y, et al. Trajectory Tracking andPath Following for Underactuated Marine Vehicles[J / OL]. IEEE Transactions onControl Systems Technology, 2019, 27(4): 1423-1437. DOI:10.1109 / TCST.2018.2834518.

[0235]

[12] HU B B, ZHANG H T, YAO W, et al. Spontaneous-Ordering PlatoonControl for Multirobot Path Navigation Using Guiding Vector Fields[J / OL].IEEE Transactions on Robotics, 2023, 39(4): 2654-2668. DOI:10.1109 / TRO.2023.3266994.

[0236]

[13] CHEN Z, ZUO Z. Non-singular cooperative guiding vector fieldunder a homotopy equivalence transformation[J / OL]. Automatica, 2025, 171:111962. DOI:10.1016 / j.automatica.2024.111962.

[0237]

[14] WANG S, WEN G, SHEN H, et al. A LOS-Based Path-Following Control ofUnderactuated Autonomous Surface Vessels for Time-Varying Formations: Theoryand Experiment[J / OL]. IEEE Transactions on Industrial Electronics, 2025, 72(10): 10804-10813. DOI:10.1109 / TIE.2025.3555021.

[0238]

[15] DU B, YANG K, ZHANG W, et al. Terminal Line-of-Sight Angle-Constrained Target Tracking Guidance for Unmanned Surface Vehicles[J / OL]. IEEE Transactions on Vehicular Technology, 2024, 73(9): 12515-12529. DOI:10.1109 / TVT.2024.3390001.

[0239]

[16] FOSSEN T I, BREIVIK M, SKJETNE R. Line-of-sight path following of underactuated marine craft[J / OL]. IFAC Proceedings Volumes, 2003, 36(21): 211-216. DOI:10.1016 / S1474-6670(17)37809-6.

[0240]

[17] LUO H, JI H, WANG X. Cooperative robust line‐of‐sight guidance law based on high‐gain observers for active defense[J / OL]. International Journal of Robust and Nonlinear Control, 2023, 33(16): 9602-9617. DOI:10.1002 / rnc.6877.

[0241]

[18] FOSSEN T I, PETTERSEN K Y, GALEAZZI R. Line-of-Sight Path Following for Dubins Paths With Adaptive Sideslip Compensation of Drift Forces[J / OL]. IEEE Transactions on Control Systems Technology, 2015, 23(2): 820-827. DOI:10.1109 / TCST.2014.2338354.

[0242]

[19] LIU F, SHEN Y, HE B, et al. Drift angle compensation-based adaptive line-of-sight path following for autonomous underwater vehicle[J / OL]. Applied Ocean Research, 2019, 93: 101943. DOI:10.1016 / j.apor.2019.101943.

[0243]

[20] KELASIDI E, LILJEBACK P, PETTERSEN K Y, et al. Integral Line-of-Sight Guidance for Path Following Control of Underwater Snake Robots: Theory and Experiments[J / OL]. IEEE Transactions on Robotics, 2017, 33(3): 610-628. DOI:10.1109 / TRO.2017.2651119.

[0244]

[21] LEKKAS A M, FOSSEN T I. Integral LOS Path Following for Curved Paths Based on a Monotone Cubic Hermite Spline Parametrization[J / OL]. IEEE Transactions on Control Systems Technology, 2014, 22(6): 2287-2301. DOI:10.1109 / TCST.2014.2306774.

[0245]

[22] WU W, PENG Z, WANG D, et al. Network-Based Line-of-Sight Path Tracking of Underactuated Unmanned Surface Vehicles With Experiment Results[J / OL]. IEEE Transactions on Cybernetics, 2022, 52(10): 10937-10947. DOI:10.1109 / TCYB.2021.3074396.

[0246]

[23] HUANG Y, WU D, LI L, et al. Event-triggered Cooperative Path following Control of Multiple Underactuated Unmanned Surface Vehicles with Complex Unknowns and Actuator Saturation[J / OL]. Ocean Engineering, 2022, 249:110740. DOI:10.1016 / j.oceaneng.2022.110740.

[0247]

[24] PENG Z, WANG D, CHEN Z, et al. Adaptive Dynamic Surface Control for Formations of Autonomous Surface Vehicles With Uncertain Dynamics[J / OL]. IEEE Transactions on Control Systems Technology, 2013, 21(2): 513-520. DOI:10.1109 / TCST.2011.2181513.

[0248]

[25] MA M, WANG T, GUO R, et al. Neural network‐based tracking control of autonomous marine vehicles with unknown actuator dead‐zone[J / OL]. International Journal of Robust and Nonlinear Control, 2022, 32(5): 2969-2982. DOI:10.1002 / rnc.5890.

[0249]

[26] WANG D, GAO N, LIU D, et al. Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications[J / OL]. IEEE / CAA Journal of Automatica Sinica, 2024, 11(1): 18-36. DOI:10.1109 / JAS.2023.123843.

[0250]

[27] BHASIN S, KAMALAPURKAR R, JOHNSON M, et al. A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systems[J / OL]. Automatica, 2013, 49(1): 82-92. DOI:10.1016 / j.automatica.2012.09.019.

[0251]

[28] WEN G, CHEN C L P, GE S S. Simplified Optimized Backstepping Control for a Class of Nonlinear Strict-Feedback Systems With Unknown Dynamic Functions[J / OL]. IEEE Transactions on Cybernetics, 2021, 51(9): 4567-4580. DOI:10.1109 / TCYB.2020.3002108.

[0252]

[29] ZHU H Y, LI Y X, TONG S. Dynamic Event-Triggered Reinforcement Learning Control of Stochastic Nonlinear Systems[J / OL]. IEEE Transactions on Fuzzy Systems, 2023, 31(9): 2917-2928. DOI:10.1109 / TFUZZ.2023.3235417.

[0253]

[30] VAMVOUDAKIS K G, LEWIS F L. Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem[J / OL]. Automatica, 2010, 46(5): 878-888. DOI:10.1016 / j.automatica.2010.02.018.

[0254]

[31] WEN G, GE S S, CHEN C L P, etc. Adaptive Tracking Control of Surface Vessel Using Optimized Backstepping Technique[J / OL]. IEEE Transactions on Cybernetics, 2019, 49(9): 3420-3431. DOI:10.1109 / TCYB.2018.2844177.

[0255]

[32] WANG N, LIU Y, LIU J, et al. Reinforcement learning swarm of self-organizing unmanned surface vehicles with unavailable dynamics[J / OL]. OceanEngineering, 2023, 289: 116313. DOI:10.1016 / j.oceaneng.2023.116313.

[0256]

[33] TANG D, PANG N, WANG X, et al. Zero-sum game-based formation controlfor multi-USVs under hybrid irregular output constraints[J / OL]. NonlinearDynamics, 2025, 113(16): 21531-21545. DOI:10.1007 / s11071-025-11257-2.

[0257]

[34] XING L, WEN C, LIU Z, et al. Event-Triggered Adaptive Control for aClass of Uncertain Nonlinear Systems[J / OL]. IEEE Transactions on AutomaticControl, 2017, 62(4): 2071-2076. DOI:10.1109 / TAC.2016.2594204.

[0258]

[35] LIU L, ZHANG W, WANG D, et al. Event-triggered extended stateobservers design for dynamic positioning vessels subject to unknown sea loads[J / OL]. Ocean Engineering, 2020, 209: 107242. DOI:10.1016 / j.oceaneng.2020.107242.

[0259]

[36] PENG Z, JIANG Y, WANG J. Event-Triggered Dynamic Surface Controlof an Underactuated Autonomous Surface Vehicle for Target Enclosing[J / OL].IEEE Transactions on Industrial Electronics, 2021, 68(4): 3402-3412. DOI:10.1109 / TIE.2020.2978713.

[0260]

[37] LUO B, HUANG T, LIU D. Periodic Event-Triggered SuboptimalControl With Sampling Period and Performance Analysis[J / OL]. IEEETransactions on Cybernetics, 2021, 51(3): 1253-1261. DOI:10.1109 / TCYB.2019.2909704.

[0261]

[38] ZHAO B, LIU D. Event-Triggered Decentralized Tracking Control ofModular Reconfigurable Robots Through Adaptive Dynamic Programming[J / OL].IEEE Transactions on Industrial Electronics, 2020, 67(4): 3054-3064. DOI:10.1109 / TIE.2019.2914571.

[0262]

[39] ZHANG Q, ZHAO D, WANG D. Event-Based Robust Control for UncertainNonlinear Systems Using Adaptive Dynamic Programming[J / OL]. IEEE Transactionson Neural Networks and Learning Systems, 2018, 29(1): 37-50. DOI:10.1109 / TNNLS.2016.2614002.

[0263]

[40] ZENG-GUANG HOU, LONG CHENG, MIN TAN. Decentralized RobustAdaptive Control for the Multiagent System Consensus Problem Using NeuralNetworks[J / OL]. IEEE Transactions on Systems, Man, and Cybernetics, Part B(Cybernetics), 2009, 39(3): 636-647. DOI:10.1109 / TSMCB.2008.2007810.

[0264]

[41] POLYCARPOU M M. Stable adaptive neural control scheme fornonlinear systems[J / OL]. IEEE Transactions on Automatic Control, 1996, 41(3):447-451. DOI:10.1109 / 9.486648.

Claims

1. An event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference, characterized in that, The control method is as follows: First, the desired trajectory of the multi-unmanned surface vessel (USV) cooperative formation follower is obtained through the LOS guidance layer, and a virtual control signal is generated. Optimal control is achieved through dynamic surface control (DSC) and a two-level IAC network. Then, an event triggering mechanism is built in the actuator. The optimal policy learning is simultaneously embedded into the LOS guidance layer and the underactuated dynamics layer; a two-level IAC network (architecture) is used to compensate for model uncertainties and suppress external disturbances in real time. The Event Trigger Mechanism (ETM) is designed based on the overall control vector, eliminating the independent monitoring and triggering of thrust and torque channels. The ETM operates on the norm of the entire control vector, avoiding the complex actuator allocation problem between propeller speed and rudder angle at the actuator level, and decoupling the triggering decision from the control allocation logic. The inertial matrix norm normalization term is added to the reference signal design in the event-triggered mechanism (ETM) to constrain the amplitude of the control input. The control input is a two-dimensional vector composed of the forces and torques generated by the servo motor and propeller, ensuring the physical feasibility of generating the control input and ultimately achieving optimal line-of-sight formation control for the unmanned surface vessel.

2. The event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in claim 1, characterized in that, The specific implementation process of the control method is as follows: For a set of USVs arranged in a leader-follower structure, the distance measured by LOS for follower i and leader j. and relative angle Defined as: （9） （10） in express The tangent value of is within the range of The formation structure is controlled by measuring the relative distance and angle between the navigator and followers, thus the formation output vector is represented as... The desired formation output vector is given by Given, among which ; The relative dynamic relationship between leaders and followers is as follows: （11） in Given by the following formula: （12） （13） The navigator's status here For followers Unavailable, therefore can This is considered as an unmodeled dynamic between the leader and the follower; Then, we conduct optimal virtual control design based on reinforcement learning, and present dynamic surface control (DSC) and optimal controller design based on event triggering.

3. The event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in claim 2, characterized in that, The optimal virtual control design process based on reinforcement learning is as follows: The goal of optimal virtual control design based on reinforcement learning is to make Tracking expected formation structure Therefore, the tracking error vector is defined. Its time derivative satisfies: （14） in It is an intermediate control signal; make For virtual control, For optimal virtual control, the optimal performance index function is defined as: （15） in It is a signal that includes the origin and the reference signal. The predefined compact set, Let the instantaneous cost function be considered. For optimal virtual control Taking the time derivative of the optimal performance index function (15), we obtain the HJB equation associated with (14) as follows: （16） By solving The optimal virtual control is obtained as follows: （17） make Estimate the nonlinear terms in the relative dynamic equations, and Divided into three parts: （18） in If the parameter matrix is ​​positive definite, then the optimal virtual control is expressed as: （19） because and Two unknown but continuous elements are used, therefore a neural network is employed in compact sets. Approximating it from above, making （20） in and It is the ideal weight. and It is a basis function vector. and It is the approximation error. By inserting (20) into (18) and (19), we obtain （21） （22） in ; To obtain an effective and optimal virtual controller, reinforcement learning is used to implement usable optimized control by constructing an IAC architecture. （23） （24） （25） in and They are and The estimate, , and These are the identifier, critic, and actor neural network weights, respectively. Weight vector of the identifier-actor-criitic network , and The update law is designed as follows: （26） （27） （28） in It is a design constant. , and These are design parameters that must be met. （29）。 4. The event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in claim 3, characterized in that, The implementation process of Dynamic Surface Control (DSC) is as follows: The yaw tracking error is stabilized by designing a dynamic surface for the derived virtual control law. ,make Inversely decode the virtual control signal and : （36） Introducing two new state variables and and let and Through two time constants and To obtain a first-order filter and ; （37） in , Then the error between the filtered signal and the true value, as well as the filtering error, are: （38） let along Differential yield The yaw control law is designed as follows: （39） Introducing new state variables ,let With a time constant To obtain a first-order filter : （40） Error legacy items Transformed using the mean value theorem and The function; make Then there is （41） According to the Mean Value Theorem for Multivariable Functions, for from arrive The change exists as a constant. , so that: （42） where constant Depends on function The upper bound of the gradient, since It contains only bounded trigonometric functions and state variables, and this upper bound always exists. Based on (38), squaring both sides of (42) and applying Young's inequality, we can obtain: （43）。 5. The event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in claim 4, characterized in that, The design process for an event-triggered optimal controller is as follows: By combining the relative threshold event triggering mechanism with the reinforcement learning-optimized backstepping control law, the final control law is obtained: First, define the tracking error as... The desired reference signal is From equations (3), (38), and (40), it can be seen that... The derivative with respect to time is: （44） make ,but: （45） set up For optimal control, the optimal performance index function of the subsystem is defined as: （46） in The cost function; The HJB equation derived from (45) is: （47） By solving Achieving optimal control: （48） make In order to estimate the nonlinear terms in the dynamic equations, The following split is performed: （49） in It is a positive definite parameter matrix. .because and For an unknown and continuous set, approximate it using an RBF neural network on a given compact set: （52） in and It is the ideal weight. and It is a basis function vector. and It is the approximation error; Substituting (50) into (48) and (49) yields: （51） （52） in ; To achieve effective optimization control, the following structure is constructed: Network and network: （53） （54） （55） in, , They are and The estimated value, , and They are , and The weights of the neural network; , and The neural network weight update rule is designed as follows: （56） （57） （58） in, It is a design constant. , and These are design parameters that must be met. （59）。 6. The event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in claim 5, characterized in that, To achieve the system control signal When the amplitude is high, a relatively large measurement error is accepted, thus obtaining a longer update interval; while when the control signal tends to the equilibrium point with the system state, more precise control is obtained through a smaller threshold, thereby improving system performance. The following relative threshold control strategy is proposed: To balance control performance and update frequency, a relative threshold strategy is introduced, where the trigger condition adapts to the control signal. The magnitude. The strategy is formulated as follows: （60） Then the triggering mechanism is defined as: （61） （62） in, To account for measurement error, as well as For design parameters, The controller updates the time, that is, whenever (62) is triggered, the time will be marked as Then control the amount It will be applied to the system. In time... Inside, the control signal remains constant, that is ; Therefore, in the interval It contains: ,in, As a unit array, , , It is to satisfy The time-varying parameters are thus obtained: （63）。 7. An event-triggered optimal line-of-sight formation control system for under-actuated unmanned surface vessels under unknown disturbances, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 6, and executes the steps of the event-triggered optimal line-of-sight formation control method for under-steering unmanned surface vessels under unknown interference when running.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the event-triggered optimal line-of-sight formation control method for under-actuated unmanned surface vessels under unknown interference as described in any one of claims 1 to 6.

Images