Unmanned surface vehicle encircle surrounding hierarchical game control method under unreliable communication

Abstract

The embodiment discloses an unmanned ship encircling surrounding layered game control method under unreliable communication, considers that target information is not completely known and unreliable communication conditions of intermittent communication and false data injection (FDI) exist simultaneously, adopts a prescribed time flexible Nash equilibrium target state estimator converging under intermittent communication, ensures that the target estimation pose can converge to a saddle point according to the prescribed time and performance under intermittent communication, and based on this, a prescribed performance guidance law under the prescribed time is designed, longitudinal reference speed of the unmanned ship and bow reference speed of the unmanned ship are acquired, a cost function is established, then Hamilton-Jacobi-Isaacs equation is established, ideal optimal dynamic control law and optimal false data are acquired, and a critic neural network learning structure is used for approximation, and ideal optimal dynamic control law after approximation and optimal false data after approximation are acquired.

Description

Technical Field

[0001] This invention relates to the field of unmanned surface vessel (USV) circling and encirclement control technology, and in particular to a hierarchical game-theoretic control method for USV circling and encirclement under unreliable communication. Background Technology

[0002] Unmanned surface vessels (USVs), as a highly intelligent unmanned surface platform, play an increasingly important role in various mission scenarios such as maritime patrol, target protection, environmental monitoring, and maritime search and rescue, thanks to their features such as autonomous navigation, flexible deployment, remote control, and strong mission adaptability.

[0003] In target encirclement missions, multiple USVs can coordinate to encircle and track moving targets at sea, forming a dynamic encirclement circle. Compared to a single hull, the multi-USV system not only enhances mission robustness and fault tolerance through spatially distributed deployment, but also maintains formation integrity under target maneuvering or external interference, significantly improving encirclement success rate and mission adaptability. In real-world mission environments, multi-vessel collaboration enables rapid response and intelligent scheduling through information sharing and distributed decision-making, effectively mitigating the risk of single-point failures and enhancing the system's survivability and mission execution capabilities under complex sea conditions and external interference, forming a resilient, reconfigurable, and intelligently collaborative unmanned intelligent unit at sea.

[0004] Currently, significant progress has been made in research on the problem of multiple unmanned surface vessels (USVs) surrounding and encircling each other in complex marine environments. However, existing strategies still have the following problems:

[0005] Based on the widely adopted multi-USV surround control scheme, most existing studies assume a fixed communication topology. However, in real-world mission scenarios, communication links may dynamically change or even be interrupted due to mission switching or physical obstruction, leading to cooperative control failure. Most methods treat interference as a single type (e.g., environmental disturbances), lacking unified modeling and cooperative processing capabilities for mixed interference (e.g., environmental noise, intermittent communication, communication delays, spurious data injection, and other multi-source concurrent interference). Furthermore, existing control methods often focus on asymptotic stability or finite-time convergence, lacking convergence mechanisms under predetermined or specified time conditions. During actual mission execution, the cooperative system needs to meet strict time constraints (e.g., it must complete the surround within a specified time), otherwise, mission failure may occur. Summary of the Invention

[0006] This invention discloses a hierarchical game control method for unmanned surface vessels (USVs) under unreliable communication to overcome the aforementioned technical problems.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A hierarchical game-theoretic control method for unmanned surface vessels (USVs) under unreliable communication conditions includes the following steps:

[0009] S1: Construct a mathematical model of the unmanned vessel that includes the injection of fake data, wherein the mathematical model of the unmanned vessel includes the dynamics model and the kinematics model of the unmanned vessel;

[0010] S2: Based on the mathematical model of the unmanned vessel, construct a time-limited elastic Nash equilibrium target state estimator to obtain the output of the time-limited elastic Nash equilibrium target state estimator, that is, the unmanned vessel's estimate of the target pose;

[0011] S3: Establish a specified performance function for a specified time; establish a pose error system based on the unmanned surface vessel's (USV) estimation of the target pose; and obtain the pose error of the USV after conversion based on the specified performance function and pose error system. Then, establish a specified performance guidance law for a specified time based on the pose error of the USV to obtain the longitudinal reference velocity and the bow reference velocity of the USV.

[0012] S4: Based on the longitudinal reference velocity and the bow reference velocity of the unmanned surface vessel, establish a cost function to establish the Hamilton-Jacobi-Isaacs equations;

[0013] S5: Based on the Hamilton-Jacobi-Isaacs equation, obtain the ideal optimal dynamic control law and the optimal spurious data; and use the Critic neural network learning structure to approximate it, and obtain the approximate ideal optimal dynamic control law and the approximate optimal spurious data.

[0014] Furthermore, the time-elastic Nash equilibrium target state estimator is constructed as follows:

[0015]

[0016]

[0017]

[0018]

[0019] In the formula: For unmanned surface vessels The output of the time-bound elastic Nash equilibrium target state estimator. ; To define a time function; To define the coefficient matrix of the time-elastic Nash equilibrium target state estimator, , It is a 3-dimensional column vector with elements all equal to 1. The total number of unmanned surface vessels in the formation. All are unmanned surface vessel (USV) designations; The gradient of the cost function for the time-elastic Nash equilibrium objective state estimator is defined. For unmanned surface vessels Estimation of target pose, , This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; The estimated heading state of the target; To eliminate unmanned surface vessels The output of the time-flexible Nash equilibrium target state estimator, in addition to the specified time; To estimate the consistency term of the error; It specifies the convergence time; It is time; These are elements of the adjacency matrix; It is an unmanned surface vessel. For unmanned surface vessels The estimation of the pose. It is an unmanned surface vessel. Estimation of the target pose; To track target weights; The target pose state; These are the weighting coefficients of the time-elastic Nash equilibrium target state estimator; Here is the estimator weight matrix. , It is a diagonal matrix. For elements with a value of 1 3D column vector, For elements with a value of 1 3D column vector; It is a 3D zero matrix; For transpose;

[0020] The cost function of the time-elastic Nash equilibrium objective state estimator is expressed as follows:

[0021]

[0022] In the formula: Let the cost function of the time-elastic Nash equilibrium target state estimator be defined. This is the norm symbol.

[0023] Furthermore, the specified performance function under the specified time is constructed as follows:

[0024]

[0025] In the formula: The specified performance function over a specified time; This is the initial value for the function; This is the steady-state value of the function; It is the hyperbolic tangent function; To approximate the velocity parameters; For exponential decay parameters; To specify the convergence time; For time.

[0026] The pose error system is represented as follows:

[0027]

[0028] in,

[0029]

[0030]

[0031]

[0032]

[0033] In the formula: This refers to longitudinal error; This refers to lateral error; This refers to the heading angle error; Represents the coordinate system rotation matrix; The desired vertical position; The desired horizontal position; Desired heading angle; The radius of the enclosure; For unmanned surface vessels The relative angle to the target; This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; For unmanned surface vessels The rate of change of the relative angle with respect to the target; For time; , , These represent unmanned surface vessels. Its longitudinal position, lateral position, and heading angle; , , These represent unmanned surface vessels. The longitudinal velocity, lateral velocity, and bow velocity.

[0034] Furthermore, the formula used to obtain the pose error of the converted unmanned surface vessel is as follows:

[0035] , ,

[0036] In the formula, This is the converted longitudinal error. This is the transformed lateral error. This is the converted heading angle error. It is a logarithmic function; This refers to longitudinal error; This refers to lateral error; This refers to the heading angle error; This is the performance function defined over a specified time period.

[0037] Furthermore, the specified performance guidance law for the specified time is expressed as follows:

[0038]

[0039]

[0040]

[0041] ,

[0042] ,

[0043] ,

[0044] In the formula: The longitudinal reference speed of the unmanned surface vessel; This is the reference speed for the unmanned surface vessel's heading. Let be the longitudinal velocity of the desired trajectory in the attached coordinate system; The forward angular velocity of the desired trajectory in the attached coordinate system; Indicates the longitudinal positive feedback gain. Indicates the gain of the lateral positive feedback. Indicates the forward positive feedback gain; Indicates longitudinal gain. Indicates the longitudinal gain rate; Indicates lateral gain. Indicates the transverse gain rate; Indicates forward gain. Indicates the forward gain rate; For reference, the error of the heading angle; Represents the vertical scaling variable. ; This represents the horizontal scaling variable. ; It is the arctangent function; The derivative of a given performance function over a specified time; This is the converted longitudinal error. This is the transformed lateral error. This represents the converted heading angle error; This refers to longitudinal error; This refers to lateral error; unmanned surface vessel The heading speed; unmanned surface vessel The velocity vector; This is the performance function defined over a specified time period.

[0045] Furthermore, the cost function is constructed as follows:

[0046]

[0047]

[0048]

[0049] In the formula: Represents the cost function; For speed error, unmanned surface vessel The velocity vector, The desired velocity vector; Represents the dynamic control law; This indicates false data; It is the lower limit of integration; This represents the velocity error weight matrix. ; This represents the control law weight matrix. ; Indicates the attenuation coefficient; Represents the weight matrix of spurious data. ;

[0050] The Hamilton-Jacobi-Isax equations are established as follows:

[0051]

[0052]

[0053] In the formula: Represents the Hamilton-Jacobi-Isaacs equations; For the cost function The gradient; The derivative representing the velocity error.

[0054] Furthermore, the formulas used to obtain the ideal optimal dynamic control law and the optimal spurious data are as follows:

[0055]

[0056]

[0057] In the formula: The inertia matrix; Represents the dynamic control law; Represents the control law weight matrix; Represents the weight matrix of spurious data; For the cost function The gradient; Indicates the attenuation coefficient; Represents the Hamilton-Jacobi-Isaacs equations; Represents the dynamic control law; This indicates false data;

[0058] Therefore, the ideal optimal dynamic control law and the optimal spurious data are expressed as follows:

[0059]

[0060]

[0061] In the formula: Represents the gradient of the cost function; It indicates the inverse.

[0062] Furthermore, the loss function of the Critic neural network learning structure is expressed as follows:

[0063]

[0064] In the formula, The loss function; , All are weights; It is a conservative gain; It is the predetermined convergence time of the adaptation law; For power-order parameters, ; Indicates radial error, , For speed error, It is an approximation of the ideal optimal dynamic control law; It is an approximation of the ideal optimal injection of fake data; For the weighted regression vector, ; For Critic network weights; This represents the gradient of the Gaussian function; For Gaussian functions, , Indicates the center point of the hidden layer. Indicates the width of the Gaussian function; For a historic moment Radial error; Indicates a historical moment; This represents the k-th historical moment; Represents the total number of historical time series; This marks the first historical moment. For time; For historical weighted regression vectors, ; This is a weighted adaptive law;

[0065] The adaptive weight law is expressed as follows:

[0066]

[0067] In the formula: This represents the learning law.

[0068] Furthermore, the approximated ideal optimal dynamic control law and the approximated optimal spurious data are represented as follows:

[0069]

[0070]

[0071] In the formula, It is the approximation of the ideal optimal dynamic control law. It is the optimal spurious data after approximation.

[0072] Furthermore, the kinematic model of the unmanned surface vessel is established as follows:

[0073]

[0074] The dynamic model of the unmanned surface vessel is established as follows:

[0075]

[0076]

[0077] In the formula: unmanned surface vessel The pose matrix, where, , , , These represent unmanned surface vessels. Its longitudinal position, lateral position, and heading angle; For transpose; Represents the coordinate system rotation matrix; unmanned surface vessel The velocity vector, where, , , , These represent unmanned surface vessels. The longitudinal velocity, lateral velocity, and bow velocity; Represents the inertia matrix; It is a first-order differential; Represent the Coriolis matrix and the centripetal matrix; Represents the fluid dynamics damping matrix; Indicates control input; Indicates longitudinal thrust. Indicates the bow moment; This indicates the injection of fake data; This indicates vertically spurious data. This indicates that the data is false.

[0078] Beneficial Effects: This invention provides a hierarchical game-theoretic control method for unmanned surface vessels (USVs) under unreliable communication conditions. Considering the incomplete knowledge of the tracking target and the simultaneous presence of intermittent communication and fictitious data injection (FDI), it employs a time-bound elastic Nash equilibrium target state estimator that converges under intermittent communication. This ensures that the estimated target pose converges to the saddle point within a specified time and performance range under intermittent communication. Based on this, a time-bound, performance-based guidance law is designed to obtain the longitudinal and bow reference velocities of the USV, establishing a cost function. This leads to the establishment of the Hamilton-Jacobi-Isax equation, yielding the ideal optimal dynamic control law and the optimal fictitious data. A Critic neural network learning structure is then used for approximation, obtaining the approximate ideal optimal dynamic control law and the approximate optimal fictitious data. This invention abandons the fixed topology assumption, enabling the system to maintain cooperative stability when the communication link dynamically changes. Multi-source interference is modeled as a two-layer game strategy space, achieving unified analysis and suppression of complex interference. It significantly improves the overall control performance of unmanned surface vessel (USV) formations, enhances their resistance to various network interferences, and provides important technical support for the widespread application of USVs. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 This is a flowchart of the unmanned surface vessel surround and hierarchical game control method under unreliable communication of the present invention;

[0081] Figure 2 This is a communication topology diagram in an embodiment of the present invention;

[0082] Figure 3 This is a block diagram of the surrounding hierarchical game control structure in an embodiment of the present invention;

[0083] Figure 4 This is the surrounding effect of 5 unmanned surface vessels in an embodiment of the present invention;

[0084] Figure 5 This is the target estimator input change curve of USV2 in this embodiment of the invention;

[0085] Figure 6 These are the longitudinal position error variation curves of the five unmanned surface vessels in this embodiment of the invention;

[0086] Figure 7 These are the lateral position error variation curves of the five unmanned surface vessels in this embodiment of the invention;

[0087] Figure 8 These are the heading angle error variation curves of the five unmanned surface vessels in this embodiment of the invention;

[0088] Figure 9 This is the evaluation network weight change curve of the five unmanned surface vessels in this embodiment of the invention. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] This embodiment introduces a hierarchical game-theoretic control method for unmanned surface vessels (USVs) under unreliable communication, including the following steps: Figure 1 As shown:

[0091] Specifically, this embodiment discusses an unmanned surface vessel (USV) surround game control method under unreliable communication including intermittent communication and spurious data injection.

[0092] S1: Construct a mathematical model of the unmanned vessel that includes the injection of fake data, wherein the mathematical model of the unmanned vessel includes the dynamics model and the kinematics model of the unmanned vessel.

[0093] The model states include the unmanned surface vessel's lateral velocity, heading velocity, longitudinal velocity, longitudinal position, lateral position, and heading angle.

[0094] Specifically, the unmanned surface vessel dynamics model constructed in this embodiment includes injected spurious data, and the pose information in the geodetic coordinate system is obtained through a kinematic model.

[0095] Preferably, the kinematic model of the unmanned surface vessel is established as follows:

[0096] (1)

[0097] The dynamic model of the unmanned surface vessel is established as follows:

[0098] (2)

[0099]

[0100] In the formula: unmanned surface vessel The pose matrix, where, , , , These represent unmanned surface vessels. Its longitudinal position, lateral position, and heading angle; For transpose; Represents the coordinate system rotation matrix; unmanned surface vessel The velocity vector, where, , , , These represent unmanned surface vessels. The longitudinal velocity, lateral velocity, and bow velocity; Represents the inertia matrix; It is a first-order differential; Represent the Coriolis matrix and the centripetal matrix; Represents the fluid dynamics damping matrix; Indicates control input; Indicates longitudinal thrust. Indicates the bow moment; This indicates the injection of fake data; This indicates vertically spurious data. This indicates that the data is false.

[0101] Among them, the inertia matrix Coriolis matrix and centripetal matrix Fluid dynamics damping matrix And coordinate system rotation matrix for:

[0102] ; ;

[0103] ; ;

[0104] in, It is the quality of the unmanned surface vessel; The longitudinal additional mass coefficient generated by longitudinal acceleration; The longitudinal additional mass coefficient generated by the lateral acceleration; The additional moment of inertia coefficient in the bow direction generated by the bow acceleration; The lateral additional mass coefficient generated by lateral acceleration; The longitudinal linear water damping coefficient generated by the longitudinal velocity; The second-order longitudinal nonlinear water damping coefficient generated by the longitudinal velocity; The lateral linear water damping coefficient generated by the lateral velocity; The second-order lateral nonlinear water damping coefficient generated by the lateral velocity; The bow linear water damping coefficient is generated by the bow angular velocity; The second-order nonlinear water damping coefficient generated by the bow angular velocity; It is the unmanned surface vessel's registration number; Let be the moment of inertia of the unmanned surface vessel about the z-axis.

[0105] S2: Based on the mathematical model of the unmanned vessel, a time-bound elastic Nash equilibrium target state estimator under intermittent communication is constructed to obtain the output of the time-bound elastic Nash equilibrium target state estimator, that is, the unmanned vessel's estimate of the target pose.

[0106] Specifically, to enable all USVs in the formation to quickly and accurately acquire the target's motion state, a time-bound elastic Nash equilibrium target state estimator is designed. The output of the time-bound elastic Nash equilibrium target state estimator for each USV includes its own estimate of the target and the estimates of the target from other USVs in the topology. Distributed consensus and Nash equilibrium search are achieved through state expansion. This yields the real-time motion state of the target USV, which will be referenced in the subsequent design of the time-bound guidance law. Furthermore, intermittent communication needs to meet certain assumptions, namely, the duration and frequency of communication interruptions must be finite before and after the specified convergence time.

[0107] Preferably, the time-elastic Nash equilibrium target state estimator is constructed as follows:

[0108] (3)

[0109]

[0110]

[0111]

[0112] In the formula: For unmanned surface vessels The output of the time-bound elastic Nash equilibrium target state estimator. ; To define a time function; To define the coefficient matrix of the time-elastic Nash equilibrium target state estimator, , It is a 3-dimensional column vector with elements all equal to 1. The total number of unmanned surface vessels in the formation. All are unmanned surface vessel (USV) designations; The gradient of the cost function for the time-elastic Nash equilibrium objective state estimator is defined. For unmanned surface vessels Estimation of target pose, , This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; The estimated heading state of the target; To eliminate unmanned surface vessels The output of the time-flexible Nash equilibrium target state estimator, in addition to the specified time; To estimate the consistency term of the error; It specifies the convergence time; It is a time variable; These are elements of the adjacency matrix; It is an unmanned surface vessel. For unmanned surface vessels The estimation of the pose. It is an unmanned surface vessel. Estimation of the target pose; To track target weights; The target pose state; These are the weighting coefficients of the time-elastic Nash equilibrium target state estimator; Here is the estimator weight matrix. , It is a diagonal matrix. For elements with a value of 1 3D column vector, For elements with a value of 1 3D column vector; It is a 3D zero matrix;

[0113] The cost function of the time-elastic Nash equilibrium objective state estimator is expressed as follows:

[0114] (4)

[0115] In the formula: Let the cost function of the time-elastic Nash equilibrium target state estimator be defined. This is the norm symbol.

[0116] S3: Establish a specified performance function for a specified time; establish a pose error system based on the unmanned surface vessel's (USV) estimation of the target pose; and obtain the pose error of the USV after conversion based on the specified performance function and pose error system. Then, establish a specified performance guidance law for a specified time based on the pose error of the USV to obtain the longitudinal reference velocity and the bow reference velocity of the USV.

[0117] Based on the estimated motion state of the surrounding target, this embodiment designs a specified performance guidance law for a specified time to provide the desired longitudinal velocity and heading angular velocity for achieving the goal of surrounding the target.

[0118] Specifically, based on the offset of the target USV's motion state estimate and the motion state of the formation member USVs, the pose error system of the target and formation member USVs is calculated. A specified performance function that meets the convergence requirement within a specified time is designed and the pose error system is transformed. Finally, a specified performance guidance law that stabilizes the pose error system within a specified time is obtained. This guidance law will then be used as the velocity reference value for subsequent calculations.

[0119] The pose error system is represented as follows:

[0120] (5)

[0121] in,

[0122]

[0123]

[0124]

[0125]

[0126] In the formula: This refers to longitudinal error; This refers to lateral error; This refers to the heading angle error; Represents the coordinate system rotation matrix; The desired vertical position; The desired horizontal position; Desired heading angle; The radius of the enclosure; For unmanned surface vessels The relative angle to the target; This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; For unmanned surface vessels The rate of change of the relative angle with respect to the target; For time;

[0127] Preferably, the specified performance function under the specified time is constructed as follows:

[0128] (7)

[0129] In the formula: The specified performance function over a specified time; This is the initial value for the function; This is the steady-state value of the function; It is the hyperbolic tangent function; To approximate the velocity parameters; For exponential decay parameters; To specify the convergence time.

[0130] Based on the differential equation of the pose error system and the specified performance function over a specified time, the error is transformed using the specified performance function over the specified time:

[0131] , , (8)

[0132] In the formula, This is the converted longitudinal error. This is the transformed lateral error. This is the converted heading angle error. It is a logarithmic function.

[0133] Preferably, a guidance law with specified performance over a specified time is established based on the converted error:

[0134] (9)

[0135] (10)

[0136] (11)

[0137] ,

[0138] ,

[0139] ,

[0140] In the formula: The longitudinal reference speed of the unmanned surface vessel; This is the reference speed for the unmanned surface vessel's heading. Let be the longitudinal velocity of the desired trajectory in the attached coordinate system; The forward angular velocity of the desired trajectory in the attached coordinate system; Indicates the longitudinal positive feedback gain. Indicates the gain of the lateral positive feedback. Indicates the forward positive feedback gain; Indicates longitudinal gain. Indicates the longitudinal gain rate; Indicates lateral gain. Indicates the transverse gain rate; Indicates forward gain. Indicates the forward gain rate; For reference, the error of the heading angle; Represents the vertical scaling variable. ; This represents the horizontal scaling variable. ; It is the arctangent function; It represents the derivative of a given performance function over a specified time.

[0141] S4: Based on the longitudinal reference speed and the bow reference speed of the unmanned surface vessel (USV), calculate the real-time speed error of each USV, thereby establishing the cost function under the zero-sum differential game, and thus establishing the Hamilton-Jacobi-Isaks equation.

[0142] Specifically, the velocity error vector is calculated using the obtained expected longitudinal and bow velocities and the real-time velocity state of the USV. The spurious data and the dynamic control law are regarded as two sides in a zero-sum game, and finally integrated into the cost function and the Hamilton-Jacobi-Isaac equation is calculated.

[0143] Preferably, the cost function is constructed as follows:

[0144] (12)

[0145]

[0146]

[0147] In the formula: Represents the cost function; For speed error, unmanned surface vessel The velocity vector, The desired velocity vector; Represents the dynamic control law; This indicates false data; It is the lower limit of integration; This represents the velocity error weight matrix. ; This represents the control law weight matrix. ; Indicates the attenuation coefficient; Represents the weight matrix of spurious data. ;

[0148] Based on the cost function, the Hamilton-Jacobi-Isaac equation is constructed as follows:

[0149] (13)

[0150]

[0151] In the formula: Represents the Hamilton-Jacobi-Isaacs equations; For the cost function The gradient; The derivative representing the velocity error;

[0152] S5: Based on the Hamilton-Jacobi-Isaacs equation, obtain the ideal optimal dynamic control law and the optimal spurious data; and use the Critic neural network learning structure to approximate it, and obtain the approximate ideal optimal dynamic control law and the approximate optimal spurious data.

[0153] Specifically, differential equations are established by taking the derivatives of the Hamilton-Jacobi-Isaac equations with respect to the dynamic control law and the spurious data injection and finding that they are equal to zero. The ideal optimal expressions for both can be obtained by solving the differential equations.

[0154] Preferably, the formulas used to obtain the ideal optimal dynamic control law and the optimal spurious data are as follows:

[0155] (14)

[0156] (15)

[0157] In the formula: The inertia matrix; Represents the dynamic control law; Represents the control law weight matrix; Represents the weight matrix of spurious data; For the cost function The gradient; Indicates the attenuation coefficient; This indicates false data;

[0158] The ideal optimal dynamic control law and the optimal spurious data are represented as follows:

[0159] (16)

[0160] (17)

[0161] In the formula: The gradient of the cost function is represented. It indicates the inverse.

[0162] In this embodiment, a Critic neural network learning structure is designed to approximate the ideal optimal control law and the optimal spurious data.

[0163] Specifically, since it is difficult to obtain a clear analytical solution based on the ideal optimal control law and optimal spurious data, a reinforcement learning (RL) method is used for approximation. The evaluation network in the RL method adopts a traditional RBF neural network structure, including an input layer, hidden layers, and an output layer. The input layer and hidden layers each contain three neurons, and the output layer contains one neuron. First, a loss function is designed using radial error, and the weight adaptive law of the Critic neural network is obtained by differentiating the weights, thereby approximating the ideal optimal dynamic control law and spurious data injection.

[0164] Preferably, the loss function of the Critic neural network learning structure is expressed as follows:

[0165] (18)

[0166] In the formula, The loss function; , All are weights; It is a conservative gain; It is the predetermined convergence time of the adaptation law; For power-order parameters, ; Indicates radial error, , For speed error, It is an approximation of the ideal optimal dynamic control law; It is an approximation of the ideal optimal injection of fake data; For the weighted regression vector, ; For Critic network weights; This represents the gradient of the Gaussian function; For Gaussian functions, , Indicates the center point of the hidden layer. Indicates the width of the Gaussian function; For a historic moment Radial error; Indicates a historical moment; This represents the k-th historical moment; Represents the total number of historical time series; This marks the first historical moment. For time; For historical weighted regression vectors, ; This is a weighted adaptive law;

[0167] The adaptive weight law is expressed as follows:

[0168] (19)

[0169] In the formula: This represents the learning law.

[0170] Preferably, the approximated ideal optimal dynamic control law and the approximated optimal spurious data are represented as follows:

[0171] (20)

[0172] (twenty one)

[0173] In the formula, It is the approximation of the ideal optimal dynamic control law. It is the optimal spurious data after approximation.

[0174] Specifically, a specific simulation experiment embodiment of the present invention is as follows:

[0175] Multi-unmanned surface vessel system communication topology such as Figure 2 As shown, five unmanned surface vessels (USVs) are selected to form a formation, with one USV acting as the target. The model parameters of the USVs are as follows: , , , , , , , , , , .

[0176] The initial states of the five unmanned surface vessels are represented as follows: , , , , , The target's trajectory is The radius of the surrounding area is... The rate of change of relative angle is The relevant parameters of the time-elastic Nash equilibrium estimator are specified as follows: , The relevant parameters of the specified performance guidance law under specified time are as follows: , , , , .

[0177] The relevant parameters for evaluating the network are: , The communication was interrupted 26 times, with each interruption lasting no more than 5 seconds.

[0178] Figure 3This is a game-theoretic control structure diagram of five unmanned surface vessels (USVs) surrounding a target under intermittent communication and FDI (Fixed Direct Identification). The USVs exchange state information via a wireless communication network. Each USV's control system mainly consists of a time-bound elastic Nash equilibrium estimator, a time-bound guidance law, a Critic learning network, and an optimal dynamic control law under a zero-sum game. As the motion state of the USVs surrounding the target changes, each USV's time-bound elastic Nash equilibrium estimator estimates the target's motion state in real time and calculates a reference trajectory that can form a surrounding situation based on the estimate. Using the obtained reference trajectory and a time-bound performance function, a guidance law is designed. Each USV uses an evaluation network to approximate the control law and injects spurious data into the optimal Nash equilibrium control law under a zero-sum game, enabling the USVs to surround the target.

[0179] Figures 4-9 These are simulation renderings, among which... Figure 4 The demonstration showed the effect of an unmanned surface vessel (USV) formation surrounding a target, demonstrating that each USV could accurately surround and encircle the target as it moved. Figure 5 The evolution of the input norm of the Nash equilibrium estimator, taking USV2 as an example, is shown under aperiodic intermittent communication. When communication resumes, the estimator recovers rapidly and achieves convergence. Figure 6 , Figure 7 and Figure 8 These are the longitudinal position, lateral position, and bow angle error curves for five USVs, respectively. It can be seen that the position and attitude error curves are all within the specified performance range. Figure 9 The curves show the changes in the evaluation network weights of each unmanned surface vessel (USV), indicating that the weight curves eventually converge during the learning process. This embodiment considers the simultaneous existence of intermittent communication and FDI, and combines a time-limited method, game-theoretic control method, and optimal control theory to achieve multi-USV target encirclement in complex environments.

[0180] Compared to existing unmanned surface vessel (USV) target encirclement methods, this embodiment considers both intermittent communication and spoofing data injection. A time-bound elastic Nash equilibrium estimator is designed for the encircled target, ensuring that each USV can acquire accurate target status within a specified time even with intermittent communication. Based on this, a time-bound performance function is designed to achieve motion guidance, enabling the pose error system to converge according to the specified time and performance. Finally, a zero-sum game model of dynamic control law and spoofing data injection is established, and the optimal control law and optimal spoofing data injection satisfying Nash equilibrium are solved through an evaluation network. In summary, this invention enables unmanned surface vessels to accurately and rapidly encircle targets even with unreliable network communication.

[0181] This invention presents a hierarchical game-theoretic control method for unmanned surface vessels (USVs) under unreliable communication conditions. It establishes a top-level Nash game and a bottom-level zero-sum game distributed control structure for kinematics and dynamics, respectively. In the top-level control framework, considering the unreliable communication situation where the tracking target information is not fully known and intermittent communication and spurious data injection (FDI) coexist, a time-bound elastic Nash equilibrium target state estimator converges under intermittent communication. This ensures that the estimated target pose converges to the saddle point within a specified time and performance limit under intermittent communication. Based on this, a time-bound, performance-based guidance law is designed to obtain the longitudinal and bow reference velocities of the USV, establishing a cost function. This leads to the establishment of the Hamilton-Jacobi-Isax equation, yielding the ideal optimal dynamic control law and the optimal spurious data. A Critic neural network learning structure is then used for approximation to obtain the approximate ideal optimal dynamic control law and the approximate optimal spurious data.

[0182] In the underlying control framework, the dynamic control law and the injection of spurious data are treated as a zero-sum game. An online reinforcement learning (RL) algorithm based on the Critic network is designed to solve the control law. The adaptive weight update law of the Critic network is designed to converge in a predetermined time.

[0183] This invention abandons the fixed topology assumption, enabling the system to maintain cooperative stability even when communication links change dynamically. It models multi-source interference as a two-layer game strategy space, achieving unified analysis and suppression of complex interference. This significantly improves the overall control performance of unmanned surface vessel (USV) formations, enhances resistance to various network interferences, and provides crucial technical support for the widespread application of USVs.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hierarchical game-theoretic control method for unmanned surface vessels (USVs) under unreliable communication, characterized in that: Includes the following steps: S1: Construct a mathematical model of the unmanned vessel that includes the injection of fake data, wherein the mathematical model of the unmanned vessel includes the dynamics model and the kinematics model of the unmanned vessel; S2: Based on the mathematical model of the unmanned vessel, construct a time-limited elastic Nash equilibrium target state estimator to obtain the output of the time-limited elastic Nash equilibrium target state estimator, that is, the unmanned vessel's estimate of the target pose; The time-elastic Nash equilibrium target state estimator is constructed as follows: In the formula: For unmanned surface vessels The output of the time-bound elastic Nash equilibrium target state estimator. ; To define a time function; To define the coefficient matrix of the time-elastic Nash equilibrium target state estimator, , It is a 3-dimensional column vector with elements all equal to 1. The total number of unmanned surface vessels in the formation. All are unmanned surface vessel (USV) designations; The gradient of the cost function for the time-elastic Nash equilibrium objective state estimator is defined. For unmanned surface vessels Estimation of target pose, , This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; The estimated heading state of the target; To eliminate unmanned surface vessels The output of the time-flexible Nash equilibrium target state estimator, in addition to the specified time; To estimate the consistency term of the error; It specifies the convergence time; It is time; These are elements of the adjacency matrix; It is an unmanned surface vessel. For unmanned surface vessels The estimation of the pose. It is an unmanned surface vessel. Estimation of the target pose; To track target weights; The target pose state; These are the weighting coefficients of the time-elastic Nash equilibrium target state estimator; Here is the estimator weight matrix. , It is a diagonal matrix. For elements with a value of 1 3D column vector, For elements with a value of 1 3D column vector; It is a 3D zero matrix; For transpose; The cost function of the time-elastic Nash equilibrium objective state estimator is expressed as follows: In the formula: Let the cost function of the time-elastic Nash equilibrium target state estimator be defined. Norm symbol; S3: Establish a specified performance function for a specified time; establish a pose error system based on the unmanned surface vessel's (USV) estimation of the target pose; and obtain the pose error of the USV after conversion based on the specified performance function and pose error system. Then, establish a specified performance guidance law for a specified time based on the pose error of the USV to obtain the longitudinal reference velocity and the bow reference velocity of the USV. S4: Based on the longitudinal reference velocity and the bow reference velocity of the unmanned surface vessel, establish a cost function to establish the Hamilton-Jacobi-Isaacs equations; S5: Based on the Hamilton-Jacobi-Isaacs equation, obtain the ideal optimal dynamic control law and the optimal spurious data; and use the Critic neural network learning structure to approximate it, and obtain the approximate ideal optimal dynamic control law and the approximate optimal spurious data.

2. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The specified performance function for the specified time is constructed as follows: In the formula: The specified performance function over a specified time; This is the initial value for the function; This is the steady-state value of the function; It is the hyperbolic tangent function; To approximate the velocity parameters; For exponential decay parameters; To specify the convergence time; For time; The pose error system is established as follows: in, In the formula: This refers to longitudinal error; This refers to lateral error; This refers to the heading angle error; Represents the coordinate system rotation matrix; The desired vertical position; The desired horizontal position; Desired heading angle; The radius of the enclosure; For unmanned surface vessels The relative angle to the target; This is the target's longitudinal state estimate; This is the estimated value of the target's lateral state; For unmanned surface vessels The rate of change of the relative angle with respect to the target; For time; , , These represent unmanned surface vessels. Its longitudinal position, lateral position, and heading angle; , , These represent unmanned surface vessels. The longitudinal velocity, lateral velocity, and bow velocity.

3. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The formula used to obtain the pose error of the converted unmanned surface vessel is as follows: ， ， In the formula, This is the converted longitudinal error. This is the transformed lateral error. This is the converted heading angle error. It is a logarithmic function; This refers to longitudinal error; This refers to lateral error; This refers to the heading angle error; This is the performance function defined over a specified time period.

4. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 3, characterized in that, The specified performance guidance law for the specified time is expressed as follows: ， ， ， In the formula: The longitudinal reference speed of the unmanned surface vessel; This is the reference speed for the unmanned surface vessel's heading. Let be the longitudinal velocity of the desired trajectory in the attached coordinate system; The forward angular velocity of the desired trajectory in the attached coordinate system; Indicates the longitudinal positive feedback gain. Indicates the gain of the lateral positive feedback. Indicates the forward positive feedback gain; Indicates longitudinal gain. Indicates the longitudinal gain rate; Indicates lateral gain. Indicates the transverse gain rate; Indicates forward gain. Indicates the forward gain rate; For reference, the error of the heading angle; Represents the vertical scaling variable. ; This represents the horizontal scaling variable. ; It is the arctangent function; The derivative of a given performance function over a specified time; This is the converted longitudinal error. This is the transformed lateral error. This represents the converted heading angle error; This refers to longitudinal error; This refers to lateral error; unmanned surface vessel The heading speed; unmanned surface vessel The velocity vector; The specified performance function over a specified time; unmanned surface vessel lateral velocity.

5. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The cost function is constructed as follows: In the formula: Represents the cost function; For speed error, unmanned surface vessel The velocity vector, The desired velocity vector; Represents the dynamic control law; This indicates false data; It is the lower limit of integration; This represents the velocity error weight matrix. ; This represents the control law weight matrix. ; Indicates the attenuation coefficient; Represents the weight matrix of spurious data. ; This is the reference speed for the unmanned surface vessel's heading. The Hamilton-Jacobi-Isax equations are established as follows: In the formula: Represents the Hamilton-Jacobi-Isaacs equations; For the cost function The gradient; The derivative representing the velocity error.

6. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The formulas used to obtain the ideal optimal dynamic control law and the optimal spurious data are as follows: In the formula: The inertia matrix; Represents the dynamic control law; Represents the control law weight matrix; Represents the weight matrix of spurious data; For the cost function The gradient; Indicates the attenuation coefficient; Represents the Hamilton-Jacobi-Isaacs equations; This indicates false data; Therefore, the ideal optimal dynamic control law and the optimal spurious data are expressed as follows: In the formula: Represents the gradient of the cost function; It indicates the inverse.

7. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The loss function of the Critic neural network learning structure is expressed as follows: In the formula, The loss function; , All are weights; It is a conservative gain; It is the predetermined convergence time of the adaptation law; For power-order parameters, ; Indicates radial error, , For speed error, It is an approximation of the ideal optimal dynamic control law; It is an approximation of the ideal optimal injection of fake data; For the weighted regression vector, ; For Critic network weights; This represents the gradient of the Gaussian function; For Gaussian functions, , Indicates the center point of the hidden layer. Indicates the width of the Gaussian function; For a historic moment Radial error; Indicates a historical moment; This represents the k-th historical moment; Represents the total number of historical time series; This marks the first historical moment. For time; For historical weighted regression vectors, ; Represents the velocity error weight matrix. ; This represents the control law weight matrix. ; Indicates the attenuation coefficient; Represents the weight matrix of spurious data. ; The adaptive weight law is expressed as follows: In the formula: This represents the learning law.

8. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The approximated ideal optimal dynamic control law and the approximated optimal spurious data are represented as follows: In the formula, It is the approximation of the ideal optimal dynamic control law. It is the best spurious data after approximation; This represents the control law weight matrix. ; Indicates the attenuation coefficient; Represents the weight matrix of spurious data. ; The inertia matrix; Indicates the attenuation coefficient; This represents the gradient of the Gaussian function; These are the weights of the Critic network.

9. The method for hierarchical game control of unmanned surface vessels under unreliable communication as described in claim 1, characterized in that, The kinematic model of the unmanned surface vessel is established as follows: The dynamic model of the unmanned surface vessel is established as follows: In the formula: unmanned surface vessel The pose matrix, where, , , , These represent unmanned surface vessels. Its longitudinal position, lateral position, and heading angle; For transpose; Represents the coordinate system rotation matrix; unmanned surface vessel The velocity vector, where, , , , These represent unmanned surface vessels. The longitudinal velocity, lateral velocity, and bow velocity; Represents the inertia matrix; It is a first-order differential; Representing the Coriolis matrix and the centripetal matrix; Represents the fluid dynamics damping matrix; Indicates control input; Indicates longitudinal thrust. Indicates the bow moment; This indicates the injection of fake data; This indicates vertically spurious data. This indicates that the data is false.

Images