An adaptive optimal control method and system for an underactuated underwater vehicle, a computer device and a storage medium

Abstract

The application provides an adaptive optimal control method and system for an underactuated underwater vehicle, and belongs to the technical field of optimal control of underactuated autonomous underwater vehicles. The method comprises the following steps: collecting a plurality of groups of sensor data in real time for different system states of the underwater vehicle; obtaining preliminary purified data and a measurement covariance matrix through an attack isolation algorithm; obtaining accurate final state data by using an unscented Kalman filter; calculating a tracking error based on the path and the final state data, and re-determining a virtual control variable according to the error by using a backstepping method; and finally generating optimal control instructions by reinforcement learning to guide the underwater vehicle to stably move along an expected three-dimensional path. The application not only improves the attack resistance and disturbance resistance of sensor data, but also realizes optimal control energy consumption while ensuring tracking accuracy by combining backstepping control and reinforcement learning, thereby significantly enhancing the tracking stability and operation efficiency of the underactuated underwater vehicle in a complex underwater environment.

Description

Technical Field

[0001] This invention belongs to the field of optimal control technology for underactuated autonomous underwater vehicles, specifically relating to an adaptive optimization control method and system for underactuated underwater vehicles. Background Technology

[0002] In practical marine engineering tasks, accurate trajectory following capability and efficient energy utilization are crucial indicators for evaluating the control performance of autonomous underwater vehicles (AUVs). Therefore, optimal 3D tracking control technology has garnered widespread attention in the field of underwater robotics due to its advantageous balance between performance and cost. The core characteristic of optimal 3D tracking control is that it eliminates the need to consider time constraints on the tracking state, and while ensuring high-precision trajectory tracking for AUVs, it minimizes control energy consumption, actuator load, and system jitter. This means that regardless of the complexity of the mission trajectory or the degree of disturbance in the marine environment, AUVs can achieve stable following of the expected path at the most economical control cost. This characteristic is particularly important in engineering applications because it not only improves the endurance and mission execution efficiency of AUVs but also allows controller designers to accurately predict performance during the mission planning phase, thereby meeting the requirements of long-term, high-precision underwater operations. Therefore, this research aims to significantly improve the trajectory following accuracy and energy efficiency of AUVs in complex marine environments through optimal 3D tracking control strategies.

[0003] Under the optimal three-dimensional tracking control framework, autonomous underwater vehicles (AUVs) are often affected by uncertainties such as the external environment, sensor malfunctions, and sensor attacks, resulting in inaccurate position and attitude sensor measurement system states. This poses a significant challenge to improving system stability and reducing energy consumption. Among sensor attack methods, sparse sensor attacks, as a type of random sensor attack, target vulnerable sensors and interfere with sensor data. To overcome this difficulty, sensor redundancy strategies have emerged. Their core principle is to use a group of sensors to measure the same data and employ attack data isolation methods to eliminate excessively large or small sensor values ​​in descending order, then calculate the median or mean of the remaining sensor values ​​as the output value. However, sensor redundancy strategies cannot effectively handle unisolated Gaussian disturbances. Notably, continuous-time unscented Kalman filters perform exceptionally well in handling Gaussian interference, thanks to their unique non-tracking transform mechanism.

[0004] In summary, it is still necessary to address the issues of sensor susceptibility to interference, low tracking accuracy, and high control energy consumption in underactuated autonomous underwater vehicles in complex environments. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides an adaptive optimization control method and system for underactuated underwater vehicles.

[0006] To achieve the above objectives, the present invention provides an adaptive optimization control method for an underactuated underwater vehicle, comprising: Multiple sets of sensor data are collected in real time at different preset positions of the target underwater vehicle. The sensor data includes the spatial coordinates and attitude data of the target underwater vehicle at its current position.

[0007] For each group of sensor data, sort the data and use an attack isolation algorithm to remove the largest and smallest data in the sorted data to obtain preliminary clean data. Calculate the variance of the preliminary clean data and design an adaptive measurement covariance matrix based on the variance. Calculate the system state estimate of the preliminary clean data using an unscented Kalman filter. Update the system state estimate using the adaptive measurement covariance matrix and use the system state estimate at the current time at each location as the final state data.

[0008] A desired three-dimensional path is preset, and the tracking error in different directions is calculated using the desired three-dimensional path and the final state data. The backstepping method recalculates the virtual control quantity at the current moment based on the tracking error in different directions. The virtual control quantity includes the desired state angular velocity and the desired oscillation velocity.

[0009] The pre-trained reinforcement learning algorithm uses the final state data and virtual control variables at the current moment to plan the optimal control command; and uses the optimal control command to guide the target underwater vehicle to move along the desired three-dimensional path.

[0010] Preferably, the unscented Kalman filter employs a bounded measurement covariance matrix. The process of using the unscented Kalman filter to calculate the system state estimate of the preliminary cleanup data includes: Initialize the filtered state variables and covariance matrix, where the filtered state variables include the spatial coordinates and attitude data of the target underwater vehicle; the initial value of the covariance matrix is ​​based on the sensor's factory accuracy parameters. Based on the motion model of the target underwater vehicle and preliminary cleanup data, predict the state of the sampling point at the next moment; The residual between the state of the predicted sampling point at the next time step and the preliminary purified data is calculated, and the filtered state variable and the bounded measurement covariance matrix are updated using the residual. The system state estimate of the final output sensor data is calculated using the updated filtered state variable and the bounded measurement covariance matrix. The system state estimate is used as the final state data at the current time step to quantify the uncertainty of the state estimate of the bounded measurement covariance matrix.

[0011] Preferably, the tracking error includes longitudinal error, lateral error, and vertical error; an adaptive variable is introduced, and the desired tilt angle and desired azimuth angle are calculated based on the lateral error and vertical error; the desired attitude angle is determined by the desired tilt angle and desired azimuth angle.

[0012] Preferably, before devising the optimal control command, the method further includes: using a second-order sliding mode integral filter to obtain an estimate of the derivative signal of the virtual control quantity, and inputting the virtual control quantity and the estimate together into the reinforcement learning module to calculate the optimal control command.

[0013] Preferably, the pre-trained reinforcement learning algorithm adopts an execution-evaluation neural network architecture; wherein, the execution network is used to approximate the optimal control law and output the optimal control command; the evaluation network is used to approximate the performance index function to evaluate the system state and control effect; the weights of the execution network and the evaluation network are based on the gradient descent method and an adaptive update law is designed according to Lyapunov stability theory for online updating.

[0014] Preferably, the execution network and the evaluation network are radial basis function neural networks, used to estimate and approximate unknown nonlinear dynamics in the system.

[0015] Preferably, the different preset positions are determined by the motion trajectory planning of the target underwater vehicle, and include at least the starting position, cruising position, turning position and target approach position; the number of sensor data sets collected at each preset position is dynamically adjusted according to the underwater environmental interference intensity at that position, and the higher the environmental interference intensity, the more sensor data sets are collected.

[0016] The present invention also provides an adaptive optimization control method system for an underactuated underwater vehicle, comprising: The data acquisition module is used to collect multiple sets of sensor data at different preset positions of the target underwater vehicle in real time. The sensor data includes the spatial coordinates and attitude data of the target underwater vehicle at its current position.

[0017] The state estimation module sorts the sensor data for each group, removes the largest and smallest data from the sorted data using an attack isolation algorithm to obtain preliminary cleaned data, calculates the variance of the preliminary cleaned data, designs an adaptive measurement covariance matrix based on the variance, calculates the system state estimate of the preliminary cleaned data using an unscented Kalman filter, updates the system state estimate using the adaptive measurement covariance matrix, and uses the system state estimate at the current time at each location as the final state data, presets a desired three-dimensional path, and calculates the tracking error in different directions using the desired three-dimensional path and the final state data; and recalculates the virtual control quantity at the current time using a backstepping method based on the tracking error in different directions, the virtual control quantity including the desired state angular velocity and the desired oscillation velocity.

[0018] The control module is used to pre-train reinforcement learning algorithms to plan optimal control commands using the final state data and virtual control variables at the current moment; and to guide the target underwater vehicle to move along the desired three-dimensional path using the optimal control commands.

[0019] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the adaptive optimization control method for the underactuated underwater vehicle.

[0020] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the adaptive optimization control method for the underactuated underwater vehicle.

[0021] The adaptive optimization control method for underactuated underwater vehicles provided by this invention has the following beneficial effects: By collecting multiple sets of sensor data in real time at preset locations, and combining an attack isolation algorithm that sorts and removes the largest and smallest data with an unscented Kalman filter, the system not only utilizes the redundancy of multiple sets of data to initially isolate abnormal attack data, but also accurately estimates state values ​​through filtering, thereby improving the anti-interference capability and accuracy of state perception. Furthermore, by calculating multi-directional tracking errors using preset paths and final state data, and relying on backstepping to generate virtual control quantities that adapt to the errors, the system provides accurate feedback for control. Finally, by using a pre-trained reinforcement learning algorithm to fuse state data and virtual control quantities, the system outputs commands with optimal energy consumption and control performance while ensuring tracking accuracy, achieving a balance between stability, efficiency, and robustness in the three-dimensional tracking of the underwater vehicle. Attached Figure Description

[0022] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an adaptive optimization control method for an underactuated underwater vehicle according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the spatial reference coordinate system of an underactuated autonomous underwater vehicle according to an embodiment of the present invention; Figure 3 This is a system structure block diagram according to an embodiment of the present invention; Figure 4 This is a three-dimensional tracking space rendering of an underactuated autonomous underwater vehicle according to an embodiment of the present invention; Figure 5 This is a graph showing the tracking error in the longitudinal direction. Figure 6 This is a graph showing the tracking error in the lateral direction. Figure 7 This is a graph showing the tracking error in the vertical direction. Figure 8 The state estimation algorithm estimates the x-axis state of the underwater unmanned vehicle in the Earth coordinate system. Figure 9 The state estimation algorithm estimates the y-axis state of the underwater unmanned vehicle in the Earth coordinate system. Figure 10 The state estimation algorithm estimates the z-axis state of the underwater unmanned vehicle in the Earth coordinate system. Figure 11 The state estimation algorithm estimates the tilt angle state of the underwater unmanned vehicle in the Earth coordinate system. Figure 12 The result of the state estimation algorithm estimating the yaw angle state of the underwater unmanned vehicle in the Earth coordinate system; Figure 13 This is a convergence curve of the weight matrix norm of the execution neural network according to an embodiment of the present invention; Figure 14 This is a convergence curve of the norm of the weight matrix of the evaluation neural network in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0025] This invention designs a novel state estimation algorithm and combines it with optimal three-dimensional tracking control for in-depth research and application in underactuated autonomous underwater vehicles (AUVs). This approach has the potential to open up new control strategies and optimization paths under conditions of sparse sensor attacks and disturbances. By employing a sensor redundancy strategy, combined with attack isolation algorithms and continuous-time unscented Kalman filtering techniques, the state estimation algorithm is designed to address potential sparse sensor attacks and disturbances. With the help of reinforcement learning frameworks and path guidance principles, an optimal tracking controller for underactuated AUVs is proposed, ultimately solving the optimal tracking problem for underactuated AUVs.

[0026] like Figure 3As shown, the optimal three-dimensional tracking control system structure of an underactuated autonomous underwater vehicle (AUV) is illustrated. The data acquisition module measures the vehicle's position and state through multiple sensors. During the measurement process, it faces sparse sensor attacks launched by attackers, and the sensor output data is affected and disturbed by the attacks. Subsequently, relying on a sensor redundancy strategy and an attack isolation algorithm, the data containing the attacks is processed. Then, a state estimation module composed of a continuous-time unscented Kalman filter (UKF) accurately estimates the vehicle's position and state information. Based on this estimation result and the three-dimensional tracking method, the control module uses optimized backstepping technology to design the optimal controller. Finally, the actuator drives the vehicle to achieve stable tracking of the desired trajectory.

[0027] Based on this, the present invention provides an adaptive optimization control method for an underactuated underwater vehicle, specifically as follows: Figure 1 As shown, it includes: S1. Collect multiple sets of sensor data at different preset positions of the target underwater vehicle in real time. The sensor data includes the spatial coordinates and attitude data of the target underwater vehicle at its current position.

[0028] First, it is necessary to clarify some relevant definitions of coordinate systems, such as Figure 2 The diagram shows a schematic of the spatial reference coordinate system for an underactuated autonomous underwater vehicle. The Earth reference coordinate system is composed of... express, It is a fixed point on the Earth's surface. The axis points to the geographical north. The axis points towards the center of the Earth, while The axis is perpendicular to The plane, whose direction is determined by the right-hand rule. The self-reference coordinate system fixed to the object is represented as... The origin Located at the center of the volume of the underactuated autonomous underwater vehicle. (Axis) The tangent direction is consistent with the trajectory of the underactuated autonomous underwater vehicle. (Axis) It is determined by the right-hand rule, and the axis Perpendicular to Plane. The path reference coordinate system is specified as... ,origin Reference points on the path Location. Axis with path points The tangents at that point coincide. (Axis) It is determined according to the right-hand rule, and the axis Then perpendicular to the plane .

[0029] Secondly, the dynamic model of the underactuated autonomous underwater vehicle is established as follows: (1) Where, vector This represents the position and state vector of an underactuated autonomous underwater vehicle in the Earth coordinate system. The position information is represented as... Attitude angle information is represented as The symbols represent the surge locations in the Earth coordinate system. swaying position heave position Pitch angle and yaw angle Attitude angles (pitch angle, yaw angle) can be used to describe the azimuth angle of one's own coordinate system relative to the Earth reference coordinate system. The velocity state vector in its own reference coordinate system can be expressed as... Speed ​​information and angular velocity information The symbols correspond to "is" and "is". along The oscillation velocity along the axial direction, It is along Roll speed in the axial direction, It is along The axial heave velocity, and These are the rotational angular velocities for pitch and yaw, respectively. It is an unknown process interference vector, and satisfies . It is a positive symmetric transformation matrix, and satisfies , It is a known constant. and These represent the centripetal force matrix and the fluid damping matrix, respectively, as detailed below:

[0030] ; ; in, This represents the hydrostatic pressure vector. (Control vector) This includes the forces exerted by the underactuated autonomous underwater vehicle on its sway. Moments in the pitch and yaw directions and . This represents the rotation matrix from its own reference coordinate system to the Earth's reference coordinate system. Specific details are as follows:

[0031] (2) The sensor model subjected to sparse sensor attacks and measurement perturbations is established as follows. Each position state of the underactuated underwater autonomous vehicle system is determined by... Each sensor performs the measurement independently. Given that the sensor outputs are vulnerable to attack, and at the detection time... The acquired measurement data can be expressed as:

[0032] (3) in, yes dimensional vector, vector The elements in represent time. The measured value of each sensor at that time. It is a column vector. Represents the position state vector The Each element. Let the non-zero elements of the Gaussian perturbation vector be denoted as . It follows a Gaussian distribution with an expected value of 0.

[0033] S2. Sort the sensor data for each group, and use an attack isolation algorithm to remove the largest and smallest data in the sorted data to obtain preliminary purified data; calculate the variance of the preliminary purified data, and design an adaptive measurement covariance matrix based on the variance; calculate the system state estimate of the preliminary purified data using an unscented Kalman filter; update the system state estimate using the adaptive measurement covariance matrix, and use the system state estimate at the current time for each state position as the final state data; preset the desired three-dimensional path, and use the desired three-dimensional path and the final state data to calculate the tracking error in different directions; recalculate the virtual control quantity at the current time using the backstepping method based on the tracking error in different directions, and the virtual control quantity includes the desired state angular velocity and the desired oscillation velocity.

[0034] First, a sensor redundancy strategy is adopted. Assume... The number of non-zero elements is less than One, when The maximum number of damaged sensors when the number is odd is However . This represents the number of sensors that have not been attacked. (Symbol) Represents a vector The elements are sorted in descending order to form a new vector. . The mean and median are respectively and The variance is then expressed as . It is a vector The Each element. Indicates by Middle One to the first A new vector composed of elements.

[0035] The algorithm design content in Part 1 is as follows: in, This is the designed isolation threshold. This part of the algorithm performs "preprocessing" on the data set through a process of "sorting - bidirectional truncation filtering - statistically driven model selection," which can achieve the isolation of data with absolute values ​​greater than [a certain threshold]. The attack data is isolated.

[0036] Secondly, some preparations need to be made before designing the second part of the algorithm: First, the state model of the underactuated autonomous underwater vehicle is transformed into the following form: (4) in, , , . It's a speed input. and It is a Gaussian perturbation and satisfies and , , It is an unknown bounded measurement covariance matrix that satisfies ≼ , ≼ , and It is a positive definite constant matrix.

[0037] generate Sigma points, It is a state vector Dimensions The covariance matrix is ​​used, and then, through weighted summation of multiple nonlinear functions, the state estimate of the underactuated autonomous underwater vehicle is generated from the Sigma points. The Sigma points are defined as follows:

[0038] (5) No. The mean weight of each Sigma point and variance weight The definition is as follows: (6) in, and , , It is a normal number.

[0039] The second part of the algorithm for design state estimation is as follows: in, , , It is a positive number. It is the designed covariance threshold. It is the upper limit of the adaptive covariance of the design. It is the covariance matrix, used to quantify the uncertainty of state estimation and update the state estimate.

[0040] The equations in the above algorithm can be written as: (7) in .

[0041] The second part of the algorithm "reprocesses" the sensor data "preprocessed" in the first part. First, a new adaptive covariance matrix is ​​designed based on the variance from the first part:

[0042] (8) By further processing using the unscented transformation characteristics of the unscented Kalman filter, the Gaussian perturbation problem in the estimated value can be effectively handled, and relatively accurate estimated data can be obtained. The specific effect will be shown in the simulation results.

[0043] According to (1), (2) and (4), Suppose there exists a compact set. ,and , The Jacobian matrix can be described as:

[0044] (9) According to known With the boundedness of velocity, there exists a constant. , making and .

[0045] Using the L2 norm property, we can obtain ,in express The largest eigenvalue. Therefore, exist The is Lipschitz continuous, and the Lipschitz constant is .

[0046] Define the estimation error as Let's choose a Lyapunov equation as: (10) Designed Sigma Points Distributed in Therefore, the following formula can be obtained from the embodiments of the present invention. , and . The derivative can be written as:

[0047] (11) because It is Lipschitz continuous, and according to the Cauchy-Schwarz inequality, it can be approximated as: (12) Using Young's inequality, we can obtain: (13) in, , It is a positive constant; according to the state estimation algorithm, it can be obtained... , , and measurement matrix They are all full-rank matrices. Therefore and They are all positive definite matrices. and yes and The largest eigenvalue.

[0048] Secondly The derivative can be further written as: (14) and yes and The smallest eigenvalue.

[0049] In addition, the measurement variance needs to be... Adjust the parameters to a suitable level to meet the conditions. .

[0050] S3. The pre-trained reinforcement learning algorithm uses the final state data and virtual control variables at the current moment to plan the optimal control command; and uses the optimal control command to guide the target underwater vehicle to move along the desired three-dimensional path.

[0051] Due to measurement disturbance and It is white noise with a mean of 0, therefore the Lyapunov equation It is inherently a stochastic process. This invention analyzes its stability from a mean-square perspective by calculating the expected value of the derivative of the Lyapunov equation:

[0052] (15) in , and . and This represents expectation and trace. This proves... The boundedness of the proposed state estimation algorithm means that the estimation error is bounded.

[0053] The desired trajectory is as follows: ,in The derivative can be designed as follows: (16) in, It is a positive constant. (Azimuth) Defined as and exist The angle between projections in the plane. The tilt angle of the trajectory. yes and The angle between the planes. Therefore, the desired yaw angle is... The desired pitch angle is .

[0054] The attitude angles of the desired trajectory in the Earth reference coordinate system can be expressed as follows: (17) in, , , .

[0055] The rotation matrix of the reference trajectory from the desired trajectory reference coordinate system to the Earth reference coordinate system can be defined as follows: (18) The rotation matrix of the reference trajectory from the desired trajectory reference coordinate system to the Earth reference coordinate system can be defined as follows: (19) According to calculations, the embodiments of the present invention can yield the following: (20) In the desired trajectory reference coordinate system, the desired trajectory velocity can be expressed as: Therefore, the motion nonlinear model equation for the required path can be expressed as: ;(twenty one) and The error vector between them can be represented in the following form: ;(twenty two) in, , and These represent longitudinal error, lateral error, and vertical error, respectively.

[0056] Based on the principle of guided path following, design a method for addressing tracking error. Lyapunov function Continuing to differentiate this Lyapunov function, we obtain the following equation:

[0057] ;(twenty three) Can Designed for ,in Designed as follows: According to Lyapunov stability theory, the required desired orbital azimuth angle and desired tilt angle It can be designed as follows: ;(twenty four) in, , , and It is a positive constant. It is expressed as the expected minimum oscillation velocity along the desired three-dimensional path.

[0058] In order to satisfy the orbital inclination angle It needs to meet the following requirements. Therefore, adaptive variables Designed as follows: (25) in, It is a constant less than 1.

[0059] Then, the expected speed Designed as: (26) Substituting the above formula into the derivative of the Lyapunov function... We can obtain the following formula: (27) in, .

[0060] The desired oscillation speed can be obtained based on tracking control. and desired attitude angle Using the state estimation algorithm designed above, the embodiments of the present invention can obtain... By combining the backstepping method, attitude stabilization loops and velocity stabilization loops can be designed.

[0061] First, design the attitude stabilization loop, which requires defining the attitude angle error vector. and angular velocity error vector for: (28) in, It is a virtual controller.

[0062] To avoid the inherent complexity explosion problem of backstepping control technology, this paper adopts a second-order sliding mode integral filter, the design of which is as follows: (29) in, and These are the filtering states, and For input. , , , , and All are positive design constants. The derivative can be expressed as ,in It is the filtering error and satisfies , .

[0063] Step 1: Construct the Lyapunov function with the angle error vector: (30) Based on the state estimation algorithm described above, and based on the AUV dynamics model, The derivative can be expressed as: (31) Virtual Controller Designed as: (32) in, It is a designed positive constant matrix, and its smallest eigenvalue is .

[0064] but Continue writing as: (33) Step 2: Estimating the nonlinear continuous function using a radial basis function neural network. , Its content is as follows: (34) in, It is an ideal weight matrix, where , It is a Gaussian function vector. , This represents the approximate error vector. It represents the number of nodes in the neural network. ,in It is the weight vector of an ideal neural network. The estimated value. A relationship exists. ,in This represents the weight error vector of the neural network.

[0065] Therefore, the design The adaptive law is: (35) in, It is a normal number.

[0066] To design another Lyapunov equation: (36) Based on the mathematical model of the underactuated autonomous underwater vehicle, the derivative of this equation can be written as: (37) in, , , , , .

[0067] Secondly, a velocity stabilization loop is designed. Based on the tracking scheme in this embodiment of the invention, the desired oscillation velocity can be obtained. Therefore, by using backstepping techniques, the oscillation velocity error can be obtained. Design a Lyapunov function with oscillation velocity error. :

[0068] (38) By differentiating it and using the dynamic model of the AUV, the present invention embodiment yields: (39) Finally, the controller was designed and optimized, as detailed below: To address the optimization problem of underactuated autonomous underwater vehicles, a performance function is designed. for: (40) For convenience, the velocity error vector is rewritten as follows: The ideal controller is rewritten as ,as well as , and A positive symmetric constant matrix for dimension matching.

[0069] For the cost function and ideal controller The following HJB equations are defined: (41) in, , .

[0070] Due to the optimal solution It is entirely determined by the optimal performance index function, therefore it necessarily corresponds to a unique solution to the HJB equation. This is assuming the optimal control scheme satisfies the first-order optimality condition. Optimal control input It can be described as:

[0071] (42) Among them, gradient term The definition of is: (43) in, , .

[0072] design for: (44) in, , and They are respectively , , The output after passing through a second-order sliding mode integral filter.

[0073] Using radial basis function neural networks, embodiments of the present invention can... Approximately: (45) in, For the ideal weight matrix, is the basis function vector.

[0074] To obtain a feasible optimized control scheme, the embodiments of the present invention design the following actuator-evaluation architecture: (46) in It is the optimal controller The estimated value, and These are the weight matrices for the evaluative neural network and the execution neural network, respectively.

[0075] Therefore, the weight matrix of the execution neural network and the weight matrix of the evaluation neural network The adaptive law is designed as follows: (47) in and All are positive numbers, and satisfy... and .

[0076] To prove that the proposed controller and adaptive law can theoretically stabilize the system, the following proof is provided: Choose a general Lyapunov equation as follows: (48) in, Represents a set .

[0077] Substituting the adaptive law (35), controller (46), and optimal weight adaptive law (47) designed above, we can obtain The derivative is: (49) in, , , .

[0078] Using relationships , From Young's inequality, we can obtain: (50) (51) (52) To avoid the complexity explosion problem, this paper adopts a second-order sliding mode integral filter. It can be rewritten as The expression is bounded and satisfies ,in It is a positive constant.

[0079] Similarly, It can also be rewritten as and satisfy , It is a normal number.

[0080] Using Young's inequality and the definition of a second-order sliding mode integral filter, we can obtain: (53) in, , and It is a normal number.

[0081] Substitute (54) into the equation. The derivative can be further written as: (54) in, , , , . Represented as Minimum eigenvalue. Because it satisfies the condition. and Therefore, the inequality can be further written as:

[0082] (55) in, .

[0083] To demonstrate the feasibility, effectiveness, and correctness of this example, the present invention conducts the following simulation experiments: In this simulation experiment, for an underactuated autonomous underwater vehicle system with sparse sensor attacks and measurement disturbances, an optimal three-dimensional tracking controller based on state estimation algorithm and reinforcement learning is designed to achieve optimal three-dimensional tracking control of the underactuated autonomous underwater vehicle.

[0084] In the optimal controller design process, the system model parameters are set as follows: Judging neural networks and execution neural network All include =7 neurons, the center point of the neural network is evenly distributed in Above, the width is: The initial weights of the neural network are: , =0.5, =0.3, .

[0085] Relevant parameters of the underactuated autonomous underwater vehicle model: , , , , , , , , , , , .

[0086] The initial state of the underactuated autonomous underwater vehicle model is as follows: , .

[0087] The relevant parameters for the state estimation algorithm based on sensor redundancy strategy and Kalman filter are as follows: , , , , , , , , .

[0088] The relevant parameters for designing the controller are: , , , , , .

[0089] The relevant parameters for tracking control are: , , , , .

[0090] The relevant parameters for the second-order sliding mode integral filter are: , , .

[0091] The effectiveness of this simulation is further illustrated by referring to the accompanying diagram: Simulation results are as follows Figures 4-14 As shown. Figure 4 This diagram illustrates the movement trajectory of the AUV in a three-dimensional environment. The solid blue line represents the underwater vehicle's path, while the dashed red line represents the desired path. It can be seen that the actual path indeed follows the desired three-dimensional path. Longitudinal, lateral, and vertical errors are represented by the solid blue lines. Figure 5 , Figure 6 and Figure 7 As shown, all solid lines eventually converge to the neighborhood of 0. Figure 8 , Figure 9 , Figure 10 , Figure 11 and Figure 12 The measurements from each sensor and the state estimated using the state estimation algorithm described in this paper are displayed. The estimated value includes both attacks and disturbances. The green dashed line represents the state value measured by each sensor; it can be observed from the figure that some sensors have been attacked, while others have been disturbed. The blue dashed line represents the true ideal state value. The state estimation algorithm used in this embodiment of the invention obtains the state estimate value as shown by the black line in the figure, which can be observed to achieve an approximate value. Figure 13 and Figure 14 The weight matrices of the execution neural network are shown respectively. Norm and weight matrix of a judgment neural network The norms of the weights are shown by the blue solid line, and it can be seen that all estimated weight norms eventually converge to the neighborhood of 0.

[0092] In summary, all signals in the system are uniformly bounded, and simulation results demonstrate the effectiveness of the proposed optimal control scheme.

[0093] This embodiment uses backstepping recursion and reinforcement learning techniques as its design framework to address the optimal 3D tracking control problem for underactuated autonomous underwater vehicles (AUVs) with unknown perturbations. It designs a state estimation algorithm based on a continuous-time unscented Kalman filter and attack isolation, employing a sensor redundancy strategy. This algorithm can estimate relatively accurate position and state information even under sparse sensor attacks and measurement perturbations. Subsequently, using the estimated state information, a guided path-following principle is adopted, and a new expected error angle is proposed to design a tracking control scheme for underactuated AUVs. Within the framework of reinforcement learning, an optimal 3D tracking controller is constructed, ensuring that all signals are semi-globally eventually bounded. The widespread application of this invention to optimal 3D tracking control of underactuated AUVs represents an important future research direction.

[0094] Based on the same inventive concept, this invention also provides an adaptive optimization control method system for an underactuated underwater vehicle, comprising: The data acquisition module is used to collect multiple sets of sensor data in real time at different preset positions of the target underwater vehicle; the sensor data includes the coordinates, attitude and operating speed of the target underwater vehicle.

[0095] The state estimation module sorts multiple groups of sensor data, uses an attack isolation algorithm to remove the largest and smallest data in the sorted data to obtain preliminary cleaned data, calculates the mean and variance of the preliminary cleaned data using an unscented Kalman filter, uses the mean and variance of multiple groups of sensor data at each location as the final state data at the current moment, presets a desired three-dimensional path, and calculates the tracking error in different directions using the desired three-dimensional path and the final state data at the current moment. The backstepping method recalculates the virtual control quantity at the current moment based on the tracking error in different directions, the virtual control quantity including the desired state angle and desired velocity.

[0096] The control module is used to pre-train reinforcement learning algorithms to plan optimal control commands using the final state data and virtual control variables at the current moment; and to guide the target underwater vehicle to move along the desired three-dimensional path using the optimal control commands.

[0097] This invention also provides a computer device, which, at the hardware level, includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the adaptive optimization control method for the underactuated underwater vehicle provided above.

[0098] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the adaptive optimization control method for the underactuated underwater vehicle provided above.

[0099] Specific limitations on the computational system for the adaptive optimization control method of underactuated underwater vehicles can be found in the limitations of the adaptive optimization control method for underactuated underwater vehicles mentioned above, and will not be repeated here. Each module in the aforementioned adaptive optimization control method system for underactuated underwater vehicles can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0100] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An adaptive optimization control method for an underactuated underwater vehicle, characterized in that, include: Multiple sets of sensor data at different preset positions of the target underwater vehicle are collected in real time. The sensor data includes the spatial coordinates and attitude data of the target underwater vehicle at its current position. The sensor data for each group is sorted, and the largest and smallest data in the sorted data are removed using an attack isolation algorithm to obtain preliminary clean data; the variance of the preliminary clean data is calculated, and an adaptive measurement covariance matrix is ​​designed based on the variance. The system state estimate of the preliminary cleanup data is calculated using unscented Kalman filtering. The system state estimate is updated using the adaptive measurement covariance matrix, and the system state estimate at the current time at each state position is used as the final state data. A desired three-dimensional path is preset, and the tracking error in different directions is calculated using the desired three-dimensional path and the final state data. The backstepping method recalculates the virtual control quantity at the current moment based on the tracking error in different directions. The virtual control quantity includes the desired state angular velocity and the desired oscillation velocity. The pre-trained reinforcement learning algorithm uses the final state data and virtual control variables at the current moment to plan the optimal control command; and uses the optimal control command to guide the target underwater vehicle to move along the desired three-dimensional path. Also includes: The tracking error includes longitudinal error, lateral error, and vertical error; an adaptive variable is introduced, and the desired tilt angle and desired azimuth angle are calculated based on the lateral error and vertical error; the desired attitude angle is determined by the desired tilt angle and desired azimuth angle. Before devising the optimal control command, the process also includes: using a second-order sliding mode integral filter to obtain an estimate of the derivative signal of the virtual control quantity, and inputting the virtual control quantity and the estimate together into the reinforcement learning module to calculate the optimal control command; The adaptive variable k p The expression is as follows: ； in, u The sway speed of the target underwater vehicle; a s A constant less than 1; e p This refers to vertical error; w The heave speed; k e It is a positive constant.

2. The adaptive optimization control method for an underactuated underwater vehicle according to claim 1, characterized in that, The unscented Kalman filter employs a bounded measurement covariance matrix. The process of calculating the system state estimate of the preliminary cleanup data using the unscented Kalman filter includes: Initialize the filtered state variables and covariance matrix, where the filtered state variables include the spatial coordinates and attitude data of the target underwater vehicle; the initial value of the covariance matrix is ​​based on the sensor's factory accuracy parameters. Based on the motion model of the target underwater vehicle and preliminary cleanup data, predict the state of the sampling point at the next moment; The residual between the state of the predicted sampling point at the next time step and the preliminary purified data is calculated, and the filtered state variable and the bounded measurement covariance matrix are updated using the residual. The system state estimate of the final output sensor data is calculated using the updated filtered state variable and the bounded measurement covariance matrix. The system state estimate is used as the final state data at the current time step to quantify the uncertainty of the state estimate of the bounded measurement covariance matrix.

3. The adaptive optimization control method for an underactuated underwater vehicle according to claim 1, characterized in that, The pre-trained reinforcement learning algorithm adopts an execution-evaluation neural network architecture; wherein, the execution network is used to approximate the optimal control law and output the optimal control command; the evaluation network is used to approximate the performance index function to evaluate the system state and control effect; the weights of the execution network and the evaluation network are based on the gradient descent method and an adaptive update law is designed based on Lyapunov stability theory for online updating.

4. The adaptive optimization control method for an underactuated underwater vehicle according to claim 3, characterized in that, The execution network and the evaluation network are radial basis function neural networks, used to estimate and approximate unknown nonlinear dynamics in the system.

5. The adaptive optimization control method for an underactuated underwater vehicle according to claim 1, characterized in that, The different preset positions are determined by the motion trajectory planning of the target underwater vehicle, and include at least the starting position, cruising position, turning position and target approach position; the number of sensor data sets collected at each preset position is dynamically adjusted according to the underwater environmental interference intensity at that position. The higher the environmental interference intensity, the more sensor data sets are collected.

6. A system for implementing the adaptive optimization control method for the underactuated underwater vehicle of claim 1, characterized in that, include: The data acquisition module is used to collect multiple sets of sensor data from different preset positions of the target underwater vehicle in real time. The sensor data includes the spatial coordinates and attitude data of the target underwater vehicle at its current position. The state estimation module is used to sort the sensor data for each group, use an attack isolation algorithm to remove the largest and smallest data in the sorted data to obtain preliminary clean data; calculate the variance of the preliminary clean data, and design an adaptive measurement covariance matrix based on the variance; The system state estimate of the preliminary cleanup data is calculated using unscented Kalman filtering. The system state estimate is updated using the adaptive measurement covariance matrix, and the system state estimate at the current time at each location is used as the final state data. A desired three-dimensional path is preset, and the tracking error in different directions is calculated using the desired three-dimensional path and the final state data. The backstepping method recalculates the virtual control quantity at the current moment based on the tracking error in different directions. The virtual control quantity includes the desired state angular velocity and the desired oscillation velocity. The control module is used to pre-train reinforcement learning algorithms to plan optimal control commands using the final state data and virtual control variables at the current moment; and to guide the target underwater vehicle to move along the desired three-dimensional path using the optimal control commands.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 5.

Images