Obstacle avoidance control method for tracked vehicle based on barrier function and reinforcement learning

Abstract

The present application belongs to the technical field of intelligent control and amphibious robot, especially relates to a control method for tracking and obstacle avoidance of amphibious tracked vehicle based on control barrier function and reinforcement learning, which comprises the following steps: establishing a dynamic mathematical model of amphibious tracked vehicle in different environments; introducing a pre-aiming point mechanism to convert the position obstacle avoidance constraint into a speed constraint; deriving a safety constraint condition based on the control barrier function, and solving a safety reference speed with the nominal reference speed as the optimization goal; taking the safety reference speed as the tracking target, and using a reinforcement learning network to estimate unknown disturbances online and output a control law; detecting the water entry state and executing water-land mode switching, switching the corresponding dynamic model and network parameters; and iteratively executing until the tracking and obstacle avoidance task is completed, so that the present application realizes the unification of real-time global optimal obstacle avoidance and high-precision trajectory tracking in complex cross-domain environments.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control and amphibious robot technology, and particularly relates to an amphibious tracked vehicle tracking and obstacle avoidance control method using obstacle control functions and reinforcement learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of marine development and emergency rescue, specialized mobile platforms capable of traversing complex terrain and water obstacles have become a research hotspot. Traditional wheeled vehicles and ships are limited to a single operating medium, making it difficult to meet the needs of continuous cross-domain operations. Amphibious tracked vehicles, through the combination of tracked structures and buoyancy propulsion devices, possess both high land mobility and water navigation capabilities, enabling continuous operations in multiple scenarios such as land, shallow waters, and aquatic areas. They have irreplaceable advantages in tasks such as flood relief and nearshore surveying.

[0004] However, it faces significant challenges in autonomous control: in land mode, the vehicle can only steer by differential speed of the left and right tracks, and the lateral speed is uncontrollable. Traditional methods directly apply positional constraints to the center of the vehicle body, but the derivative of this constraint cannot be mapped to the only two control inputs, longitudinal speed and angular speed, resulting in a relative degree mismatch problem.

[0005] Furthermore, existing obstacle avoidance methods mostly remain at the level of path replanning, replanning the path upon encountering an obstacle. This not only results in poor real-time performance and difficulty in handling dynamic obstacles, but also makes them prone to getting trapped in local optima. Meanwhile, amphibious tracked vehicles are affected by ground friction and track slippage on land, and by hydrodynamics and wave disturbances in water. These time-varying factors are difficult to model accurately, and the accumulation of model errors can easily lead to control instability. Sudden model changes during mode switching can also cause control shocks. More critically, existing technologies often treat safety and stability separately, failing to theoretically unify obstacle avoidance safety and tracking stability. This results in the inability to guarantee dynamic stability during obstacle avoidance, and even stable tracking cannot guarantee collision avoidance. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides an amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function control and reinforcement learning. The method aims to open up the channel between the safety constraints and control input of the underactuated system through the pre-aiming point mechanism, transform the obstacle avoidance problem into a quadratic programming problem using CBF-QP to ensure real-time performance and global optimality, employ three-layer reinforcement learning to estimate dynamic unknowns online to enhance robustness, and combine Lyapunov stability analysis to ensure the stability of the system during the obstacle avoidance process.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a method for controlling the tracking and obstacle avoidance of an amphibious tracked vehicle using a control obstacle function and reinforcement learning is disclosed, including: Establish kinematic and dynamic mathematical models of amphibious tracked vehicles in under-actuated land mode and full-actuated water mode; A pre-aiming point mechanism is introduced to establish a Jacobian mapping relationship between the pre-aiming point speed and the speed of the tracked vehicle itself; real-time vehicle status and environmental obstacle information are obtained, and safety constraints are derived based on the control obstacle function; the desired speed under obstacle-free conditions, i.e., the nominal reference speed, is used as the optimization objective, and the safety reference speed is solved based on the safety constraints and speed amplitude constraints. Using the safe reference speed as the tracking target, a reinforcement learning network is used to estimate unknown disturbance terms online, and the output control law is applied to the amphibious tracked vehicle to make the actual speed track the safe reference speed. The system monitors the vehicle's water entry status in real time. When the switching conditions are met, it performs a water-land mode switch, switching the corresponding dynamic model and reinforcement learning network parameters. The above process is repeated until the trajectory tracking and obstacle avoidance tasks are completed.

[0008] Furthermore, the aiming point is set at the front of the vehicle body, and its position is determined by the center position of the vehicle body and the aiming distance.

[0009] Furthermore, the derivation of safety constraints based on the control obstacle function includes: constructing a candidate control obstacle function based on the difference between the distance from the pre-aiming point to the obstacle center and the safety distance margin, and defining the safety set as the state region where the value of the candidate control obstacle function is greater than or equal to zero; Based on the forward invariance condition of the control obstacle function, and combined with the derivative of the control obstacle function along the system trajectory and the mapping relationship of the aiming point velocity, a linear inequality constraint with respect to the control input is obtained.

[0010] Furthermore, the linear inequality constraint includes, in land mode, the linear inequality constraint relating to longitudinal velocity and yaw rate; In the water mode, the linear inequality constraints involve the sway velocity, roll velocity, and yaw rate. When there are multiple obstacles in the environment, the constraints corresponding to all obstacles are merged to obtain global linear inequality constraints.

[0011] Furthermore, the solution for the safe reference speed includes: constructing an optimization objective based on minimizing the L2 norm of the difference between the safe reference speed and the nominal reference speed, and constructing a convex quadratic programming problem using the safety constraints of the control barrier function and the speed amplitude constraints as constraints. The safe reference speed with the smallest deviation from the nominal reference speed is obtained by solving the convex quadratic programming problem based on the interior point method. When the convex quadratic programming problem becomes unsolvable due to constraint conflicts, it reverts to the nominal reference speed and triggers path replanning.

[0012] Furthermore, the reinforcement learning network includes an identifier network, a critic network, and an actor network; the identifier network takes the system state as input and outputs an estimate of the unknown dynamics. The critic network takes the system state and velocity tracking error as input and outputs an estimate of the optimal cost function. The actor network generates a control law based on the outputs of the identifier network and the critic network. The control law includes a proportional feedback term, an unknown term compensation term, and an optimal control term. The three networks are updated simultaneously online to achieve adaptive compensation for unknown disturbances; A Lyapunov function incorporating velocity tracking error and the estimation errors of three neural network weights is constructed to ensure the stability of the amphibious tracked vehicle during obstacle avoidance.

[0013] Furthermore, the water-land mode switching adopts a water pressure triggering mechanism. When the water pressure is detected to exceed a preset threshold and the switching has not yet been completed, the current control input value is frozen, and within a preset sampling period, the parameters of the land dynamics model and reinforcement learning network are gradually transitioned to the water model and parameters in a weighted average manner.

[0014] Secondly, a tracking and obstacle avoidance control system for an amphibious tracked vehicle based on a control obstacle function and reinforcement learning is disclosed, including: The model building module is configured to: establish kinematic and dynamic mathematical models of amphibious tracked vehicles in land under-drive mode and water full-drive mode. The aiming point conversion module is configured to: introduce an aiming point mechanism and establish a Jacobian mapping relationship between the aiming point speed and the tracked vehicle's own speed; The safety constraint generation module is configured to: acquire real-time vehicle status and environmental obstacle information, and derive safety constraint conditions based on the control obstacle function; The safe reference speed solution module is configured to: use the desired speed under unobstructed working conditions, i.e., the nominal reference speed, as the optimization objective, and solve for the safe reference speed based on safety constraints and speed amplitude constraints. The reinforcement learning control module is configured to: take the safe reference speed as the tracking target, use a reinforcement learning network to estimate unknown disturbance terms online, and output a control law to act on the amphibious tracked vehicle so that the actual speed tracks the safe reference speed; The mode switching module is configured to: detect the vehicle's water entry status in real time, and execute the water-land mode switching when the switching conditions are met, switching the corresponding dynamic model and reinforcement learning network parameters. The iterative execution module is configured to repeatedly execute the functions of the above modules until the trajectory tracking and obstacle avoidance tasks are completed.

[0015] Thirdly, a computer device is disclosed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the steps of the method described above.

[0016] Fourthly, a computer-readable storage medium is disclosed having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0017] The above one or more technical solutions have the following beneficial effects: This invention introduces a pre-aiming point mechanism, sets up a virtual pre-aiming point in front of the vehicle body and establishes a Jacobian mapping relationship between its speed and the vehicle body speed, transforming the vehicle body position obstacle avoidance constraint into a pre-aiming point speed constraint that can be directly adjusted by the control input, fundamentally solving the problem of relative degree mismatch between control input and position constraint in land underactuated mode.

[0018] This invention abandons the traditional path replanning approach of "replanning upon encountering an obstacle" and uses the CBF-QP safety filter to transform the obstacle avoidance safety requirements into a convex quadratic programming problem. With the nominal reference speed as the optimization objective, it solves for the safe reference speed that minimizes the deviation from the nominal speed under the premise of satisfying safety constraints and speed amplitude constraints. This achieves real-time solution and global optimality for obstacle avoidance, overcoming the shortcomings of existing methods such as large computational load, poor real-time performance, and easy getting trapped in local optima.

[0019] This invention employs a three-layer reinforcement learning architecture consisting of an identifier network, a critic network, and an actor network. Through the synchronous online updates of the three networks, it achieves adaptive compensation for time-varying unknown disturbances such as ground friction and hydrodynamics. It does not rely on an accurate prior dynamic model, significantly improving the robustness of the system in complex cross-land and water environments and avoiding the control instability problem caused by the accumulation of model errors in traditional methods.

[0020] This invention achieves a continuous and smooth transition between land mode and water mode through a seamless switching mechanism triggered by water pressure. Combined with the construction of a Lyapunov function that includes speed tracking error and neural network weight estimation error, it theoretically proves that the speed tracking error is eventually bounded and the system trajectory is always within the safe set. This achieves the unity of obstacle avoidance safety and trajectory tracking stability, and makes up for the theoretical defect of existing technologies that treat the two separately.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is a flowchart of the obstacle control function and reinforcement learning-based amphibious tracked vehicle tracking and obstacle avoidance control method according to Embodiment 1 of the present invention. Figure 2 This is a logical framework diagram of the CBF-QP safety filter in the amphibious tracked vehicle tracking and obstacle avoidance control method of reinforcement learning in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the architecture of a three-layer reinforcement learning controller in the amphibious tracked vehicle tracking and obstacle avoidance control method of the control obstacle function and reinforcement learning in Embodiment 1 of the present invention; Figure 4 This is a diagram of the amphibious tracked vehicle tracking and obstacle avoidance control system architecture based on the obstacle control function and reinforcement learning in Embodiment 2 of the present invention. Detailed Implementation

[0024] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0026] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] Example 1 like Figure 1 As shown, this embodiment discloses an amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning, including: Step 1: Establish kinematic and dynamic mathematical models of the amphibious tracked vehicle in under-actuated land mode and fully actuated water mode; To uniformly describe the vehicle's motion in different scenarios, two orthogonal right-handed coordinate systems are defined in this embodiment: Among them, the world coordinate system To ensure the mission is fixed in the ground-based inertial frame, the origin can be chosen as the mission's starting point. The axis points in the direction the mission is progressing. Axis perpendicular to The axis to the left is used to describe the vehicle's absolute position, heading angle, and global trajectory.

[0028] Vehicle coordinate system The center point coincides with the vehicle's center of gravity. The axle points forward along the longitudinal axis of symmetry of the vehicle body. The axle points to the left along the side of the vehicle body. The axis is vertically upward and satisfies the right-hand rule, which is used to describe the relative speed, forces, and control inputs of a vehicle.

[0029] Based on the coordinate system described above, the complete state vector of the system is defined.

[0030] Position and heading status in the world coordinate system ,in , Let x and y be the coordinates of the vehicle's center of gravity in the world coordinate system. The vehicle's heading angle, i.e., the vehicle's... Axis and the World The included angle of the axis is positive when it is counterclockwise.

[0031] Velocity state in the vehicle coordinate system ,in For longitudinal velocity, along Axial forward is positive. For lateral velocity, along The axis to the left is positive. Angular velocity, around The axis is positive when rotated counterclockwise.

[0032] Based on the above definition of coordinate system and state vector, a mathematical model of amphibious tracked vehicle under typical operating scenarios is further constructed.

[0033] For land driving conditions, a mathematical model of the land mode is constructed based on underactuated characteristics. When the amphibious tracked vehicle travels on land, it relies on the friction between the left and right tracks and the ground to generate traction. Its core characteristic is nonholonomic constraint: because the lateral adhesion between the tracks and the ground is much greater than the longitudinal adhesion, the vehicle cannot generate active lateral movement, and the lateral velocity... It can only steer by the differential speed of the left and right tracks, so the land mode is a typical 2-input 2-output underactuated system.

[0034] To describe the mapping relationship between position and heading states and velocity states, after considering nonholonomic constraints, the kinematic equations for land driving conditions simplify to:

[0035] in This is the effective control input vector in land mode. The equation shows that the vehicle's longitudinal displacement is determined by its longitudinal velocity. and heading angle The heading angle is determined jointly by angular velocity alone. Decide.

[0036] The dynamic model describes the mapping relationship between velocity state and control input (driving torque of left and right tracks), considering the traction force of left and right tracks, longitudinal rolling resistance, and track rotation resistance, and establishes the complete Lagrange dynamic equations:

[0037] in, , For vehicle body mass; , The longitudinal resistance of the left and right tracks; , For turning resistance torque, , It is the moment of inertia; ; ; The control input vector represents the driving force of the left and right tracks.

[0038] By introducing the transformation matrix By projecting the dynamic equations in the generalized coordinate system onto the velocity space and eliminating nonholonomic constraint terms, we obtain the reduced-order dynamic equations:

[0039] in: ; ; ; ; ; For water-based driving conditions, a mathematical model for the water-based driving mode is constructed based on the full-drive characteristics. When the vehicle is fully immersed in water, the tracks lose contact with the ground, and thrust and torque are generated by the propeller. At this time, the vehicle can produce motion in three directions: longitudinal, lateral, and yaw. Therefore, the water-based driving mode is a 3-input, 3-output full-drive system. The kinematic equations for water-based driving conditions are:

[0040] in, This represents the effective control input vector in the water-based model. Considering the added mass of the water body, Coriolis force and centrifugal force, hydrodynamic damping, etc., the dynamic equations are established as follows:

[0041] in, and The additional mass along the longitudinal and lateral directions of the vehicle body when moving in water; For rotational inertia, Add a moment of inertia to the vehicle body; , , For hydrodynamics and torque; The thrust generated by the propeller; , , The hydrodynamic force and torque generated for steering control input.

[0042] By establishing the above mathematical model, the mapping relationship between the vehicle's input and output is rigorously characterized, enabling the subsequent control law design to be carried out based on a clear model. At the same time, it provides a clear object to be processed for the introduction of disturbance observers, robust compensation, or adaptive mechanisms.

[0043] Step 2: Introduce a pre-aiming point mechanism and establish a Jacobian mapping relationship between the pre-aiming point speed and the tracked vehicle's own speed; Traditional obstacle avoidance methods directly impose constraints on the vehicle's center position. However, for land-based underactuated systems, the lateral position of the vehicle's center cannot be directly controlled, resulting in the derivative of the position constraint not being able to map to the only two control inputs. In other words, there is a mismatch between the relative degrees of the control inputs and the position constraints.

[0044] To address this issue, this embodiment proposes a distance from the front of the vehicle body. Define a virtual "pre-aiming point". :

[0045] in The center position of the vehicle body This refers to the aiming distance. The physical meaning of the aiming point is to shift the control target from the current center of the vehicle to the position in front of the vehicle, which is equivalent to the driver controlling the vehicle by observing the road ahead.

[0046] Taking the derivative with respect to the aiming point position, we obtain a linear mapping relationship between the aiming point velocity and the vehicle's own velocity: For land mode:

[0047] in Given a 2×2 invertible matrix, its determinant is... This proves that the two-dimensional velocity of the aiming point can be controlled by two vehicle control inputs. and Adjustable at will.

[0048] For water-based models:

[0049] in Given a 2×3 full-rank matrix, it is proven that the two-dimensional velocity of the aiming point can be arbitrarily adjusted through the vehicle's three control inputs.

[0050] By using the pre-aiming point mechanism, the vehicle center position constraint, which was originally impossible to execute directly, is transformed into the pre-aiming point speed constraint, which can be executed directly, thus mathematically achieving a relative degree match between safety constraints and control inputs.

[0051] Step 3: Obtain real-time vehicle status and environmental obstacle information, and derive safety constraints based on the control obstacle function; The real-time vehicle status includes: the vehicle's position coordinates in the world coordinate system. Heading angle and longitudinal velocity in the vehicle coordinate system lateral velocity With angular velocity The aforementioned real-time vehicle status information is obtained through the collection and fusion estimation of the GPS / IMU integrated navigation system and wheel speed encoder mounted on the vehicle.

[0052] Environmental obstacle information includes: the center coordinates of obstacles in the environment. and obstacle radius The aforementioned environmental obstacle information is obtained through real-time scanning and detection using a lidar system mounted on the vehicle.

[0053] Based on the Control Barrier Function (CBF) theory, obstacle avoidance safety requirements are transformed into linear inequality constraints on the control input, providing hard constraints for the subsequent CBF-QP safety filter. The core idea of ​​the control barrier function is forward invariance: if the initial state of the system is within the safe set and satisfies the CBF condition, the system state will always remain within the safe set, and collisions will not occur.

[0054] In the specific implementation process, it is assumed that the environment contains The first circular obstacle, the position of which can be detected by lidar, is the [missing information]. The center position of each obstacle is , radius is To allow for a safety margin, the safety distance margin is defined as follows: Therefore, the candidate control barrier function is constructed as follows:

[0055] This function intuitively depicts the relative distance between the preview point and the i-th obstacle; a function value less than zero indicates that the preview point is within the danger zone. This yields the system's safety set. for:

[0056] Furthermore, to ensure that the system state always remains within the safe set, the CBF forward invariance condition must be satisfied:

[0057] in, It is a constant, at this time Can be considered an extension Class function A linear special case of .

[0058] right Differentiating along the system trajectory, we get:

[0059] in Let be the distance vector from the aiming point to the center of the obstacle. Substituting the expression for the aiming point velocity into the CBF condition, we obtain the linear inequality constraints for the two operating modes: Specifically, for the land mode, substitute... get:

[0060] For the water area mode, substitute get:

[0061] For complex environments with multiple obstacles, the CBF constraints corresponding to all obstacles are merged to obtain global linear inequality constraints. ,in For the constraint matrix, This is the constraint vector.

[0062] Step 4: Using the desired speed under unobstructed working conditions, i.e. the nominal reference speed, as the optimization objective, solve for the safe reference speed based on safety constraints and speed amplitude constraints.

[0063] Traditional obstacle avoidance methods employ the concept of "path replanning," where a new path is planned for each obstacle encountered. This approach is computationally intensive and prone to getting trapped in local optima. For example... Figure 2 As shown, this embodiment adopts the "speed correction" concept. After deriving the global linear inequality constraint in step 3, this embodiment further constructs a CBF-QP safety filter. The CBF-QP safety filter uses the desired speed, i.e., the nominal reference speed, in an obstacle-free environment as the optimization target. The CBF linear inequality constraint obtained in step 3 is used as a hard safety constraint. At the same time, a speed amplitude constraint is introduced to characterize the physical limit of the vehicle actuator, using the nominal reference speed in an obstacle-free environment. To optimize the objective, they collectively constitute a convex quadratic programming problem:

[0064]

[0065] Among them, optimization objectives The physical meaning is: to minimize the L2 norm of the safe reference speed and the nominal reference speed, that is, to maintain the original trajectory tracking performance as much as possible, and to adjust the speed only when necessary to avoid obstacles; the first set of constraints The first set of constraints is a CBF safety constraint, a hard constraint that must be satisfied to ensure the system does not collide; the second set of constraints... The speed amplitude constraint is a physical constraint that ensures the control input is within the actuator's capability range.

[0066] Nominal reference speed The position tracking error in the body-fixed coordinate system is calculated using proportional feedback: The nominal reference speed in land mode is set as follows:

[0067] The nominal reference speed in water mode is set as follows:

[0068] in , , For the position tracking error in the body-fixed coordinate system, This is the adjustment coefficient.

[0069] The interior-point method is used to solve this convex quadratic programming problem. Since convex optimization problems have a unique global optimum and the solution time is in the millisecond range, the real-time performance and global optimum of obstacle avoidance can be guaranteed. When the QP problem becomes unsolvable due to constraint conflicts (such as an obstacle completely blocking the path), the system backtracks to the nominal reference speed and triggers upper-level path replanning to ensure that the system does not stop due to constraint conflicts.

[0070] Step 5: Using the safe reference speed as the tracking target, a reinforcement learning network is used to estimate the unknown disturbance terms online, and the output control law is applied to the vehicle to make the actual speed track the safe reference speed. This embodiment employs a three-layer reinforcement learning architecture: "identifier-critic-actor". Figure 3 As shown, the unknown dynamic disturbance terms of amphibious tracked vehicles in land and water operations are estimated online, and the optimal control law is output to achieve high-precision tracking of the safe reference speed.

[0071] First, the speed tracking error is defined as the difference between the vehicle's actual speed and the safe reference speed output by the CBF-QP safety filter:

[0072] Even if the speed tracking error is detected, the target can still be controlled. Ultimately, it is bounded and as small as possible. Based on this, the three neural networks each perform their respective functions and work together. Their specific design and update process is as follows: The identifier network is specifically responsible for estimating unknowns in the dynamic model, separating the disturbances of unknowns from the control optimization problem. As input, Gaussian radial basis functions are used as basis functions, and the output is the estimated value of the unknown term:

[0073] in The weight matrix of the identifier network. Let be the Gaussian radial basis function vector. The weight update rule uses gradient descent with a regularization term to prevent overfitting.

[0074] in For learning rate, The regularization coefficient is . It is an identity matrix.

[0075] The critic network is specifically responsible for approximating the Hamilton-Jacobi-Bellman (HJB) equations, evaluating the merits of the current control strategy, and providing update directions for the actor network. The critic network takes the system state and velocity tracking error as input and outputs an approximation of the optimal cost function.

[0076] in , is a design constant. For the weight matrix of the critic network, Let be the basis function vector. The weight update rule is:

[0077] in For learning rate, The regularization coefficient is . It is an identity matrix.

[0078] The actor network is specifically responsible for outputting the optimal control law. Based on the unknown estimates provided by the identifier network and the evaluations provided by the critic network, it adjusts the control input to minimize the speed tracking error. The expression for the optimal control law is:

[0079] in This is the weight matrix of the actor network. The control law consists of three parts: This is a proportional feedback term to ensure tracking speed; This is a compensation term for unknown terms, used to counteract dynamic disturbances; To achieve the optimal control term, the performance index is minimized. The weight update rule is as follows:

[0080] in The learning rate is used to update the three networks simultaneously online, enabling adaptive compensation for unknown disturbances such as ground friction and hydrodynamics, without the need to establish an accurate dynamic model in advance.

[0081] Furthermore, to theoretically verify the stability and security of the aforementioned control framework, and to prove that the velocity tracking error is eventually bounded and that the system trajectory always remains within the safe set, this embodiment proposes a stability proof scheme based on the Lyapunov method, the specific implementation of which includes: First, construct a Lyapunov function that includes the velocity tracking error and the estimation errors of the three neural network weights:

[0082] in , , For weight estimation error, , These are ideal weights (i.e. weights that can accurately approximate the unknowns and the optimal cost function).

[0083] right By differentiating and using Young's inequality to handle the cross terms, transforming the product terms into a sum of squared terms, we obtain:

[0084] in A positive constant. To determine the minimum eigenvalue of the control input gain matrix g; Let be a bounded constant, composed of bounded terms such as the approximation error of the neural network, the derivative of the reference velocity, and the norm of the ideal weights. Integrating both sides of the above inequality, we obtain:

[0085] From this, we can draw the following core conclusions: (1) Speed ​​error Ultimately bounded, and By adjusting design parameters , , , The tracking error can be limited to an arbitrarily small range; (2) Weight estimation errors of the three neural networks , , Ultimately bounded, ensuring the network does not diverge; (3) When hour, That is, the actual speed of the vehicle Converging to the safe reference speed .because It is a solution to CBF-QP, satisfying all CBF safety constraints; therefore, the system trajectory always lies within the safety set. Inside, no collision will occur.

[0086] Step 6: Detect the vehicle's water entry status in real time. When the switching conditions are met, execute the water-land mode switch and switch the corresponding dynamic model and reinforcement learning network parameters. Repeat the above process until the trajectory tracking and obstacle avoidance tasks are completed.

[0087] Traditional mode switching methods often employ time-triggered or location-triggered approaches, which suffer from poor accuracy and are easily affected by environmental factors. This embodiment proposes using a water pressure-triggered mechanism to achieve a smooth transition for amphibious tracked vehicles between land and water, avoiding control shocks and instability during mode switching.

[0088] To achieve this, a water pressure sensor is installed at the bottom of the vehicle to detect the water pressure in real time. To accurately determine the vehicle's water entry status.

[0089] Threshold switching via preset mode When water pressure exceeds this threshold and the switchover has not yet been completed, the land-to-water switchover process is triggered. For example: When detected When not switching, the following seamless switching process is executed: at the moment of switching, the current control input value is frozen to avoid shock caused by step change; using a weighted average method, the system gradually transitions from the land dynamics model and land reinforcement learning network to the water dynamics model and water reinforcement learning network within 10 sampling periods (0.1s); when the vehicle returns from the water to the land, the same switching logic is used, and when the water pressure is lower than the threshold, the system smoothly switches back to the land mode.

[0090] The aforementioned water pressure triggering and smooth transition mechanism ensures the continuity of control commands and the stability of vehicle movement throughout the entire operation. Afterward, the system returns to acquire vehicle state and environmental information again, sequentially executing the kinematic transformation of the aiming point, CBF safety constraint derivation, CBF-QP safety filtering solution, reinforcement learning control law output, and mode switching discrimination, forming a continuous control closed loop. In each iteration, the current center position of the vehicle is calculated. and the set target location Euclidean distance between .

[0091] When distance Less than or equal to the preset position tolerance If the vehicle has safely reached the target location, the set trajectory tracking and obstacle avoidance tasks are completed, and the iteration stops; otherwise, the entire process is repeated.

[0092] Example 2 Based on the method described in Embodiment 1, the purpose of this embodiment is to provide an amphibious tracked vehicle tracking and obstacle avoidance control system that combines obstacle function control with reinforcement learning, including: The model building module is configured to: establish kinematic and dynamic mathematical models of amphibious tracked vehicles in land under-drive mode and water full-drive mode. The aiming point conversion module is configured to: introduce an aiming point mechanism and establish a Jacobian mapping relationship between the aiming point speed and the tracked vehicle's own speed; The safety constraint generation module is configured to: acquire real-time vehicle status and environmental obstacle information, and derive safety constraint conditions based on the control obstacle function; The safe reference speed solution module is configured to: use the nominal reference speed as the optimization target, and solve for the safe reference speed based on the safety constraints and speed amplitude constraints; The reinforcement learning control module is configured to: take the safe reference speed as the tracking target, use a reinforcement learning network to estimate unknown disturbance terms online, and output a control law to act on the amphibious tracked vehicle so that the actual speed tracks the safe reference speed; The mode switching module is configured to: detect the vehicle's water entry status in real time, and execute the water-land mode switching when the switching conditions are met, switching the corresponding dynamic model and reinforcement learning network parameters. The iterative execution module is configured to repeatedly execute the functions of the above modules until the trajectory tracking and obstacle avoidance tasks are completed.

[0093] like Figure 4 As shown, the system is logically divided into a perception data layer, a security decision-making and planning layer, an intelligent control layer, and an execution switching layer. Each layer collaborates through a data bus, and its specific functions are as follows: The perception data layer integrates sensor units and a state estimation module. It acquires raw information about vehicle pose, speed, and obstacles through GPS / IMU integrated navigation, wheel speed encoders, and LiDAR, and monitors the environmental medium pressure in real time using a water pressure sensor. The state estimation module fuses the above multi-source data to generate a unified system state vector.

[0094] Safety Decision Planning Layer: This layer contains the core logic for underactuated obstacle avoidance, including a preview point transformation unit, a safety constraint generation unit, and a safety filtering unit. The system establishes the kinematic relationship between the vehicle center and the target through a preview point mechanism; the safety constraint generation unit derives the CBF linear inequality based on obstacle information; and the safety filtering unit corrects the nominal speed to a safe reference speed that satisfies the obstacle avoidance constraints by solving a convex quadratic programming (QP) problem.

[0095] The intelligent control layer consists of a three-layer reinforcement learning controller comprised of an identifier network, a critic network, and an actor network. This layer receives state information from the perception layer and the safety reference velocity from the decision layer. The identifier network identifies and compensates for environmental disturbances online. The critic network is specifically responsible for approximating the Hamilton-Jacobi-Bellman (HJB) equations, evaluating the merits of the current control strategy, and providing update directions for the actor network. The actor network outputs the optimal control law, achieving high-precision tracking under complex disturbances.

[0096] The execution switching layer includes a water pressure-triggered switching module, a multi-mode dynamics model library, and robot actuators. This layer determines the current environment (land or water) based on real-time water pressure data and calls the corresponding mode's parameter set from the model library to ensure the continuity of control commands and the stability of vehicle movement during mode switching.

[0097] Example 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0098] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0099] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0100] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0101] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0102] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for tracking and obstacle avoidance control of an amphibious tracked vehicle using obstacle function control and reinforcement learning, characterized in that, include: Establish kinematic and dynamic mathematical models of amphibious tracked vehicles in under-actuated land mode and full-actuated water mode; Introduction A pre-aiming point mechanism is established to create a Jacobian mapping relationship between the pre-aiming point speed and the speed of the tracked vehicle itself. Acquire real-time vehicle status and environmental obstacle information, and derive safety constraints based on the control obstacle function; The desired speed under unobstructed working conditions, i.e., the nominal reference speed, is used as the optimization objective. The safe reference speed is solved based on safety constraints and speed amplitude constraints. Using the safe reference speed as the tracking target, a reinforcement learning network is used to estimate unknown disturbance terms online, and the output control law is applied to the amphibious tracked vehicle to make the actual speed track the safe reference speed. The system monitors the vehicle's water entry status in real time. When the switching conditions are met, it performs a water-land mode switch, switching the corresponding dynamic model and reinforcement learning network parameters. The above process is repeated, and the distance between the vehicle's current position and the target position is calculated in real time. When the distance is less than or equal to a preset position tolerance threshold, the system determines that the trajectory tracking and obstacle avoidance task is complete and stops iterating.

2. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 1, characterized in that, The aiming point is set in front of the vehicle body, and its position is determined by the center position of the vehicle body and the aiming distance.

3. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 1, characterized in that, The method of deriving safety constraints based on control obstacle functions includes: constructing candidate control obstacle functions based on the difference between the distance from the pre-aiming point to the center of the obstacle and the safety distance margin, and defining the safety set as the state region where the value of the candidate control obstacle function is greater than or equal to zero; Based on the forward invariance condition of the control obstacle function, and combined with the derivative of the control obstacle function along the system trajectory and the mapping relationship of the aiming point velocity, a linear inequality constraint with respect to the control input is obtained.

4. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 3, characterized in that, The linear inequality constraint includes, in land mode, the linear inequality constraint relating to longitudinal velocity and yaw rate; In the water mode, the linear inequality constraints involve the sway velocity, roll velocity, and yaw rate. When there are multiple obstacles in the environment, the constraints corresponding to all obstacles are merged to obtain global linear inequality constraints.

5. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 1, characterized in that, The process of finding the safe reference speed includes: constructing an optimization objective based on minimizing the L2 norm of the difference between the safe reference speed and the nominal reference speed; and constructing a convex quadratic programming problem using the safety constraints of the control barrier function and the speed amplitude constraints as constraints. The safe reference speed with the smallest deviation from the nominal reference speed is obtained by solving the convex quadratic programming problem based on the interior point method. When the convex quadratic programming problem becomes unsolvable due to constraint conflicts, it reverts to the nominal reference speed and triggers path replanning.

6. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 1, characterized in that, The reinforcement learning network includes: an identifier network, a critic network, and an actor network; the identifier network takes the system state as input and outputs an estimate of the unknown dynamics. The critic network takes the system state and velocity tracking error as input and outputs an estimate of the optimal cost function; The actor network generates a control law based on the outputs of the identifier network and the critic network. The control law includes a proportional feedback term, an unknown term compensation term, and an optimal control term. The three networks are updated simultaneously online to achieve adaptive compensation for unknown disturbances; A Lyapunov function incorporating velocity tracking error and the estimation errors of three neural network weights is constructed to ensure the stability of the amphibious tracked vehicle during obstacle avoidance.

7. The amphibious tracked vehicle tracking and obstacle avoidance control method based on obstacle function and reinforcement learning as described in claim 1, characterized in that, The water-land mode switching adopts a water pressure triggering mechanism. When the water pressure exceeds the preset threshold and the switching has not been completed, the current control input value is frozen, and within the preset sampling period, the parameters of the land dynamics model and reinforcement learning network are gradually transitioned to the water model and parameters in a weighted average manner.

8. An amphibious tracked vehicle tracking and obstacle avoidance control system based on obstacle function control and reinforcement learning, characterized in that, include: The model building module is configured to: establish kinematic and dynamic mathematical models of amphibious tracked vehicles in land under-drive mode and water full-drive mode. The aiming point conversion module is configured to: introduce an aiming point mechanism and establish a Jacobian mapping relationship between the aiming point speed and the tracked vehicle's own speed; The safety constraint generation module is configured to: acquire real-time vehicle status and environmental obstacle information, and derive safety constraint conditions based on the control obstacle function; The safe reference speed solution module is configured to: use the desired speed under unobstructed working conditions, i.e., the nominal reference speed, as the optimization objective, and solve for the safe reference speed based on safety constraints and speed amplitude constraints. The reinforcement learning control module is configured to: take the safe reference speed as the tracking target, use a reinforcement learning network to estimate unknown disturbance terms online, and output a control law to act on the amphibious tracked vehicle so that the actual speed tracks the safe reference speed; The mode switching module is configured to: detect the vehicle's water entry status in real time, and execute the water-land mode switching when the switching conditions are met, switching the corresponding dynamic model and reinforcement learning network parameters. The iterative execution module is configured to repeatedly execute the functions of the above modules, calculate the distance between the vehicle's current position and the target position in real time, and determine that the trajectory tracking and obstacle avoidance task is completed and stop iterating when the distance is less than or equal to a preset position tolerance threshold.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the obstacle control function and reinforcement learning amphibious tracked vehicle tracking and obstacle avoidance control method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the obstacle control function and the amphibious tracked vehicle tracking and obstacle avoidance control method of any one of claims 1-7.

Images