Vehicle underwater driving control method, vehicle and computer equipment

Abstract

The invention relates to a vehicle underwater driving control method, a vehicle and computer equipment, sensing data collected by a sensing system in the vehicle at the current moment is obtained, a current state vector is generated according to the sensing data, and the current state vector comprises water area environment information and vehicle state information; the current state vector is input into a sequence decision maker in a control system of the vehicle, the sequence decision maker takes a sequence formed by the current state vector, a historical state vector, a historical action instruction and a historical environment reward as input, a long-term dependency relationship in the sequence is understood through a self-attention mechanism, a decision context of the current moment is generated, and the decision context of the current moment is obtained. Outputting a multi-dimensional current action instruction according to the decision context; the current action instruction is sent to an execution system of the vehicle so as to instruct execution units of the execution system to complete corresponding actions; and the running stability of the vehicle in water is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle engineering technology, and in particular to a method for controlling vehicle movement in water, a vehicle, and computer equipment. Background Technology

[0002] Traditional amphibious vehicles rely primarily on mechanical stabilization mechanisms (such as fixed ballast tanks and passive hydrofoils) and the driver's experience (such as controlling the throttle and control surfaces) for underwater attitude control. This results in response lag and non-global optimum issues. Specifically, the vehicle cannot predict attitude changes caused by the next speed change in water and can only compensate afterward. However, by the time the system or driver detects the attitude change and reacts, the vehicle may already be in a dangerous and uncontrollable position.

[0003] Currently, there is no effective solution to the problem of difficulty in controlling the vehicle's posture while driving in water in related technologies. Summary of the Invention

[0004] Therefore, it is necessary to provide a vehicle underwater driving control method, vehicle, and computer equipment that can improve vehicle driving stability in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for controlling vehicle movement in water. The method includes:

[0006] Acquire sensor data collected by the vehicle's sensor system at the current moment, and generate a current state vector based on the sensor data. The current state vector includes water environment information and vehicle state information.

[0007] The current state vector is input to the sequence decision-maker in the vehicle's control system. The sequence decision-maker takes the sequence consisting of the current state vector, historical state vector, historical action command, and historical environmental reward as input, understands the long-term dependencies in the sequence through a self-attention mechanism, generates the decision context for the current moment, and outputs a multi-dimensional current action command based on the decision context. The historical state vector is a state vector generated from sensor data collected before the current moment, the historical action command is a multi-dimensional action command generated by the sequence decision-maker before the current moment, and the historical environmental reward includes the immediate environmental reward obtained by the vehicle each time it executes the historical action command to enter the next state.

[0008] The current action command is sent to the vehicle's execution system to instruct each execution unit of the execution system to complete the corresponding action.

[0009] In one embodiment, the sequence decision unit includes an input unit, a linear projection layer, a position encoder, a decoder, and an output unit connected in sequence.

[0010] The input unit is used to receive the current state vector, the historical state vector, the historical action instruction, and the historical environmental reward to form the sequence;

[0011] The linear projection layer is used to fuse the sequences and project them onto a unified vector space;

[0012] The position encoder is used to add position information to each time step vector in the sequence;

[0013] The decoder understands the long-term dependencies in the sequence through a self-attention mechanism and generates the decision context for the current moment.

[0014] The output unit generates a multi-dimensional current action instruction based on the decision context.

[0015] In one embodiment, the training process of the sequence decision-maker includes:

[0016] A virtual environment for vehicle floating on water is constructed based on a water area model and a vehicle model; wherein the water area model is constructed using initial water area state parameters, and the vehicle model is constructed using initial vehicle state parameters and vehicle motion parameters.

[0017] The sequence decision-maker to be trained is used as an Actor network to interact with the vehicle floating virtual environment, generate interaction data, and update the parameters of the Critic network based on the interaction data.

[0018] Based on the updated Critic network, the output of the Actor network is evaluated for value, and the parameters of the Actor network are updated according to the evaluation results.

[0019] The steps of updating the parameters of the Critic network and the Actor network are executed repeatedly until the preset conditions are met.

[0020] In one embodiment, the water state parameters include water depth and water flow;

[0021] The vehicle status parameters include roll angle, pitch angle, yaw angle, and vehicle speed;

[0022] The vehicle's motion parameters include the active air intake grille angle, ballast tank water inlet and outlet volume, wing-shaped pedal angle, wheel hub motor speed, propeller speed, air suspension telescopic height, and variable wheel arch angle.

[0023] In one embodiment, the sequence decision-maker to be trained is used as an Actor network to interact with the vehicle floating virtual environment, generating interaction data, including:

[0024] A first state vector is generated based on the current state parameters in the vehicle floating virtual environment. The first state vector is input into the Actor network. After processing by the Actor network, a multi-dimensional first action command is output.

[0025] The vehicle model responds to the first action command, updates the corresponding vehicle action parameters, obtains new state parameters, and generates a second state vector and an immediate environmental reward based on the new state parameters.

[0026] The first state vector, the first action instruction, the immediate environmental reward, and the second state vector are used as the interaction data and stored in the database.

[0027] In one embodiment, generating an immediate environmental reward based on the new state parameters includes:

[0028] From the new vehicle state parameters of the vehicle model, the roll angle, pitch angle and yaw rate are extracted. The roll angle, pitch angle and yaw rate are added together according to the corresponding penalty coefficients to obtain the stability bonus.

[0029] From the new vehicle state parameters of the vehicle model, the actual steering angle is extracted, and the tracking reward is obtained based on the deviation between the actual steering angle and the desired steering angle.

[0030] The stability reward and the tracking reward are added together according to their respective weights to obtain the immediate environment reward.

[0031] In one embodiment, updating the parameters of the Critic network based on the interaction data includes:

[0032] The first state vector and the first action command are input into the Critic network to obtain the first Q value;

[0033] A set of interactive data is randomly selected from the database, and the second state vector is input into the Actor network to obtain the second action instruction;

[0034] The second state vector and the second action command are input into the target Critic network to obtain the second Q value;

[0035] The first Q value and the second Q value are compared to calculate the mean squared error loss, and the parameters of the Critic network are updated by minimizing the mean squared error loss.

[0036] In one embodiment, the output of the Actor network is evaluated based on the updated parameters of the Critic network, and the parameters of the Actor network are updated according to the evaluation results, including:

[0037] The first state vector and the first action command are input into the updated Critic network to obtain the third Q value;

[0038] The policy gradient is calculated based on the third Q value, and the parameters of the Actor network are updated using the gradient ascent method.

[0039] In one embodiment, the training process of the sequence decision-maker further includes:

[0040] During the process of the vehicle entering the water, floating in the water and exiting the water, multiple driving modes are set, and corresponding state parameters of the water area model are set for each driving mode.

[0041] The sequence decision-maker is trained based on the state parameters of the water model under each of the aforementioned driving modes.

[0042] In one embodiment, the action instructions output by the sequence decision-maker include at least two of the following instructions:

[0043] The commands include: air intake grille operation command, ballast water tank operation command, wing-shaped pedal operation command, wheel hub motor operation command, air suspension operation command, propeller operation command, and variable wheel cover operation command.

[0044] Secondly, this application also provides a vehicle, including: a vehicle body, on which a sensing system, a control system and an execution system are installed;

[0045] The sensing system is used to collect sensing data, which includes aquatic environment information and vehicle status information;

[0046] The control system is equipped with a sequence decision unit, and the control system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0047] The execution system includes multiple execution units, each of which performs a corresponding action in response to the action command generated by the control system.

[0048] In one embodiment, the execution system includes at least two execution units:

[0049] Air intake grille control unit, ballast water tank control unit, wing-shaped pedal control unit, wheel hub motor control unit, air suspension control unit, propeller control unit, and variable wheel cover control unit.

[0050] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0051] The aforementioned vehicle underwater driving control method, vehicle, and computer equipment collect real-time aquatic environment information and vehicle status information. The current aquatic environment information and vehicle status information are used as the current state vector. Combined with historical trajectory data (historical state vector, historical action command, and historical environmental reward), a sequence is formed. The sequence is input into the sequence decision-maker, which understands the time evolution law of buoyancy dynamics based on the sequence and outputs multi-dimensional current action commands. It supports the coordinated control of multiple execution units, accurately predictively responds to the surrounding environment, and improves the stability of the vehicle driving in water. Attached Figure Description

[0052] Figure 1 This is a schematic cross-sectional view of the active grille integrated into the front bumper in one embodiment;

[0053] Figure 2 This is a schematic cross-sectional view of the active grille opening angle integrated into the front bumper in one embodiment. Figure 1 ;

[0054] Figure 3 This is a schematic cross-sectional view of the active grille opening angle integrated into the front bumper in one embodiment. Figure 2 ;

[0055] Figure 4 This is a schematic diagram of a multi-segment integrated pedal under the side skirt in one embodiment;

[0056] Figure 5 This is a schematic diagram of a retractable wheel cover in one embodiment;

[0057] Figure 6 This is a schematic diagram of a vehicle underwater driving control method in one embodiment;

[0058] Figure 7 This is a schematic diagram of the training process of a sequence decision-maker in one embodiment;

[0059] Figure 8 This is a schematic diagram of the vehicle floating process stages in one embodiment;

[0060] Figure 9 This is a schematic diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0063] In one embodiment, a vehicle is provided, comprising: a vehicle body, on which a sensing system, a control system, and an execution system are mounted. The sensing system is used to collect sensing data, including aquatic environment information and vehicle status information. The sensing system may include an ultrasonic ranging sensor, a water level and pressure sensor, a millimeter-wave radar, and an inertial measurement unit (IMU), the IMU being used to collect vehicle roll angle, pitch angle, yaw angle, and vehicle speed.

[0064] The control system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the vehicle underwater driving control method in any of the following embodiments. Specifically, a sequence decision unit is deployed in the control system. The sequence decision unit processes sensor data and generates multi-dimensional action commands adapted to the current driving mode. The control system may be a domain controller, used to process multi-domain sensor signals, integrate control models, and output multi-dimensional action commands.

[0065] The actuation system comprises multiple actuation units, each of which performs a corresponding action in response to an action command. The actuation system includes at least two of the following actuation units: air intake grille control unit, ballast tank control unit, wing-shaped pedal control unit, wheel hub motor control unit, air suspension control unit, propeller control unit, and variable wheel arch control unit.

[0066] The driver generates directional control requests and vehicle speed requests by operating the vehicle itself. In addition to the sensing system, control system, and actuator system, the vehicle itself also includes:

[0067] (1) A vector propeller thruster located below the rear bumper of the vehicle body is used to provide underwater thrust in different directions.

[0068] (2) The tire rim assembly generates forward or backward thrust through tire rotation, and at the same time assists in yaw control.

[0069] (3) An active air intake grille integrated into the front bumper controls the pitch angle of the vehicle head by opening and closing. Figure 1 This is a cross-sectional schematic diagram of an active grille integrated into the front bumper, wherein the active grille includes an upper grille 11 and a lower grille 12. Figure 2 Schematic diagram of the opening angle of the active grille integrated into the front bumper Figure 1 , Figure 3 Schematic diagram of the opening angle of the active grille integrated into the front bumper Figure 2 ,exist Figure 2 and Figure 3 In the diagram, A1 represents the opening angle of the upper grille 11, and A2 represents the opening angle of the lower grille 12.

[0070] (4) A ballast water tank located at the rear of the vehicle is used to assist in controlling pitch by injecting and draining water from the tank.

[0071] (5) Integrated airfoil pedals are arranged under the side skirts. They assist in pitching in the front and back directions by telescoping and rotating, and at the same time assist in controlling roll. Figure 4 This is a schematic diagram of a multi-section integrated pedal under the side skirt. In this diagram, B represents the vehicle's forward direction, C represents the angle adjustment direction, D represents the side skirt, and D1, D2, D3, and D4 represent the pedals, respectively.

[0072] (6) Air suspension, which controls the balance of resistance and thrust by adjusting the suspension travel.

[0073] (7) Variable wheel cover, the pitch angle is controlled by the extension and retraction of the wheel cover. Figure 5 This is a schematic diagram of a variable wheel cover, where E represents the variable wheel cover, E1 represents the variable wheel cover in the retracted state, E2 represents the variable wheel cover in the extended state, and E3 represents the variable wheel cover in the deflected state.

[0074] In one embodiment, Figure 6 A method for controlling vehicle movement in water is provided, which can be applied to the aforementioned vehicles, such as... Figure 6 As shown, the process includes the following steps:

[0075] Step S101: Obtain the sensing data collected by the sensing system in the vehicle at the current moment, and generate the current state vector based on the sensing data. The current state vector includes water environment information and vehicle state information.

[0076] The aquatic environment information includes water depth and current information, with current information including direction and velocity. Vehicle status information includes roll angle, pitch angle, yaw angle, vehicle speed, and position.

[0077] Step S102: The current state vector is input to the sequence decision-maker in the vehicle's control system. The sequence decision-maker takes the sequence consisting of the current state vector, historical state vector, historical action command, and historical environmental reward as input, understands the long-term dependencies in the sequence through a self-attention mechanism, generates the decision context at the current moment, and outputs a multi-dimensional current action command based on the decision context.

[0078] Among them, the historical state vector is the state vector generated from the sensor data collected before the current moment, the historical action command is the multi-dimensional action command generated by the sequence decision-maker before the current moment, and the historical environment reward includes the instantaneous environment reward obtained by the vehicle each time it executes a historical action command to enter the next state.

[0079] A sequence decision-maker can be a Transformer, a Recurrent Neural Network (RNN), or a Long Short-Term Memory (LSTM) network. The sequence decision-maker comprises an input unit, a linear projection layer, a position encoder, a decoder, and an output unit connected in sequence. The input unit receives the current state vector, historical state vectors, historical action instructions, and historical environmental rewards to form a sequence. The linear projection layer fuses the sequences and projects them onto a unified vector space. The position encoder adds positional information to each time step vector in the sequence to capture temporal relationships. The decoder uses a self-attention mechanism to understand long-term dependencies in the sequence and generates the decision context for the current moment. The output unit generates a multi-dimensional current action instruction based on the decision context.

[0080] The action commands output by the sequence decision-maker include at least two of the following commands: air intake grille action command, ballast water tank action command, wing-shaped pedal action command, wheel hub motor action command, air suspension action command, propeller action command, and variable wheel cover action command.

[0081] The sequence decision-maker is generated through reinforcement training in a virtual vehicle floating environment. It understands the long-term dependencies in the input sequence through a self-attention mechanism and generates the decision context, i.e., action instructions, for the current moment. Therefore, the sequence decision-maker has the ability to understand time sequence and generate decisions, making action instructions more accurate and effective.

[0082] Step S103: Send the current action command to the vehicle's execution system to instruct each execution unit of the execution system to complete the corresponding action.

[0083] After receiving the multidimensional current action command, the execution system will perform the following control tasks:

[0084] (1) By adjusting the angle of the variable wheel cover, the water flow around the vehicle can be changed to achieve water flow optimization.

[0085] (2) By adjusting the angle of the vehicle's wing-shaped pedals, the opening and closing angle of the front grille, and the water inlet and outlet of the ballast water tank, the pitch angle and roll angle are controlled to achieve vehicle attitude control.

[0086] (3) Propulsion of the vehicle is achieved by controlling the hub motor and the propeller.

[0087] (4) By adjusting the height of the air suspension, the buoyancy of the vehicle in the water can be controlled, thus achieving buoyancy control.

[0088] It should be noted that the aforementioned flow optimization, attitude, propulsion, and buoyancy control tasks can be combined in pairs to support the coordinated control of multiple execution units, in order to predictively respond to changes in water flow, wave cycles, and attitude transitions.

[0089] Traditional amphibious vehicles rely primarily on mechanical stabilization mechanisms (such as fixed ballast tanks and passive hydrofoils) and the driver's experience (such as controlling the throttle and control surfaces) for underwater attitude control. This results in response lag and non-global optimum issues. Specifically, the vehicle cannot predict attitude changes caused by the next speed change in water and can only compensate afterward. However, by the time the system or driver detects the attitude change and reacts, the vehicle may already be in a dangerous and uncontrollable position. Furthermore, traditional amphibious vehicles struggle to coordinate the control of multiple actuators (such as propellers, hydrofoils, and ballast tanks).

[0090] In steps S101 to S103 above, by collecting aquatic environment information and vehicle status information in real time, the current aquatic environment information and vehicle status information are used as the current state vector. Combined with historical trajectory data (historical state vector, historical action command and historical environmental reward) to form a sequence. The sequence is input to the sequence decision-maker, which understands the time evolution law of buoyancy dynamics based on the sequence, outputs multi-dimensional current action command, supports the coordinated control of multiple execution units, and can more accurately predict and respond to the surrounding environment, thereby improving the stability of the vehicle driving in water.

[0091] In one embodiment, a DDPG learning system (reinforcement learning training framework) is provided, which consists of an Actor network and a Critic network.

[0092] Actor network: namely the sequence decision-maker mentioned above, responsible for generating multi-dimensional action commands and performing combined control of buoyancy, propulsion and attitude.

[0093] Critic Network: Evaluate the long-term benefits of adopting the current combination of buoyancy, propulsion, and attitude control in the current environment.

[0094] Target Network: Includes the target Actor network and the target Critic network, used to provide a short-term fixed learning target for the training of online networks (Actor network and Critic network), thereby improving training stability.

[0095] Neural network parameters:

[0096] Current sequence decision maker parameters: w;

[0097] Current Critic network parameters: θ;

[0098] Target Actor network parameters: w'←w;

[0099] Target Critic network parameters: θ'←θ.

[0100] Experience replay pool: A database that stores interaction data (first state vector, first action instruction, immediate environmental reward, and second state vector) for batch training of online networks.

[0101] The immediate environmental reward r_t is defined using a multi-objective composite reward function, and the calculation formula is as follows:

[0102] r_t=w_1×R_stability+w_2×R_tracking;

[0103] Here, w_1 and w_2 are weighting coefficients used to balance the importance of the two objectives: stability and tracking performance. The specific reward components are calculated as follows:

[0104] Stability bonus R_stability: This is designed to penalize drastic attitude changes and encourage the vehicle to remain stable. The calculation formula is as follows:

[0105] ;

[0106] : The roll angle at the current moment, the angle at which the vehicle tilts to the left or right.

[0107] θ: The pitch angle at the current moment, the angle of the vehicle's forward and backward pitch.

[0108] ψ_dot: The yaw rate at the current moment, i.e., the angular velocity at which the vehicle turns. Excessive turning speed is penalized, not the yaw angle itself, because the yaw angle is determined by the path-following target.

[0109] λ_roll, λ_pitch, λ_yaw: Penalty coefficients that determine the tolerance for each unstable posture. The larger the value, the stricter the requirement for stability of that posture.

[0110] Tracking reward R_tracking: designed to encourage vehicle control responses to match the driver's path intent.

[0111] ;

[0112] δ_desired: Desired direction angle, given by the upper-level path planner or driver instructions, representing the desired driving path.

[0113] δ_actual: Actual direction angle, the action command output by the sequence decision-maker.

[0114] κ: Penalty coefficient. The larger the value, the higher the requirement for tracking accuracy.

[0115] Figure 7 A schematic diagram of the training process of the sequence decision-maker in this embodiment is provided, such as... Figure 7 As shown, the process includes the following steps:

[0116] Step S201: Construct a virtual environment for vehicle floating on water based on the water area model and the vehicle model.

[0117] The water area model is constructed using initial water state parameters, including water depth and current. The vehicle model is constructed using initial vehicle state parameters and vehicle motion parameters; the vehicle state parameters include roll angle, pitch angle, yaw angle, and vehicle speed; the vehicle motion parameters include active grille angle, ballast tank inlet / outlet volume, winglet pedal angle, hub motor speed, propeller speed, air suspension telescopic height, and variable wheel arch angle.

[0118] Step S202: The sequence decision-maker to be trained is used as an Actor network to interact with the virtual environment of the vehicle floating on water, generating interaction data, and updating the parameters of the Critic network based on the interaction data.

[0119] (1) Generate a first state vector based on the current state parameters in the vehicle floating virtual environment, input the first state vector into the Actor network, and output a multi-dimensional first action command after processing by the Actor network.

[0120] In the vehicle floating virtual environment, the first state vector S_t is obtained and input into the current sequence decision-maker. The sequence decision-maker outputs the multi-dimensional first action command a_t (such as adjusting buoyancy, propulsion power, etc.).

[0121] (2) The vehicle model responds to the first action command, updates the corresponding vehicle action parameters, obtains new state parameters, and generates a second state vector and immediate environmental reward based on the new state parameters.

[0122] The vehicle model responds to the multidimensional first action command a_t, updating the corresponding vehicle action parameters. After a period of time, the state parameters of both the water model and the vehicle model change, generating a second state vector S_{t+1} and an immediate environmental reward r_t. That is, the new state parameters include new water state parameters and new vehicle state parameters. The generation of the environmental reward based on the new state parameters includes: extracting roll angle, pitch angle, and yaw rate from the new vehicle state parameters of the vehicle model; adding the roll angle, pitch angle, and yaw rate according to their respective penalty coefficients to obtain a stability reward; extracting the actual steering angle from the new vehicle state parameters of the vehicle model; obtaining a tracking reward based on the deviation between the actual steering angle and the desired steering angle; and adding the stability reward and tracking reward according to their respective weights to obtain the immediate environmental reward. For the specific calculation formula, please refer to the calculation formula for the environmental reward r_t mentioned above, which will not be repeated here.

[0123] (3) The first state vector, the first action instruction, the environmental reward and the second state vector are used as interaction data and stored in the database.

[0124] The experience gained from this interaction is stored in the experience replay pool as a quadruple (S_t, a_t, r_t, S_{t+1}).

[0125] (4) Input the first state vector and the first action command into the Critic network to obtain the first Q value.

[0126] Input the first state vector S and the first action instruction a actually executed into the current Critic network to obtain the first Q value: Q(S, a, θ).

[0127] (5) Randomly select a set of interactive data from the database, input the second state vector into the Actor network to obtain the second action instruction; input the second state vector and the second action instruction into the target Critic network to obtain the second Q value.

[0128] A small batch of experience samples (S, a, r, S') is randomly sampled from the experience replay pool. The second state vector S' is input into the sequence decision-maker to obtain the predicted second action instruction a'.

[0129] Inputting (S', a') into the target Critic network yields the Q-value estimate: Q(S', a', θ').

[0130] The target Q value is estimated and calculated based on this Q value: Where γ is the discount factor. The target Q value is used as the second Q value.

[0131] (6) Compare the first Q value and the second Q value, calculate the mean squared error loss, and update the parameters of the Critic network by minimizing the mean squared error loss.

[0132] Specifically, the mean squared error loss of the first Q value and the second Q value is calculated: The gradient descent method is used to minimize the loss L, and the parameters θ of the current Critic network are updated so that the Critic network's evaluation of action value becomes increasingly accurate.

[0133] Step S203: Evaluate the output of the Actor network based on the updated Critic network, and update the parameters of the Actor network according to the evaluation results.

[0134] The first state vector and the first action command are input into the updated Critic network to obtain the third Q value; the policy gradient is calculated based on the three Q values, and the parameters of the Actor network are updated by the gradient ascent method.

[0135] Specifically, the first state vector S is input into the sequence decision-maker to obtain its output first action instruction a. (S, a) is then input into the updated Critic network to obtain the third Q-value Q(S, a, θ) for this action. The goal is to maximize this third Q-value and find the action policy that maximizes future cumulative rewards. The gradient of the loss function J with respect to the Actor network parameters w is calculated using the chain rule.

[0136] ;

[0137] Using the calculated policy gradient, update the parameters of the current Actor network via gradient ascent:

[0138] ;

[0139] This causes the Actor network to tend to choose actions that the Critic network considers to be of higher value, thus continuously optimizing the strategy.

[0140] Step S204: Repeatedly execute the steps of updating the parameters of the Critic network and the Actor network until the preset conditions are met.

[0141] In this embodiment, the preset condition can be the end of the vehicle simulation time or the arrival of the target location. By repeating the above steps S202 to S203, at regular intervals, the parameters of the current network are slowly synchronized to the target network using a soft update method:

[0142] Target Actor Network Update: ;

[0143] Target Critic network update: .

[0144] Here, τ can be a very small value (such as 0.001). By employing a slow tracking strategy, the stability of the entire learning process can be improved.

[0145] In steps S201 to S204 above, the vehicle continuously interacts with the virtual environment, constantly learning from its own experience. By storing historical trajectories in the experience replay pool, the sequence decision-maker is trained in batches, ultimately enabling the sequence decision-maker to generate control strategies that adapt to the aquatic environment. The Critic network becomes an accurate decision value evaluator, realizing autonomous evolution from perception, decision-making, execution to learning, enabling amphibious vehicles to dynamically optimize their water passage strategies.

[0146] In one embodiment, training the sequence decision-maker based on a vehicle floating water virtual environment further includes:

[0147] During the vehicle's entry into, floating, and exiting the water, multiple driving modes are set, and corresponding state parameters of the water model are configured for each driving mode. Sequence decision-makers are then trained based on the state parameters of the water model for each driving mode. Taking the Transformer decision-maker as an example... Figure 8 This is a schematic diagram of the vehicle floating process in this embodiment, as shown below. Figure 8 As shown, vehicle operation in water mainly includes two stages: water entry preparation and buoyancy establishment, and water exit and land transition. During these two stages, depending on the water depth and current, the vehicle may face calm waters, turbulent currents, and wave areas. Therefore, this embodiment defines the possible driving modes of the vehicle as follows: water entry preparation and buoyancy establishment mode, calm water cruising mode, turbulent current combat mode, wave area navigation mode, and water exit and land transition mode. The vehicle driving control for each driving mode will be described below.

[0148] (1) Water entry preparation and buoyancy establishment mode.

[0149] As the vehicle approaches the water from land, the water depth gradually increases, and the vehicle speed begins to decrease appropriately in preparation for water entry. The sensor detects that the water depth change rate h > 0. The Transformer decision-maker begins to analyze the historical sequence [water depth, vehicle speed, air intake grille, ballast tank], establishes the correlation between "water depth" + "grill" and "water depth" + "ballast tank", and executes the corresponding actions in advance. The air intake grille is slightly closed to establish basic buoyancy preparation; the ballast tank begins to slowly fill with water to increase the stability of the vehicle weight.

[0150] The vehicle's front wheels contact the water surface and generate initial impact and resistance. Buoyancy begins to play a significant role, affecting the vehicle's attitude. The tire drainage effect generates lateral instability. The Transformer decision-maker makes adaptive adjustments. If lateral instability is detected and the roll angle is greater than the threshold angle d, emergency drainage is performed to restore stability, while the vehicle speed is reduced to decrease the impact on the water surface.

[0151] As the vehicle's buoyancy exceeds its weight, the tires gradually detach from the riverbed. At this point, the propulsion method switches from wheel drive to thruster drive, and the control mode changes from ground dynamics to hydrodynamics. Therefore, the Transformer decision-maker adjusts the grid closure angle and the water volume in the ballast tanks based on pitch angle fluctuations to ensure the vehicle's pitch angle meets the target requirements. Simultaneously, the airfoil pedals deploy their initial angle according to historical sequences for lateral stability. Finally, the thrusters activate for acceleration while simultaneously reducing wheel speed.

[0152] (2) Stable floating navigation - calm water cruise mode.

[0153] In calm water driving mode, the vehicle operates at its optimal navigation level. The water depth is typically greater than 2 meters, the current speed is less than 0.5 meters per second, and the surface is calm. At this time, the core control objectives of the Transformer decision-maker are optimal energy efficiency and precise trajectory.

[0154] The system first establishes an economical cruising state: the ballast tank enters a "hold" state, making only minor adjustments (flow rate within ±0.2 kg / s) to maintain perfect neutral buoyancy. The air suspension extends to its shortest length, maintaining the vehicle at optimal aerodynamic height. The variable wheel arches fully open, guiding water flow smoothly through the wheel wells, further reducing drag.

[0155] The propulsion system enters a highly efficient collaborative mode. The main motor maintains a speed of 800 RPM to provide power to the onboard systems, while keeping the wheels idling at a low speed in case of emergency. In this mode, the attention mechanism initiated by the Transformer will evenly monitor the entire input sequence, looking for any subtle signs of disturbance that might disrupt this balance, and making gentle compensatory adjustments in advance to ensure a silky smooth ride.

[0156] (3) Stable floating navigation - turbulent water flow countermeasure mode.

[0157] Once the sensor detects a lateral flow velocity exceeding 1.0 m / s, the system immediately switches to turbulent flow countermeasure mode. The core objective of this mode is to prevent drift and maintain course, with energy efficiency taking a secondary role. The Transformer decision-maker sensitively identifies the direction of the flow. If a continuous left-side flow is detected, it initiates an asymmetric countermeasure strategy.

[0158] In terms of attitude control, it instructs the right ballast tank to fill with water at a higher flow rate, while the left tank remains empty or drains slightly, artificially creating a rightward tilting restoring moment to counteract the water flow's tendency to push the vehicle to the left. The wing-shaped pedals extend to over 35 degrees, providing significant lateral stability like underwater rudder surfaces; the wheel arches are adjusted to a specific angle, partially exposing the wheels.

[0159] The suspension height remains at its shortest extension position to lower the vehicle's center of gravity and enhance its anti-roll capability;

[0160] The propulsion system switches to differential drive mode. The thruster on the front side (left) will significantly increase its speed to 1800 RPM, generating stronger thrust to "hold back" the water flow; while the thruster on the back side (right) will remain at 1200 RPM. The torque generated by the two helps the front of the vehicle always point towards the target course.

[0161] At this point, the Transformer decision-maker focuses its attention on the lateral displacement and yaw angle deviation in the most recent time steps, dynamically adjusting this asymmetric control strategy to form a negative feedback loop.

[0162] (4) Stable floating navigation - wave area navigation mode.

[0163] When the vehicle enters open water or a waterway and encounters periodic waves, the system enters a wave zone navigation mode. In this mode, the focus of control shifts from position holding, steering attitude stability, and occupant comfort.

[0164] The Transformer decision-maker demonstrates its powerful time-series prediction capabilities here. By analyzing the roll and pitch angle sequences over the past few tens of seconds, it accurately identifies the main cycle of the wave (e.g., 3 seconds of roll and 4 seconds of pitch). Based on this, it no longer performs passive responsive control, but instead implements predictive active control.

[0165] During the wave's uplift phase, the system proactively instructs the ballast tanks to drain water and reduces the airfoil step angle to actively "meet" the wave's uplift force, avoiding direct resistance. During the wave's descent phase, the system reverses this process, injecting water and increasing the step angle to cushion the descent. The air intake grille undergoes high-frequency, micro-adjustments (dynamically varying between 0.5 and 0.8), utilizing the rapid response characteristics of air buoyancy to actively suppress pitching motion.

[0166] The propeller speed is synchronized with the pitch cycle, increasing thrust appropriately when the front of the car pitches up and decreasing it when it pitches down, thus maintaining a relatively stable forward speed and avoiding a jerky, uneven feeling. The suspension system acts as the last line of defense, serving as an active damper to absorb high-frequency vibrations.

[0167] (5) Transition mode between water and land.

[0168] As the vehicle approaches the opposite bank and sensors detect a continuous decrease in water depth, the crucial water-to-land transition mode is initiated. The core of this process is the smooth switching of the power system and the safe recovery of attitude. The Transformer decision-maker identifies the key sequence pattern of "continuously decreasing water depth and increased riverbed contact signal." Preparation for the transition begins when the water depth is approximately 1.0 meter.

[0169] The system first instructs the air intake grille to open gradually, and the ballast tank to start draining water vigorously. The purpose is to gradually increase the traction of the wheels on the riverbed and prepare for the switching of driving force.

[0170] When the water depth drops to about 0.5 meters and the tires gain some traction, the power system begins a "soft switch." Electric motor power is gradually transferred from the propellers to the wheels—a dynamic process calculated in real-time based on tire slip ratio. The propeller speed then gradually decreases, shifting from primary propulsion to a role of auxiliary steering and anti-skid.

[0171] During the phase from a water depth of 0.3 meters to complete emergence from the water, the vehicle is in its most unstable "amphibious" state. The Transformer coordinates control with extreme care: the wing-shaped pedals and wheel arches retract completely to avoid collision with the ground; the suspension gradually lowers from a high position to standard land height to restore suspension geometry and handling; once all sensors confirm that the vehicle is completely back on solid ground, the system executes the final procedure: the thrusters are shut off, the grille is completely sealed, and a brief period of brake pad drying and sensor recalibration is performed to complete the entire buoyancy cycle.

[0172] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores sensor data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for controlling vehicle navigation in water.

[0173] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0174] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0175] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0176] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0177] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for controlling vehicle movement in water, characterized in that, include: Acquire sensor data collected by the vehicle's sensor system at the current moment, and generate a current state vector based on the sensor data. The current state vector includes water environment information and vehicle state information. The current state vector is input to the sequence decision-maker in the vehicle's control system. The sequence decision-maker takes the sequence consisting of the current state vector, historical state vector, historical action command, and historical environmental reward as input, understands the long-term dependencies in the sequence through a self-attention mechanism, generates the decision context for the current moment, and outputs a multi-dimensional current action command based on the decision context. The historical state vector is a state vector generated from sensor data collected before the current moment, the historical action command is a multi-dimensional action command generated by the sequence decision-maker before the current moment, and the historical environmental reward includes the immediate environmental reward obtained by the vehicle each time it executes the historical action command to enter the next state. The current action command is sent to the vehicle's execution system to instruct each execution unit of the execution system to complete the corresponding action.

2. The vehicle underwater driving control method according to claim 1, characterized in that, The sequence decision unit includes an input unit, a linear projection layer, a position encoder, a decoder, and an output unit connected in sequence; wherein... The input unit is used to receive the current state vector, the historical state vector, the historical action instruction, and the historical environmental reward to form the sequence; The linear projection layer is used to fuse the sequences and project them onto a unified vector space; The position encoder is used to add position information to each time step vector in the sequence; The decoder understands the long-term dependencies in the sequence through a self-attention mechanism and generates the decision context for the current moment. The output unit generates a multi-dimensional current action instruction based on the decision context.

3. The vehicle underwater driving control method according to claim 1 or claim 2, characterized in that, The training process of the sequence decision-maker includes: A virtual environment for vehicle floating on water is constructed based on a water area model and a vehicle model; wherein the water area model is constructed using initial water area state parameters, and the vehicle model is constructed using initial vehicle state parameters and vehicle motion parameters. The sequence decision-maker to be trained is used as an Actor network to interact with the vehicle floating virtual environment, generate interaction data, and update the parameters of the Critic network based on the interaction data. Based on the updated Critic network, the output of the Actor network is evaluated for value, and the parameters of the Actor network are updated according to the evaluation results. The steps of updating the parameters of the Critic network and the Actor network are executed repeatedly until the preset conditions are met.

4. The vehicle underwater driving control method according to claim 3, characterized in that, The water area state parameters include water depth and water flow; The vehicle status parameters include roll angle, pitch angle, yaw angle, and vehicle speed; The vehicle's motion parameters include the active air intake grille angle, ballast tank water inlet and outlet volume, wing-shaped pedal angle, wheel hub motor speed, propeller speed, air suspension telescopic height, and variable wheel arch angle.

5. The vehicle underwater driving control method according to claim 4, characterized in that, The sequence decision-maker to be trained is used as an Actor network to interact with the virtual environment of the vehicle floating on water, generating interaction data, including: A first state vector is generated based on the current state parameters in the vehicle floating virtual environment. The first state vector is input into the Actor network. After processing by the Actor network, a multi-dimensional first action command is output. The vehicle model responds to the first action command, updates the corresponding vehicle action parameters, obtains new state parameters, and generates a second state vector and an immediate environmental reward based on the new state parameters. The first state vector, the first action instruction, the immediate environmental reward, and the second state vector are used as the interaction data and stored in the database.

6. The vehicle underwater driving control method according to claim 5, characterized in that, Generate an instantaneous environmental reward based on the new state parameters, including: From the new vehicle state parameters of the vehicle model, the roll angle, pitch angle and yaw rate are extracted. The roll angle, pitch angle and yaw rate are added together according to the corresponding penalty coefficients to obtain the stability bonus. From the new vehicle state parameters of the vehicle model, the actual steering angle is extracted, and the tracking reward is obtained based on the deviation between the actual steering angle and the desired steering angle. The stability reward and the tracking reward are added together according to their respective weights to obtain the immediate environment reward.

7. The vehicle underwater driving control method according to claim 5, characterized in that, Updating the parameters of the Critic network based on the interaction data includes: The first state vector and the first action command are input into the Critic network to obtain the first Q value; A set of interactive data is randomly selected from the database, and the second state vector is input into the Actor network to obtain the second action instruction; The second state vector and the second action command are input into the target Critic network to obtain the second Q value; The first Q value and the second Q value are compared to calculate the mean squared error loss, and the parameters of the Critic network are updated by minimizing the mean squared error loss.

8. The vehicle underwater driving control method according to claim 3, characterized in that, The Critic network, with updated parameters, performs a value assessment on the output of the Actor network, and updates the parameters of the Actor network based on the value assessment results, including: The first state vector and the first action command are input into the updated Critic network to obtain the third Q value; The policy gradient is calculated based on the third Q value, and the parameters of the Actor network are updated using the gradient ascent method.

9. The vehicle underwater driving control method according to claim 3, characterized in that, The training process of the sequence decision-maker also includes: During the process of the vehicle entering the water, floating in the water and exiting the water, multiple driving modes are set, and corresponding state parameters of the water area model are set for each driving mode. The sequence decision-maker is trained for the state parameters of the water model under each of the aforementioned driving modes.

10. The vehicle underwater driving control method according to claim 1, characterized in that, The action instructions output by the sequence decision-maker include at least two of the following instructions: The commands include: air intake grille operation command, ballast water tank operation command, wing-shaped pedal operation command, wheel hub motor operation command, air suspension operation command, propeller operation command, and variable wheel cover operation command.

11. A vehicle, characterized in that, include: The vehicle body, on which a sensing system, a control system, and an execution system are installed; wherein, The sensing system is used to collect sensing data, which includes aquatic environment information and vehicle status information; The control system is equipped with a sequence decision unit, and the control system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10. The execution system includes multiple execution units, each of which performs a corresponding action in response to the action command generated by the control system.

12. The vehicle according to claim 11, characterized in that, The execution system includes at least two execution units: Air intake grille control unit, ballast water tank control unit, wing-shaped pedal control unit, wheel hub motor control unit, air suspension control unit, propeller control unit, and variable wheel cover control unit.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

Images