Autonomous underwater vehicle control method and system based on delta regularized network

Abstract

The autonomous underwater vehicle control method and system based on the incremental regularization network belong to the intelligent control technical field, and solve the problems that the traditional path planning algorithm is prone to low calculation efficiency and insufficient real-time when facing complex environments.The method of the present application comprises the following steps: introducing an L2 regularization method when training a deep reinforcement learning model with historical data to quickly realize real-time planning of a path; introducing an incremental path dynamic decision mechanism, extracting parameters of the trained deep reinforcement learning model, and performing LoRA fine-tuning on the model parameters in batches, so that the model can dynamically update its Q value function during the movement of the autonomous underwater vehicle. Based on the deep reinforcement learning method, an incremental path decision network is designed for the autonomous underwater vehicle, so that the autonomous underwater vehicle can make incremental decisions according to the new information obtained during the movement. The present application is suitable for autonomous path planning of autonomous underwater vehicles in complex environments.

Description

Technical Field

[0001] This application relates to the field of intelligent control technology, and in particular to the control of autonomous underwater vehicles. Background Technology

[0002] With the increasing exploration and development of marine resources, autonomous underwater vehicles (AUVs) are being used more and more widely in marine scientific research, environmental monitoring, and marine resource exploration. However, the complex and ever-changing underwater environment, including factors such as ocean currents, marine life, and obstacles, poses significant challenges to the autonomous navigation and path planning of AUVs.

[0003] Existing autonomous underwater vehicle (AUV) control technologies largely rely on pre-defined path planning methods, typically making decisions based on static environment models. While these methods are effective under certain conditions, they often lack flexibility in dynamic underwater environments due to their inability to adapt to changing conditions in real time, increasing the risk of collisions with underwater obstacles. Furthermore, traditional path planning algorithms are prone to computational inefficiency and insufficient real-time performance in complex environments, making them unsuitable for practical applications.

[0004] To improve the navigation capabilities of autonomous underwater vehicles (AUVs), artificial intelligence technologies such as deep learning and reinforcement learning have been increasingly applied to path planning in recent years. By learning from historical data, these algorithms can improve their adaptability to complex environments. However, existing deep reinforcement learning models are susceptible to overfitting during training, resulting in insufficient generalization ability in practical applications. Summary of the Invention

[0005] The purpose of this invention is to address the problems of low computational efficiency and insufficient real-time performance of traditional path planning algorithms when facing complex environments, and to provide an autonomous underwater vehicle control method and system based on incremental regularization networks.

[0006] This invention is achieved through the following technical solution. One aspect of this invention provides a control method for an autonomous underwater vehicle based on an incremental regularization network, the method comprising:

[0007] Step 1: Obtain the historical underwater navigation database of AUVs, train the model using a deep Q-network, and control the model complexity by introducing L2 regularization.

[0008] Step 2: The autonomous underwater vehicle starts from the starting point and performs initial path planning based on the initial environmental information collected by the sensors;

[0009] Step 3: Train a deep reinforcement learning model using the incremental path dynamic decision-making mechanism to obtain an incremental path decision network model. The autonomous underwater vehicle's sensors collect surrounding environmental data in real time, and use LoRA fine-tuning technology to fine-tune the parameters of the incremental path decision network model in batches; LoRA is a low-rank decomposition model fine-tuning technology.

[0010] Step 4: Update the path planning of the autonomous underwater vehicle using the incremental path decision network model fine-tuned in Step 3;

[0011] Step 5: The autonomous underwater vehicle checks whether the return-to-home constraint is met. If the return-to-home constraint is not met, it returns to Step 3 to continue underwater navigation and uses the collected data to fine-tune the model and update the path; if the return-to-home constraint is met, it proceeds to Step 6.

[0012] Step 6: The autonomous underwater vehicle (AUV) calls the incremental path decision network to generate a path planning scheme for returning to the starting point from the current position, and returns according to the scheme; after the AUV arrives at the starting point, it transmits the underwater navigation data of this trip back to the historical underwater navigation database of the data center.

[0013] Furthermore, in step 1, the control of model complexity by introducing L2 regularization specifically involves adding the sum of squared parameters to the loss function to make the model smoother.

[0014] Furthermore, in step 1, the control of model complexity by introducing L2 regularization specifically includes:

[0015] Step 1.1: Construct the state space representation of the autonomous underwater vehicle:

[0016]

[0017] in, S This represents the state space representation of an autonomous underwater vehicle. st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The state vector consists of four variables, which represent the vector of the current position of the autonomous underwater vehicle (AUV), the vector of the current velocity of the AUV, the current heading angle of the AUV, and the spatial information of the obstacles around the AUV.

[0018] Step 1.2: Construct the motion space representation of the autonomous underwater vehicle:

[0019]

[0020] in, A This represents the action space representation of an autonomous underwater vehicle. at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The resulting state vector consists of two variables, which represent the vector representation of the change in the autonomous underwater vehicle's velocity and the vector representation of the change in the autonomous underwater vehicle's heading angle, respectively.

[0021] Step 1.3: Train a deep Q-network. During the iterative process based on deep reinforcement learning, calculate the estimated future cumulative reward for each attempt using the following formula:

[0022]

[0023] in, s t This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, s t+1 This indicates that the autonomous underwater vehicle is at a time step t Environmental information at +1, t Represents the current timestamp, a t This indicates that the autonomous underwater vehicle is at a time step t The action space of time, a’ This indicates the next state. s t+1 The optimal action among all possible actions; max represents the function that takes the maximum value; Represents the parameters of a deep Q-network. Indicates in t Always in the environment s t Next action a t When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; r t Indicates the state s t Execute action a t The instant reward obtained afterward This represents the discount factor, which controls the importance of future rewards. These represent the parameters of the target network, which are used as parameters for delayed updates to improve the stability of training.

[0024] Step 1.4: During the training of the deep Q-network, initialize its loss function according to the following formula:

[0025]

[0026] in, The initial value of the loss function is... Represents the parameters of a deep Q-network. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, st +1 indicates that the autonomous underwater vehicle is at a certain time step. t Environmental information at +1, t Represents the current timestamp, at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, rt Indicates the state st Execute action at The instant reward obtained afterward; D This represents the experience replay pool. Indicates from the experience replay pool D The operation function for extracting samples for training; yt Indicates time t The target Q value is given below, where the Q value represents an estimated reward.

[0027] Step 1.5: Introduce L2 regularization to modify the loss function;

[0028] Step 1.6: Update the model parameters according to the following formula To minimize the loss function:

[0029]

[0030] in, Represents the parameters of a deep Q-network. The symbol represents an update operation; α The learning rate is used to control the step size for parameter updates. Indicates to The operation function for calculating the gradient, This indicates that the loss function is applied to the parameters. θ The gradient is used to update the model weights; This represents the final loss function obtained after introducing L2 regularization for correction.

[0031] Furthermore, in step 1.5, L2 regularization is introduced to modify the loss function according to the following formula:

[0032]

[0033] in, This represents the final loss function obtained after introducing L2 regularization for correction. The initial value of the loss function is... Represents the parameters of a deep Q-network. This represents the regularization strength coefficient, used to control the weights of L2 regularization; i represents the index of the weight parameter. Represents the weight parameters. Represents all network weight parameters The sum of squares.

[0034] Furthermore, in step 3, the step of using LoRA fine-tuning technology to fine-tune the parameters of the incremental path decision network model in batches specifically includes:

[0035] Step 3.1: During the dynamic decision-making process, the autonomous underwater vehicle continuously updates its Q-value function based on real-time data collected by sensors. The Q-value function represents the calculation function of the reward estimate. Based on the current state, it makes the optimal path decision. The Q-value function is incrementally updated over time to adapt to the constantly changing environment. Specifically, each time the sensor detects new information, the model incrementally updates the Q-value function based on this new data, gradually approaching the optimal path planning.

[0036] Step 3.2: Perform low-rank decomposition according to the following formula:

[0037]

[0038] in, W This is the updated weight matrix obtained using the low-rank decomposition operation; Let represent the original weight matrix, and represent the parameters of the model pre-training, given by . It is extracted from a deep Q-network; Indicates the amount of model parameter updates; A and B All are low-rank matrices. d , r , k Both represent the dimensions of the matrix; Represent a matrix over the real number field. Represent a d OKr A matrix over the real field of columns, Represent a r OK k A matrix over the real number field of a column;

[0039] Step 3.3: The new environmental data collected by the sensors is transmitted to the incremental path decision network model in batches, and each batch triggers a LoRA fine-tuning; the operation of updating the low-rank matrix using this batch of data satisfies the following formula:

[0040]

[0041] in, This represents the batch loss function used to update the parameters of LoRA fine-tuning; This represents the parameters of the deep Q-network; N represents the batch size, which is the number of data samples contained in each batch. yt Indicates time t The target Q value is given below, where the Q value represents an estimated reward. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; i, j, k Both represent the ID of the current data sample in the current batch; si This indicates that autonomous underwater vehicles are in the data sample i The following environmental information, ai This indicates that autonomous underwater vehicles are in the data sample i The action space below; This represents the target Q value, and the actual reward value for the current state. A and B All are low-rank matrices. Aj and Bk These represent the low-rank matrices in LoRA fine-tuning. A and B The column vector is used to represent some of the parameters updated by the model; This represents the regularization coefficient, used to control the magnitude of low-rank matrix updates;

[0042] Step 3.4: Incrementally update the Q-value function.

[0043] Furthermore, in step 3.4, the Q-value function is incrementally updated according to the following formula:

[0044]

[0045] in, This represents the Q value after the incremental update. This represents the Q-value of the model before the update; This represents the maximum Q value among all possible actions for the next step; This represents the parameters of the depth-Q network after incremental updates. This represents the depth-Q network parameters before the incremental update. This represents the parameters of the deep Q-network when the maximum Q-value is obtained among all possible actions in the next step; max represents the function that takes the maximum value. The learning rate determines the update step size. This indicates that the autonomous underwater vehicle is at a time step t When in a state st Next action at The instant reward obtained afterward.

[0046] Furthermore, in step 5, the formula for calculating the return constraint of the autonomous underwater vehicle is:

[0047]

[0048] in, CON This represents the return-to-home constraint. A value of 0 indicates that the return-to-home constraint is not met, while a value of 1 indicates that the return-to-home constraint is met. This indicates a command constraint; its value is 1 when a return command is received from the control center, and 0 otherwise. This indicates the maximum sailing time constraint. t Represents the current timestamp. Indicates the maximum sailing time; Indicates battery constraint. Indicates the parameters of a deep Q-network; Indicates time t The remaining battery power; The function for estimating the power consumption during the return trip represents the function that utilizes the parameter as... The parameters of the deep Q-network, in time t It is estimated that the amount of electricity it will consume to return to the starting point will be used in time; This indicates that reserved power is available.

[0049] Secondly, the present invention provides an autonomous underwater vehicle control system based on an incremental regularization network for the method described above, the system comprising:

[0050] The data acquisition module is used to collect historical data and real-time environmental data during the navigation of the autonomous underwater vehicle.

[0051] The data transmission module is used to obtain historical navigation data from the data center and transmit the navigation data back to the data center after the navigation is completed;

[0052] The model training module is used to train a deep reinforcement learning model with L2 regularization for path planning based on the collected data.

[0053] The fine-tuning module, based on LoRA fine-tuning technology, dynamically updates model parameters to achieve real-time path planning adjustments;

[0054] The path decision module is used to output the motion path of the autonomous underwater vehicle based on the incremental path decision network, avoid dynamic obstacles, and optimize the navigation path.

[0055] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, it executes the steps of an autonomous underwater vehicle control method based on an incremental regularization network as described above.

[0056] Fourthly, the present invention provides a computer-readable storage medium storing a plurality of computer instructions, the plurality of computer instructions being used to cause a computer to execute an autonomous underwater vehicle control method based on an incremental regularization network as described above.

[0057] The beneficial effects of this invention are:

[0058] This invention provides a path planning method for autonomous underwater vehicles (AUVs) based on incremental regularization networks. The AUV control method based on incremental path decision networks introduces L2 regularization when training a deep reinforcement learning model using historical data. This effectively controls model complexity while preventing overfitting, enabling rapid real-time path planning. An incremental path dynamic decision mechanism is introduced, extracting parameters from the trained deep reinforcement learning model and performing LoRA fine-tuning on the model parameters in batches, allowing the model to dynamically update its Q-value function during the AUV's movement. Based on deep reinforcement learning, this invention designs an incremental path decision network for the AUV, enabling it to make incremental decisions based on newly acquired information during movement. This provides autonomous path planning capabilities to prevent collisions with underwater dynamic targets such as marine life, reducing the risk of AUV recovery.

[0059] This invention is applicable to autonomous path planning for autonomous underwater vehicles in complex environments. Attached Figure Description

[0060] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating an embodiment of an autonomous underwater vehicle control method based on an incremental path decision network provided by the present invention.

[0062] Figure 2 This is the planning result of an autonomous underwater vehicle using an incremental path decision network according to the present invention. Detailed Implementation

[0063] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0064] Implementation Method 1: A control method for an autonomous underwater vehicle based on incremental regularization networks, the method comprising:

[0065] S1: Obtain the historical underwater navigation database of AUVs from the data center. AUVs refer to Autonomous Underwater Vehicles. Train the model using a deep Q-network (deep reinforcement learning model), and control the model complexity by introducing L2 regularization to prevent overfitting and enhance the model's generalization ability in complex underwater environments. L2 regularization is a method that smooths the model by adding a sum of squared parameters to the loss function.

[0066] S2: The autonomous underwater vehicle starts from the starting point and performs initial path planning based on the initial environmental information collected by the sensors;

[0067] S3: An incremental path dynamic decision-making mechanism is introduced. The autonomous underwater vehicle's sensors collect surrounding environmental data in real time, and the parameters of the incremental path decision-making network model are fine-tuned in batches using LoRA fine-tuning technology, thereby enabling the model's Q-value function to be updated in real time. The Q-value function means the reward function simulated using a deep reinforcement learning model. The incremental path decision-making network model means a deep reinforcement learning model trained using the incremental path dynamic decision-making mechanism. LoRA refers to the low-rank decomposition model fine-tuning technology.

[0068] S4: The incremental path decision network model finely tuned by LoRA is used to update the path planning of the autonomous underwater vehicle, thereby realizing the incremental path dynamic decision-making for the newly collected environmental information, avoiding dynamic obstacles such as marine organisms, and optimizing the path planning of the autonomous underwater vehicle.

[0069] It should be noted that the incremental path decision network model is an incremental path decision network model fine-tuned using LoRA fine-tuning technology. After fine-tuning, the model will also update the path planning.

[0070] S5: The autonomous underwater vehicle checks whether the return-to-home constraint is met. If the return-to-home constraint is not met, return to step S3, continue underwater navigation, and use the collected data to fine-tune the model and update the path. If the return-to-home constraint is met, proceed to step S6;

[0071] S6: The autonomous underwater vehicle (AUV) invokes the incremental path decision network to generate a path planning scheme for returning to the starting point from the current position, and then returns according to the scheme. After arriving at the starting point, the AUV transmits the underwater navigation data back to the historical underwater navigation database in the data center.

[0072] In this embodiment, L2 regularization is introduced to control model complexity, prevent overfitting, and enhance the model's generalization ability in complex underwater environments. LoRA fine-tuning technology is used to fine-tune the parameters of the incremental path decision network model in batches, thereby enabling the model's Q-value function to be updated in real time. The Q-value function is a reward function simulated using a deep reinforcement learning model. The incremental path decision network model updated after LoRA fine-tuning is used to update the path planning of the autonomous underwater vehicle, thereby realizing incremental path dynamic decision-making based on the newly collected batch of environmental information, avoiding dynamic obstacles such as marine life, and optimizing the path planning of the autonomous underwater vehicle.

[0073] This embodiment provides a control method for autonomous underwater vehicles (AUVs) capable of rapidly responding to changes in dynamic environments. The proposed AUV control method, based on an incremental path decision network, effectively reduces model complexity and improves real-time decision-making capabilities by introducing L2 regularization and an incremental path dynamic decision-making mechanism. This method not only reduces the risk of collisions with dynamic underwater targets but also enables autonomous path planning in complex environments, thereby enhancing the safety and reliability of AUVs.

[0074] Implementation Method Two: This implementation method further defines the autonomous underwater vehicle control method based on incremental regularized networks described above. In this implementation method, the deep Q-network with L2 regularization in step S1 is further defined, specifically including:

[0075] To improve the generalization performance of the model, in step S1, the deep Q-network with L2 regularization is constructed according to the following steps:

[0076] Step 1-1: Construct the state-space representation of the autonomous underwater vehicle according to the following formula:

[0077]

[0078] in, SThis represents the state space representation of an autonomous underwater vehicle. s t This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, t Represents the current timestamp, t end A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The state vector consists of four variables, which represent the vector of the current position of the autonomous underwater vehicle (AUV), the vector of the current velocity of the AUV, the current heading angle of the AUV, and the spatial information of the obstacles around the AUV.

[0079] Steps 1-2: Construct the motion space representation of the autonomous underwater vehicle based on the following formula:

[0080]

[0081] in, A This represents the action space representation of an autonomous underwater vehicle. a t This indicates that the autonomous underwater vehicle is at a time step t The action space of time, t Represents the current timestamp, t end A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The resulting state vector consists of two variables, which represent the vector representation of the change in the autonomous underwater vehicle's velocity and the vector representation of the change in the autonomous underwater vehicle's heading angle, respectively.

[0082] Steps 1-3: Train a deep Q-network, in which, during the iterative process based on deep reinforcement learning, the estimated future cumulative reward obtained for each attempt is calculated according to the following formula:

[0083]

[0084] in, s t This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, s t+1 This indicates that the autonomous underwater vehicle is at a time step t Environmental information at +1, t Represents the current timestamp, a t This indicates that the autonomous underwater vehicle is at a time step tThe action space of time, a’ This indicates the next state. s t+1 The optimal action among all possible actions; max represents the function that takes the maximum value; Represents the parameters of a deep Q-network. Indicates in t Always in the environment s t Next action a t When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; r t Indicates the state s t Execute action a t The instant reward obtained afterward This represents the discount factor, which controls the importance of future rewards. These represent the parameters of the target network, which are used as parameters for delayed updates to improve the stability of training.

[0085] Steps 1-4: During the training of the deep Q-network, initialize its loss function according to the following formula:

[0086]

[0087] in, The initial value of the loss function is... Represents the parameters of a deep Q-network. Indicates in t Always in the environment s t Next action a t When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; s t This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, s t+1 This indicates that the autonomous underwater vehicle is at a time step t Environmental information at +1, t Represents the current timestamp, a t This indicates that the autonomous underwater vehicle is at a time step t The action space of time, r t Indicates the state s t Execute action at The instant reward obtained afterward; D This represents the experience replay pool. Indicates from the experience replay pool D The operation function for extracting samples for training; y t Indicates time t The target Q value is defined below, where the Q value represents an estimated reward.

[0088] Steps 1-5: During the calculation of the loss function in the deep Q-network, L2 regularization is introduced to correct the loss function, thereby preventing the network parameters from becoming too large and improving the model's generalization ability.

[0089] Steps 1-6: During the model update process using the loss function in the deep Q-network, the model parameters θ are updated according to the following formula to minimize the loss function with L2 regularization:

[0090]

[0091] in, Represents the parameters of a deep Q-network. The symbol represents an update operation; α The learning rate is used to control the step size for parameter updates. Indicates to The operation function for calculating the gradient, This indicates that the loss function is applied to the parameters. θ The gradient is used to update the model weights; This represents the final loss function obtained after introducing L2 regularization for correction.

[0092] This embodiment provides a specific method for constructing a deep Q-network with L2 regularization to improve the generalization performance of the model. Steps 1-3 are used to improve the stability of training; the formulas in steps 1-5 introduce L2 regularization to correct the loss function, thereby preventing the network parameters from becoming too large and improving the generalization ability of the model; steps 1-6 are used to minimize the loss function with L2 regularization.

[0093] By introducing L2 regularization, the aim is to effectively reduce model complexity. The introduction of L2 regularization improves the overall performance evaluation by 21.0% compared to the DQN model. The DQN model refers to the Deep Q-Network model, a reinforcement learning algorithm combining deep learning and Q-learning. The overall performance refers to the average of planning ability and obstacle avoidance ability. Planning ability is the inverse of the path length planned by the model; a larger value indicates a shorter path. Obstacle avoidance ability is the inverse of the number of times the model encounters obstacles or risk areas along the planned path; a larger value indicates a higher probability of the autonomous underwater vehicle avoiding obstacles.

[0094] Implementation method three further defines the autonomous underwater vehicle control method based on incremental regularization networks described above. In this implementation, the formula for introducing L2 regularization is further defined, specifically including:

[0095] In steps 1-5, L2 regularization is introduced to modify the loss function according to the following formula:

[0096]

[0097] in, This represents the final loss function obtained after introducing L2 regularization for correction. The initial value of the loss function is... Represents the parameters of a deep Q-network. This represents the regularization strength coefficient, used to control the weights of L2 regularization; i represents the index of the weight parameter. Represents the weight parameters. Represents all network weight parameters The sum of squares.

[0098] This embodiment provides a method for modifying the loss function by introducing L2 regularization. The loss function is initialized during the training process of the deep Q-network using the formulas in steps 1-4. Then, the L2 regularization formula from steps 1-5 is incorporated into the loss function minimization optimization process of the deep Q-network in steps 1-6. This embodiment combines the method of combining formulas from steps 1-4 with those from steps 1-6 (i.e., the deep Q-network training and optimization process) with the method of formulas from steps 1-5 (i.e., the L2 regularization process). This allows for a more effective introduction of L2 regularization to control model complexity, and the model becomes smoother by adding the sum of squared parameters to the loss function.

[0099] Implementation Method Four: This implementation method further defines the autonomous underwater vehicle control method based on incremental regularized networks described above. In this implementation method, the parameters of the incremental path decision network model in step 3, using LoRA fine-tuning technology to fine-tune in batches, are further defined, specifically including:

[0100] To enhance the decision-making ability of autonomous underwater vehicles (AUVs) in dynamic underwater environments, in step S3, the parameters of the incremental path decision network model are fine-tuned in batches using LoRA fine-tuning technology according to the following steps. The incremental path decision network model refers to a deep reinforcement learning model trained using an incremental path dynamic decision-making mechanism. LoRA refers to the low-rank decomposition model fine-tuning technique.

[0101] Step 3-1: During the dynamic decision-making process, the autonomous underwater vehicle continuously updates its Q-value function based on real-time data collected by sensors and makes the optimal path decision according to the current state. The key is that the Q-value function is incrementally updated over time to adapt to the constantly changing environment. This incremental update mechanism means that each time the sensor detects new information, the model incrementally updates the Q-value function based on this new data, gradually approaching the optimal path planning. The Q-value function represents the function for calculating the reward estimate.

[0102] Step 3-2: Perform low-rank decomposition according to the following formula:

[0103]

[0104] in, W This is the updated weight matrix obtained using the low-rank decomposition operation; Let represent the original weight matrix, and represent the parameters of the model pre-training, given by . It is extracted from a deep Q-network; Indicates the amount of model parameter updates; A and B All are low-rank matrices. d , r , k Both represent the dimensions of the matrix; Represent a matrix over the real number field. Represent a d OK r A matrix over the real field of columns, Represent a r OK k A matrix over the real number field of a column;

[0105] Step 3-3: LoRA fine-tuning is performed in batches. New environmental data collected by the sensors is transmitted to the incremental path decision network model in batches, and each batch triggers a LoRA fine-tuning. This mechanism ensures that the model can gradually adapt to the constantly changing environment without requiring frequent full updates. Each batch of data collected and the operation of updating the low-rank matrix using this batch of data satisfies the following formula:

[0106]

[0107] in, This represents the batch loss function used to update the parameters of LoRA fine-tuning, where LoRA refers to the low-rank decomposition model fine-tuning technique. This represents the parameters of the deep Q-network. N represents the batch size, which is the number of data samples contained in each batch. yt Let Q represent the target Q value at time t, where Q value represents an estimated reward. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward is obtained by fitting a deep Q-network. i, j, k Both represent the number of the current data sample in the current batch. si This indicates that autonomous underwater vehicles are in the data sample i The following environmental information, ai This indicates that autonomous underwater vehicles are in the data sample i The action space below; This represents the target Q value, and the actual reward value for the current state. A and B All are low-rank matrices. Aj and Bk These represent the low-rank matrices in LoRA fine-tuning. A and B The column vector is used to represent some of the parameters updated by the model. This represents the regularization coefficient, used to control the magnitude of low-rank matrix updates and avoid overfitting.

[0108] Steps 3-4: After each LoRA fine-tuning, the Q-value function of the incremental path decision network is updated in real time, ensuring that the autonomous underwater vehicle (AUV) can dynamically adjust its path planning strategy based on new information. The incremental update of the Q-value function means that whenever the AUV's sensors collect new environmental data, the network parameters adjusted through LoRA fine-tuning are reflected in the Q-value function, allowing the model to better adapt to the current environment. The Q-value represents the estimated reward.

[0109] This embodiment provides a method for batch fine-tuning the parameters of an incremental path decision network model using LoRA fine-tuning technology. The incremental path decision network model refers to a deep reinforcement learning model trained using an incremental path dynamic decision-making mechanism. LoRA refers to low-rank decomposition model fine-tuning technology, which is used to enhance the decision-making ability of autonomous underwater vehicles in dynamic underwater environments.

[0110] In step 3-3, the batch-based LoRA fine-tuning mechanism ensures that the model can gradually adapt to the constantly changing environment without requiring frequent full updates; the regularization coefficient used to control the magnitude of low-rank matrix updates... To avoid overfitting, the Q-value function of the incremental path decision network is updated in real time after each LoRA fine-tuning, ensuring that the autonomous underwater vehicle (AUV) can dynamically adjust its path planning strategy based on new information. The incremental update mechanism of the Q-value function means that whenever the AUV's sensors collect new environmental data, the network parameters adjusted through LoRA fine-tuning are reflected in the Q-value function, allowing the model to better adapt to the current environment.

[0111] By introducing LoRA fine-tuning technology, the aim is to improve the model's real-time decision-making ability. The introduction of LoRA fine-tuning technology resulted in a 27.6% improvement in the overall performance evaluation compared to the DQN model. The DQN model refers to the Deep Q-Network model, a reinforcement learning algorithm model that combines deep learning and Q-learning. The overall performance refers to the average of planning ability and obstacle avoidance ability. Planning ability is the inverse of the path length planned by the model; a larger value indicates a shorter path. Obstacle avoidance ability is the inverse of the number of times the model encounters obstacles or risk areas along the planned path; a larger value indicates a higher probability that the autonomous underwater vehicle (AUV) will avoid obstacles.

[0112] Implementation method five further defines the autonomous underwater vehicle control method based on incremental regularization networks described above. In this implementation, the update formula for the Q-value function is further defined, specifically including:

[0113] In steps 3-4, the Q-value function is incrementally updated according to the following steps:

[0114]

[0115] in, This represents the Q value after the incremental update. This represents the Q-value of the model before the update. This represents the maximum Q value among all possible actions for the next step. This represents the parameters of the depth-Q network after incremental updates. This represents the depth-Q network parameters before the incremental update. This represents the parameters of the deep Q-network when the maximum Q-value is obtained among all possible actions for the next step. st This represents the environmental information of the autonomous underwater vehicle at time step t. st +1 indicates the environmental information of the autonomous underwater vehicle at time step t+1, where t represents the current timestamp. at This represents the action space of an autonomous underwater vehicle at time step t. a’ This indicates the next state. st +1 represents the optimal action among all possible actions. max denotes the function that takes the maximum value. The learning rate determines the update step size. This indicates that the autonomous underwater vehicle is at a time step t When in a state st Next action at The instant reward obtained afterward.

[0116] This embodiment provides an incremental update method for the Q-value function. This method, combined with the LoRA low-rank decomposition method in step 3-2 and the network parameter update method in step 3-3, can make the application of LoRA technology in deep Q-networks more effective in the control of the autonomous underwater vehicle of this application, and improve the real-time decision-making capability of the model.

[0117] Implementation method six is ​​a further refinement of the autonomous underwater vehicle control method based on incremental regularization networks described above. In this implementation method, the calculation formula for the return constraint of the autonomous underwater vehicle in step 5 is further refined, specifically including:

[0118] To reduce the risk of the autonomous underwater vehicle (AUV) being unable to return, in step S5, the return constraint of the AUV is calculated according to the following formula:

[0119]

[0120] Where CON represents the return constraint, a value of 0 indicates that the return constraint is not met, and a value of 1 indicates that the return constraint is met. max is the function to find the maximum value. This indicates a command constraint; its value is 1 when a return command is received from the control center, and 0 otherwise. This indicates the maximum sailing time constraint. t Represents the current timestamp. Indicates the maximum sailing time; Indicates battery constraint. Indicates the parameters of a deep Q-network; Indicates timet The remaining battery power; The function for estimating the power consumption during the return trip represents the function that utilizes the parameter as... The parameters of the deep Q-network, in time t It is estimated that the amount of electricity it will consume to return to the starting point will be used in time; This indicates that reserve power is in case unexpected situations occur during the return journey, resulting in additional power consumption and preventing the return to the starting point.

[0121] Implementation method seven is an embodiment of an autonomous underwater vehicle control method based on incremental regularization networks as described above. Figure 1 As shown, an autonomous underwater vehicle control method based on incremental path decision networks includes:

[0122] S1: Obtain the historical underwater navigation database of AUVs from the data center. AUV stands for Autonomous Underwater Vehicle. Train the model using a deep Q-network and introduce L2 regularization to control model complexity, prevent overfitting, and enhance the model's generalization ability in complex underwater environments. L2 regularization is a method that smooths the model by adding a sum of squared parameters to the loss function.

[0123] In step S1, the deep Q-network with L2 regularization is constructed according to the following steps:

[0124] Step 1-1: Construct the state-space representation of the autonomous underwater vehicle according to the following formula:

[0125]

[0126] in, S This represents the state space representation of an autonomous underwater vehicle. st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The state vector consists of four variables, which represent the vector of the current position of the autonomous underwater vehicle (AUV), the vector of the current velocity of the AUV, the current heading angle of the AUV, and the spatial information of the obstacles around the AUV.

[0127] Steps 1-2: Construct the motion space representation of the autonomous underwater vehicle based on the following formula:

[0128]

[0129] in,A This represents the action space representation of an autonomous underwater vehicle. at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The resulting state vector consists of two variables, which represent the vector representation of the change in the autonomous underwater vehicle's velocity and the vector representation of the change in the autonomous underwater vehicle's heading angle, respectively.

[0130] Steps 1-3: Train a deep Q-network, in which, during the iterative process based on deep reinforcement learning, the estimated future cumulative reward obtained for each attempt is calculated according to the following formula:

[0131]

[0132] in, st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, st +1 indicates that the autonomous underwater vehicle is at a certain time step. t Environmental information at +1, t Represents the current timestamp, at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, a’ This indicates the next state. st +1 represents the optimal action among all possible actions; max represents the function that takes the maximum value. Represents the parameters of a deep Q-network. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; rt Indicates the state st Execute action at The instant reward obtained afterward This represents the discount factor, which controls the importance of future rewards. These represent the parameters of the target network, which are used as parameters for delayed updates to improve the stability of training.

[0133] Steps 1-4: During the training of the deep Q-network, initialize its loss function according to the following formula:

[0134]

[0135] in, The initial value of the loss function is... Represents the parameters of a deep Q-network. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, st +1 indicates that the autonomous underwater vehicle is at a certain time step. t Environmental information at +1, t Represents the current timestamp, at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, rt Indicates the state st Execute action at The instant reward obtained afterward; D This represents the experience replay pool. Indicates from the experience replay pool D The operation function for extracting samples for training; yt Indicates time t The target Q value is defined below, where the Q value represents an estimated reward.

[0136] Steps 1-5: During the calculation of the loss function in the deep Q-network, L2 regularization is introduced according to the following formula to correct the loss function, thereby preventing the network parameters from becoming too large and improving the model's generalization ability:

[0137]

[0138] in, This represents the final loss function obtained after introducing L2 regularization for correction. The initial value of the loss function is... Represents the parameters of a deep Q-network. This represents the regularization strength coefficient, used to control the weights of L2 regularization; i represents the index of the weight parameter. Represents the weight parameters. Represents all network weight parameters The sum of squares.

[0139] Steps 1-6: During the model update process using the loss function in the deep Q-network, the model parameters are updated according to the following formula. To minimize the loss function with L2 regularization:

[0140]

[0141] in, Represents the parameters of a deep Q-network. The symbol represents an update operation; α The learning rate is used to control the step size for parameter updates. Indicates to The operation function for calculating the gradient, This indicates that the loss function is applied to the parameters. θ The gradient is used to update the model weights; This represents the final loss function obtained after introducing L2 regularization for correction.

[0142] S2: The autonomous underwater vehicle starts from the starting point and performs initial path planning based on the initial environmental information collected by the sensors;

[0143] S3: An incremental path dynamic decision-making mechanism is introduced. The autonomous underwater vehicle's sensors collect surrounding environmental data in real time, and the parameters of the incremental path decision-making network model are fine-tuned in batches using LoRA fine-tuning technology, thereby enabling the model's Q-value function to be updated in real time. The Q-value function means the reward function simulated using a deep reinforcement learning model. The incremental path decision-making network model means a deep reinforcement learning model trained using the incremental path dynamic decision-making mechanism. LoRA refers to the low-rank decomposition model fine-tuning technology.

[0144] In step S3, the parameters of the incremental path decision network model are fine-tuned in batches using the LoRA fine-tuning technique according to the following steps. The incremental path decision network model refers to a deep reinforcement learning model trained using the incremental path dynamic decision-making mechanism. LoRA refers to the low-rank decomposition model fine-tuning technique.

[0145] Step 3-1: During the dynamic decision-making process, the autonomous underwater vehicle continuously updates its Q-value function based on real-time data collected by sensors and makes the optimal path decision according to the current state. The key lies in the incremental update mechanism of the Q-value function over time steps to adapt to the constantly changing environment. This incremental update mechanism means that each time the sensor detects new information, the model incrementally updates the Q-value function based on this new data, gradually approaching the optimal path planning. The Q-value function represents the function for calculating the reward estimate.

[0146] Step 3-2: Perform low-rank decomposition according to the following formula:

[0147]

[0148] in, W This is the updated weight matrix obtained using the low-rank decomposition operation; Let represent the original weight matrix, and represent the parameters of the model pre-training, given by . It is extracted from a deep Q-network; Indicates the amount of model parameter updates; A and B All are low-rank matrices. d , r , k Both represent the dimensions of the matrix; Represent a matrix over the real number field. Represent a d OK r A matrix over the real field of columns, Represent a r OK k A matrix over the real number field of a column;

[0149] Step 3-3: LoRA fine-tuning is performed in batches. New environmental data collected by the sensors is transmitted to the incremental path decision network model in batches, and each batch triggers a LoRA fine-tuning. This mechanism ensures that the model can gradually adapt to the constantly changing environment without requiring frequent full updates. Each batch of data collected and the operation of updating the low-rank matrix using this batch of data satisfies the following formula:

[0150]

[0151] in, This represents the batch loss function used to update the parameters of LoRA fine-tuning, where LoRA refers to the low-rank decomposition model fine-tuning technique. This represents the parameters of the deep Q-network; N represents the batch size, which is the number of data samples contained in each batch. yt Indicates time t The target Q value is given below, where the Q value represents an estimated reward. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; i, j, k Both represent the ID of the current data sample in the current batch; si This indicates that autonomous underwater vehicles are in the data sample i The following environmental information, ai This indicates that autonomous underwater vehicles are in the data sample i The action space below; This represents the target Q value, and the actual reward value for the current state. A and B All are low-rank matrices. Aj and Bk These represent the low-rank matrices in LoRA fine-tuning.A and B The column vector is used to represent some of the parameters updated by the model; This represents the regularization coefficient, used to control the magnitude of low-rank matrix updates and avoid overfitting.

[0152] Steps 3-4: After each LoRA fine-tuning, the Q-value function of the incremental path decision network is updated in real time, ensuring that the autonomous underwater vehicle (AUV) can dynamically adjust its path planning strategy based on new information. The incremental update mechanism of the Q-value function means that whenever the AUV's sensors collect new environmental data, the network parameters adjusted through LoRA fine-tuning are reflected in the Q-value function, allowing the model to better adapt to the current environment. The Q-value represents the estimated reward. The Q-value function is incrementally updated according to the following steps:

[0153]

[0154] in, This represents the Q value after the incremental update. This represents the Q-value of the model before the update. This represents the maximum Q value among all possible actions for the next step. This represents the parameters of the depth-Q network after incremental updates. This represents the depth-Q network parameters before the incremental update. This represents the parameters of the deep Q-network when the maximum Q-value is obtained among all possible actions for the next step. st This represents the environmental information of the autonomous underwater vehicle at time step t. [[ID= +1 indicates the environmental information of the autonomous underwater vehicle at time step t+1, where t represents the current timestamp. ​ This represents the action space of an autonomous underwater vehicle at time step t. a’ This indicates the next state. ​ +1 represents the optimal action among all possible actions. max denotes the function that takes the maximum value. The learning rate determines the step size for updating.

[0155] ​ This represents the path planning results of an autonomous underwater vehicle based on an incremental path decision network according to one or more embodiments. ​ As shown, the autonomous underwater vehicle (AUV) travels from the starting point to the destination. The gray area represents an obstacle detected by the AUV at that location. The planning results based on the simulation experiment are plotted as a polyline on... ​ In. ​ Cases 1, 2, 3, and 4 described in the text refer to the different results of multiple path planning by the autonomous underwater vehicle based on the incremental path decision network, illustrating the convergence and robustness of the results.

[0156] S4: The incremental path decision network model after LoRA fine-tuning is used to update the path planning of the autonomous underwater vehicle, thereby realizing the incremental path dynamic decision-making for the newly collected environmental information, avoiding dynamic obstacles such as marine organisms, and optimizing the path planning of the autonomous underwater vehicle.

[0157] In one or more embodiments disclosed in this application, the evaluation results of the autonomous underwater vehicle based on the incremental path decision network model are shown in Table 1. The evaluation results include three dimensions: planning ability, obstacle avoidance ability, and overall level. The value of each dimension is obtained by dividing by the maximum value of the row, so the results are all distributed in the interval between 0 and 1. The planning ability refers to the negative number of the path length planned by the model, and the larger the value, the shorter the path. The obstacle avoidance ability refers to the negative number of times the autonomous underwater vehicle encounters obstacles or risk areas in the path planned by the model, and the larger the value, the greater the probability of the autonomous underwater vehicle avoiding obstacles. The overall level refers to the average of the planning ability and the obstacle avoidance ability. Table 1 shows the evaluation results of the method of this application, the method of this application-L2 regularization, the method of this application-LoRA fine-tuning, and the DQN model in the three dimensions of planning ability, obstacle avoidance ability, and overall level. The method-L2 regularization in this application refers to the model obtained by removing the L2 regularization module from the model proposed in this application. The method-LoRA fine-tuning in this application refers to the model obtained by removing the LoRA fine-tuning module from the model proposed in this application. The DQN model refers to the Deep Q-Network model, which is a reinforcement learning algorithm model that combines deep learning and Q-learning. It can be seen that by introducing the L2 regularization method or the LoRA fine-tuning technique, the overall performance evaluation results are improved by 21.0% and 27.6% respectively compared to the DQN model. This invention integrates L2 regularization and LoRA fine-tuning techniques, resulting in a 46.1% improvement in overall performance evaluation compared to the DQN model. Furthermore, the planning and obstacle avoidance capabilities are improved by 24.7% and 67.5% respectively compared to the DQN model. This demonstrates that the invention effectively utilizes L2 regularization to reduce model complexity and improve computational efficiency, while simultaneously leveraging LoRA fine-tuning to enhance real-time decision-making capabilities. Ultimately, this significantly improves the autonomous underwater vehicle's (AUV) path planning ability based on dynamic environmental information and substantially enhances its obstacle avoidance capabilities in dynamic environments.

[0158] Table 1 Evaluation results of autonomous underwater vehicles based on incremental path decision network model.

[0159]

[0160] S5: The autonomous underwater vehicle checks whether the return-to-home constraint is met. If the return-to-home constraint is not met, return to step S3, continue underwater navigation, and use the collected data to fine-tune the model and update the path. If the return-to-home constraint is met, proceed to step S6;

[0161] S6: The autonomous underwater vehicle (AUV) invokes the incremental path decision network to generate a path planning scheme for returning to the starting point from the current position, and then returns according to the scheme. After arriving at the starting point, the AUV transmits the underwater navigation data back to the historical underwater navigation database in the data center.

Claims

1. An autonomous underwater vehicle control method based on an incrementally regularized network, characterized by, The method includes: Step 1: Obtain the historical underwater navigation database of AUVs, train the model using a deep Q-network, and control the model complexity by introducing L2 regularization. Step 2: The autonomous underwater vehicle starts from the starting point and performs initial path planning based on the initial environmental information collected by the sensors; Step 3: Train a deep reinforcement learning model using the incremental path dynamic decision-making mechanism to obtain an incremental path decision network model. The autonomous underwater vehicle's sensors collect surrounding environmental data in real time. Use LoRA fine-tuning technology to fine-tune the parameters of the incremental path decision network model in batches. The LoRA is a low-rank decomposition model fine-tuning technology. Step 4: Update the path planning of the autonomous underwater vehicle using the incremental path decision network model fine-tuned in Step 3; Step 5: The autonomous underwater vehicle checks whether the return-to-home constraint is met. If the return-to-home constraint is not met, it returns to Step 3 to continue underwater navigation and uses the collected data to fine-tune the model and update the path; if the return-to-home constraint is met, it proceeds to Step 6. Step 6: The autonomous underwater vehicle (AUV) calls the incremental path decision network to generate a path planning scheme for returning to the starting point from the current position, and returns according to the scheme; after the AUV arrives at the starting point, it transmits the underwater navigation data of this trip back to the historical underwater navigation database of the data center.

2. The autonomous underwater vehicle control method based on the incremental regularization network according to claim 1, wherein, In step 1, the method of controlling model complexity by introducing L2 regularization specifically involves adding the sum of squared parameters to the loss function to make the model smoother.

3. The autonomous underwater vehicle control method based on the incremental regularization network according to claim 1, wherein, In step 1, the method of controlling model complexity by introducing L2 regularization specifically includes: Step 1.1: Construct the state space representation of the autonomous underwater vehicle: in, S This represents the state space representation of an autonomous underwater vehicle. st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The state vector consists of four variables, which represent the vector of the current position of the autonomous underwater vehicle (AUV), the vector of the current velocity of the AUV, the current heading angle of the AUV, and the spatial information of the obstacles around the AUV. Step 1.2: Construct the motion space representation of the autonomous underwater vehicle: in, A This represents the action space representation of an autonomous underwater vehicle. at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, t Represents the current timestamp, tend A timestamp indicating the final moment of the voyage's end. end Mark the end of the voyage; Indicates by The resulting state vector consists of two variables, which represent the vector representation of the change in the autonomous underwater vehicle's velocity and the vector representation of the change in the autonomous underwater vehicle's heading angle, respectively. Step 1.3: Train a deep Q-network. During the iterative process based on deep reinforcement learning, calculate the estimated future cumulative reward for each attempt using the following formula: in, s t This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, s t+1 This indicates that the autonomous underwater vehicle is at a time step t Environmental information at +1, t Represents the current timestamp, a t This indicates that the autonomous underwater vehicle is at a time step t The action space of time, a’ This indicates the next state. s t+1 The optimal action among all possible actions; max represents the function that takes the maximum value; Represents the parameters of a deep Q-network. Indicates in t Always in the environment s t Next action a t When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; r t Indicates the state s t Execute action a t The instant reward obtained afterward This represents the discount factor, which controls the importance of future rewards. These represent the parameters of the target network, which are used as parameters for delayed updates to improve the stability of training. Step 1.4: During the training of the deep Q-network, initialize its loss function according to the following formula: in, The initial value of the loss function is... Represents the parameters of a deep Q-network. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; st This indicates that the autonomous underwater vehicle is at a time step t Environmental information at the time, st +1 indicates that the autonomous underwater vehicle is at a certain time step. t Environmental information at +1, t Represents the current timestamp, at This indicates that the autonomous underwater vehicle is at a time step t The action space of time, rt Indicates the state st Execute action at The instant reward obtained afterward; D This represents the experience replay pool. Indicates from the experience replay pool D The operation function for extracting samples for training; yt Indicates time t The target Q value is given below, where the Q value represents an estimated reward. Step 1.5: Introduce L2 regularization to modify the loss function; Step 1.6: Update the model parameters according to the following formula To minimize the loss function: in, Represents the parameters of a deep Q-network. The symbol represents an update operation; α The learning rate is used to control the step size for parameter updates. Indicates to The operation function for calculating the gradient, This indicates that the loss function is applied to the parameters. θ The gradient is used to update the model weights; This represents the final loss function obtained after introducing L2 regularization for correction.

4. The autonomous underwater vehicle control method based on incremental regularization networks according to claim 3, characterized in that, In step 1.5, L2 regularization is introduced to modify the loss function according to the following formula: in, This represents the final loss function obtained after introducing L2 regularization for correction. The initial value of the loss function is... Represents the parameters of a deep Q-network. This represents the regularization strength coefficient, used to control the weights of L2 regularization; i represents the index of the weight parameter. Represents the weight parameters. Represents all network weight parameters The sum of squares.

5. The autonomous underwater vehicle control method based on incremental regularization networks according to claim 1, characterized in that, Step 3, specifically the step of using LoRA fine-tuning technology to fine-tune the parameters of the incremental path decision network model in batches, includes: Step 3.1: During the dynamic decision-making process, the autonomous underwater vehicle continuously updates its Q-value function based on real-time data collected by sensors. The Q-value function represents the calculation function of the reward estimate. Based on the current state, it makes the optimal path decision. The Q-value function is incrementally updated over time to adapt to the constantly changing environment. Specifically, each time the sensor detects new information, the model incrementally updates the Q-value function based on this new data, gradually approaching the optimal path planning. Step 3.2: Perform low-rank decomposition according to the following formula: in, W This is the updated weight matrix obtained using the low-rank decomposition operation; Let represent the original weight matrix, and represent the parameters of the model pre-training, given by . It is extracted from a deep Q-network; Indicates the amount of model parameter updates; A and B All are low-rank matrices. d , r , k Both represent the dimensions of the matrix; Represent a matrix over the real number field. Represent a d OK r A matrix over the real field of columns, Represent a r OK k A matrix over the real number field of a column; Step 3.3: The new environmental data collected by the sensors is transmitted to the incremental path decision network model in batches, and each batch triggers a LoRA fine-tuning; the operation of updating the low-rank matrix using this batch of data satisfies the following formula: in, This represents the batch loss function used to update the parameters of LoRA fine-tuning; This represents the parameters of the deep Q-network; N represents the batch size, which is the number of data samples contained in each batch. yt Indicates time t The target Q value is given below, where the Q value represents an estimated reward. Indicates in t Always in the environment st Next action at When, the parameter is The estimated value of the future cumulative reward obtained by fitting a deep Q-network; i, j, k Both represent the ID of the current data sample in the current batch; si This indicates that autonomous underwater vehicles are in the data sample i The following environmental information, ai This indicates that autonomous underwater vehicles are in the data sample i The action space below; This represents the target Q value, and the actual reward value for the current state. A and B All are low-rank matrices. Aj and Bk These represent the low-rank matrices in LoRA fine-tuning. A and B The column vector is used to represent some of the parameters updated by the model; This represents the regularization coefficient, used to control the magnitude of low-rank matrix updates; Step 3.4: Incrementally update the Q-value function.

6. The autonomous underwater vehicle control method based on incremental regularization networks according to claim 5, characterized in that, In step 3.4, the Q-value function is incrementally updated according to the following formula: in, This represents the Q value after the incremental update. This represents the Q-value of the model before the update; This represents the maximum Q value among all possible actions for the next step; This represents the parameters of the depth-Q network after incremental updates. This represents the depth-Q network parameters before the incremental update. This represents the parameters of the deep Q-network when the maximum Q-value is obtained among all possible actions in the next step; max represents the function that takes the maximum value. The learning rate determines the update step size. This indicates that the autonomous underwater vehicle is at a time step t When in a state st Next action at The instant reward obtained afterward.

7. The autonomous underwater vehicle control method based on incremental regularization networks according to claim 1, characterized in that, In step 5, the formula for calculating the return constraint of the autonomous underwater vehicle is: in, CON This represents the return-to-home constraint. A value of 0 indicates that the return-to-home constraint is not met, while a value of 1 indicates that the return-to-home constraint is met. This indicates a command constraint; its value is 1 when a return command is received from the control center, and 0 otherwise. This indicates the maximum sailing time constraint. t Represents the current timestamp. Indicates the maximum sailing time; Indicates battery constraint. Indicates the parameters of a deep Q-network; Indicates time t The remaining battery power; The function for estimating the power consumption during the return trip represents the function that utilizes the parameter as... The parameters of the deep Q-network, in time t It can estimate the amount of electricity it will consume to return to the starting point. This indicates that reserved power is available.

8. An autonomous underwater vehicle control system based on an incremental regularized network according to any one of claims 1-7, characterized in that, The system includes: The data acquisition module is used to collect historical data and real-time environmental data during the navigation of the autonomous underwater vehicle. The data transmission module is used to obtain historical navigation data from the data center and transmit the navigation data back to the data center after the navigation is completed; The model training module is used to train a deep reinforcement learning model with L2 regularization for path planning based on the collected data. The fine-tuning module, based on LoRA fine-tuning technology, dynamically updates model parameters to achieve real-time path planning adjustments; The path decision module is used to output the motion path of the autonomous underwater vehicle based on the incremental path decision network, avoid dynamic obstacles, and optimize the navigation path.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The steps of the method according to any one of claims 1 to 7 are performed when the processor runs the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer instructions, which are used to cause a computer to perform the method of any one of claims 1 to 7.

Images