An underwater autonomous vehicle dynamic obstacle avoidance method and system based on a threshold adaptive brain model, and a storage medium

Abstract

This invention belongs to the field of obstacle avoidance technology for underwater autonomous vehicles (AUVs). First, it acquires the original observation state of the AUV; then, it constructs a dynamic threshold adaptive module to rationally adjust the pulse trigger rate based on dynamic threshold changes; next, it constructs a pulse encoder and a pulse decoder to achieve mutual conversion between continuous information and pulse sequence information; finally, it constructs a threshold adaptive brain-like model, including a pulse dynamic threshold Actor network and a deep Critic network, deploying the dynamic threshold adaptive module in each spiking neuron of the Actor network; and finally, it inputs the original observation state into the trained model and outputs action commands to control the vehicle's obstacle avoidance and navigation. This invention utilizes the dynamic threshold changes of spiking neurons to rationally adjust the trigger rate, maintaining model equilibrium and ensuring that the AUV possesses safe, stable, and efficient dynamic obstacle avoidance capabilities. It has significant application value for achieving dynamic obstacle avoidance tasks for AUVs in complex and unknown environments.

Description

Technical Field

[0001] This invention belongs to the field of obstacle avoidance technology for underwater autonomous vehicles, specifically relating to a dynamic obstacle avoidance method, system, and storage medium for underwater autonomous vehicles based on a threshold adaptive brain-like model. Background Technology

[0002] As a crucial platform for ocean exploration missions, autonomous underwater vehicles (AUVs) leverage their versatility, high maneuverability, and robust safety to perform complex tasks in the ever-changing and harsh marine environment, replacing human personnel. Obstacle avoidance technology is a core element ensuring the safe navigation of AUVs. Due to the highly dynamic and uncertain nature of the underwater environment, and the lack of prior information, AUVs often face various types of obstacles when operating in unfamiliar waters, especially dynamic obstacles with changing speed and direction. The motion of these obstacles is often difficult to predict accurately; therefore, AUVs need to make reasonable judgments about their behavior and maintain overall system stability even in the presence of external disturbances. This ensures the obstacle avoidance model has generalization capabilities, enabling safe and efficient obstacle avoidance and navigation tasks.

[0003] Spiking neural networks (SNNs) have garnered widespread attention as a new generation of neural network models. Unlike traditional artificial neural networks, SNNs communicate with each other through pulse sequences, firing pulse signals when the membrane potential exceeds a threshold, thus simulating the information transmission mechanism of real biological neurons. SNNs exhibit significant advantages in biological interpretability, spatiotemporal information processing, and energy-efficient computing. These characteristics are highly compatible with the needs of autonomous underwater vehicles (AUVs) for obstacle avoidance in unknown environments, especially in complex, unknown, and dynamic underwater environments with external interference and obstacles. The introduction of SNNs provides a new direction for enhancing the intelligence and autonomy of AUVs.

[0004] However, most current spiking neural network models use fixed static thresholds, which weakens the neurons' ability to maintain internal homeostasis and prevents them from dynamically and adaptively adjusting their balance according to external changes. Especially when facing dynamic obstacles in complex and unknown environments, the model may struggle to maintain stability, affecting obstacle avoidance performance and even causing collisions. Therefore, enabling spiking neurons to adaptively adjust thresholds and maintain the overall model's steady-state balance is crucial for underwater autonomous vehicles to perform dynamic obstacle avoidance tasks in complex and unknown waters. Summary of the Invention

[0005] The purpose of this invention is to...

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold-adaptive brain-like model includes the following steps:

[0008] Step 1: Obtain the raw observation state obtained from the interaction between the underwater autonomous vehicle and the underwater environment, and set the reward function;

[0009] Step 2: Construct a dynamic threshold adaptive module to reasonably adjust the pulse trigger rate according to the dynamic adaptive changes of the threshold;

[0010] Step 3: Construct a pulse encoder and a pulse decoder. The pulse encoder is used to convert continuous information into pulse sequence information, and the pulse decoder is used to convert pulse sequence information into continuous information.

[0011] Step 4: Construct a threshold-adaptive brain-like model, including a spiking dynamic threshold Actor network and a deep Critic network, and deploy the dynamic threshold adaptation module in each spiking neuron of the spiking dynamic threshold Actor network;

[0012] Step 5: Input the original observation state into the trained threshold adaptive brain-like model, and the pulse dynamic threshold Actor network outputs the action command converted by the pulse decoder to control the underwater autonomous vehicle to perform dynamic obstacle avoidance and navigation tasks.

[0013] Furthermore, the original observation state described in step 1 includes: the distance and angle between the underwater autonomous vehicle and the target point, the linear velocity and angular velocity of the underwater vehicle, and the obstacle ranging information returned by the forward-looking sonar;

[0014] The reward function ,

[0015] in,

[0016]

[0017] in, Indicates the reward value. This represents the speed reward function. This represents the reward value for reaching the target point. This indicates a penalty for colliding with an obstacle. and These represent the minimum distances from the center of the autonomous underwater vehicle to the target point and the obstacle, respectively. and These represent the arrival threshold and collision threshold, respectively; d represents the scaling factor; and t represents time. , and It is a constant. and These represent the linear velocity and angular velocity of the autonomous underwater vehicle, respectively.

[0018] Furthermore, the dynamic threshold adaptive module described in step 2, at time t+1, represents the j-th neuron in the n-th layer as follows: ,in, This indicates a dynamic threshold adaptive module. Indicates the adaptive energy threshold. Indicates the dynamic depolarization threshold. and Indicates hyperparameters;

[0019] The adaptive energy threshold It is positively correlated with the membrane potential of spiking neurons and is used to adjust the pulse trigger rate based on the neuron membrane potential level.

[0020]

[0021] The dynamic depolarization threshold It shows a negative correlation with the membrane potential changes of spiking neurons over the past two moments, and is used to adjust the pulse trigger rate based on the trend of membrane potential changes.

[0022]

[0023] in, and Let represent the membrane potential and threshold of the j-th neuron in the n-th layer at time t, respectively. and These represent the average membrane potential and the average trigger threshold of all neurons in the same layer, respectively. and It's a hyperparameter. This represents the change in membrane potential of the j-th neuron in the n-th layer at time t+1 and time t. and It is a constant.

[0024] Furthermore, in step 3, the pulse encoder encodes the original observation state information into a pulse state sequence through normalization and Poisson sampling operations; the pulse decoder obtains the action command information of the underwater autonomous vehicle by averaging the network output results in the time dimension and then transforming and mapping them.

[0025] Furthermore, the spiking dynamic threshold Actor network described in step 4 includes multiple fully connected layers, and each spiking neuron in the fully connected layer deploys the dynamic threshold adaptive module; the deep Critic network includes multiple fully connected layers composed of traditional artificial neurons.

[0026] Furthermore, the neurons in the pulse dynamic threshold Actor network are LIF neurons, and the dynamic threshold adaptive module is deployed in each pulse neuron to dynamically adjust the membrane current, membrane voltage, and pulse firing process.

[0027]

[0028]

[0029]

[0030]

[0031] in, and These represent the membrane current and membrane voltage of a spiking neuron, respectively. and These represent the synaptic weight matrix and the synaptic bias vector, respectively. and These represent the current decay factor and the voltage decay factor, respectively. Indicates an update to the door. Indicates the pulse delivery gate. This refers to the dynamic adaptive threshold.

[0032] Furthermore, the training process of the threshold-adaptive brain-like model described in step 5 includes:

[0033] The original observation state information is converted into a pulse state sequence by the pulse encoder and then input into the pulse dynamic threshold Actor network. The pulse decoder then converts and outputs action information. A reward value is obtained based on the interaction between the AUV and the underwater environment. The network training is stabilized and converged, and a pulse dynamic threshold Actor target network and a deep Critic target network are constructed. A TD loss function is constructed based on the deep Critic network, the deep Critic target network, and the reward value. The deep Critic network is updated using the TD loss function, and the pulse dynamic threshold Actor network is updated using the Q loss function.

[0034]

[0035]

[0036]

[0037] in, The loss function of a deep Critic network is represented. This represents the loss function of the impulse dynamic threshold Actor network. Indicates the reward value. This represents the Q value at the current moment. Represents the original state information. This indicates the output action information. and These represent the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the corresponding target networks. Indicates the discount factor. and represent the parameters of the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the parameters of the corresponding target network.

[0038] Furthermore, the impulsive dynamic threshold Actor network uses STBP for policy gradient backpropagation, and the deep Critic network uses the normal backpropagation algorithm for updating; the parameters of the impulsive dynamic threshold Actor network and the deep Critic network are periodically soft-updated to the parameters of the target network to improve the stability of the learning process, as shown in the following equation:

[0039]

[0040] in, It is a constant.

[0041] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive neuromorphic model.

[0042] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive neuromorphic model.

[0043] The beneficial effects of this invention are as follows:

[0044] 1. The present invention designs a dynamic threshold adaptive module that combines an adaptive energy threshold and a dynamic depolarization threshold. Unlike a fixed static threshold, it uses threshold transformation to dynamically adjust the trigger rate of spiking neurons, thereby maintaining the steady-state balance of the spiking model.

[0045] 2. This invention designs a threshold-adaptive brain-like model that integrates a pulse dynamic threshold Actor network and a deep Critic network. The dynamic threshold adaptation module is deployed in each pulse neuron, and a pulse encoder-decoder is designed to convert continuous information and pulse sequence information. This model is applied to the dynamic obstacle avoidance task of underwater autonomous vehicles in complex and unknown underwater environments, ensuring the safety, stability and efficiency of underwater dynamic operations, and has important application value. Attached Figure Description

[0046] Figure 1 This is a flowchart of the dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold adaptive brain-like model, as described in this invention.

[0047] Figure 2 This is a schematic diagram illustrating the dynamic obstacle avoidance of the underwater autonomous vehicle of the present invention in an underwater environment;

[0048] Figure 3 This is a schematic diagram of the dynamic threshold adaptive module of the present invention;

[0049] Figure 4 This is a diagram showing the overall structure of the threshold-adaptive brain-like model of the present invention.

[0050] Figure 5 This is a schematic diagram of the dynamic threshold change of the spiking neuron in this invention;

[0051] Figure 6 Visualization results of the successful trajectories of the present invention and the static threshold comparison method in a dynamic testing environment;

[0052] Figure 7 Visualization results of the failure trajectories of the present invention and the static threshold comparison method in a dynamic testing environment;

[0053] Figure 8 Visualization results of the successful trajectories of the present invention and the static threshold comparison method in a mixed dynamic and static test environment;

[0054] Figure 9 This is a visualization of the failure trajectory of the present invention and the static threshold comparison method in a mixed dynamic and static test environment. Detailed Implementation

[0055] The present invention will now be further described with reference to the accompanying drawings.

[0056] This invention discloses a dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold-adaptive brain-like model, such as... Figure 1 This includes the following steps:

[0057] Step 1: Interact with the underwater environment through an autonomous underwater vehicle to obtain the original observation status.

[0058] The initial observation data includes the distance and angle between the autonomous underwater vehicle (AUV) and the target point, the AUV's linear and angular velocities, and the ranging information returned by the forward-looking sonar. Due to the low visibility and poor contrast of optical sensors underwater, and the rapid attenuation and short propagation distance of electromagnetic waves underwater, forward-looking sonar is used as the primary sensor for obstacle detection by the AUV. It can detect a horizontal 120° fan-shaped area, obtaining 18 ranging values ​​as the obstacle detection ranging information returned by the forward-looking sonar. A schematic diagram of dynamic obstacle avoidance in the underwater environment is shown below. Figure 2 As shown; set the reward function as follows:

[0059]

[0060]

[0061]

[0062] in, Indicates the reward value. This represents the speed reward function. This represents the reward value for reaching the target point. This indicates a penalty for colliding with an obstacle. and These represent the minimum distances from the center of the autonomous underwater vehicle to the target point and the obstacle, respectively. and These represent the arrival threshold and collision threshold, respectively; d represents the scaling factor; and t represents time. , and It is a constant. and These represent the linear velocity and angular velocity of the autonomous underwater vehicle, respectively.

[0063] Step 2: Design a dynamic threshold adaptive module to adjust the pulse trigger rate reasonably according to the dynamic adaptive changes of the threshold.

[0064] Unlike fixed static thresholds, dynamic thresholds are a fundamental characteristic of biological nervous systems, more closely mimicking the complex dynamic properties of biological neurons. The dynamic threshold adaptive module combines adaptive energy thresholds and dynamic depolarization thresholds, such as... Figure 3 As shown. Especially when the underwater autonomous vehicle encounters underwater dynamic obstacles, the membrane potential of the spiking neurons is in a state of rapid increase. The dynamic threshold adaptive module should lower the threshold to encourage pulse triggering and respond to external dynamic interference in a timely manner. By dynamically and adaptively adjusting the pulse triggering rate through the threshold, the optimal threshold is achieved and the model balance and stability are maintained.

[0065] The dynamic threshold adaptive module at time t+1, for the j-th neuron in layer n, is as follows:

[0066]

[0067] in, This indicates a dynamic threshold adaptive module. Indicates the adaptive energy threshold. Indicates the dynamic depolarization threshold. and This represents hyperparameters.

[0068] The adaptive energy threshold is positively correlated with the membrane potential of the spiking neuron, as shown below:

[0069]

[0070]

[0071]

[0072] in, and Let represent the membrane potential and threshold of the j-th neuron in the n-th layer at time t, respectively. and These represent the average membrane potential and the average trigger threshold of all neurons in the same layer, respectively. and It's a hyperparameter.

[0073] The dynamic depolarization threshold is negatively correlated with the membrane potential changes of the spiking neuron over the past two time points, as shown below:

[0074]

[0075] in, This represents the change in membrane potential of the j-th neuron in the n-th layer at time t+1 and time t. and It is a constant.

[0076] Step 3: Design a pulse encoder and a pulse decoder. The pulse encoder is used to convert continuous information into pulse sequence information, and the pulse decoder is used to convert pulse sequence information into continuous information.

[0077] The raw observation states acquired by the autonomous underwater vehicle (AUV) through interaction with the underwater environment are in the form of continuous real numbers, which cannot be directly input into the spiking neural network model. Therefore, a spiking encoder is designed to perform normalization and Poisson sampling operations on the raw state information, processing it into spiking state probabilities and spiking state sequences, which are then used as input to the spiking model. After network processing, a spiking decoder is designed to process the network output results into a data format that the AUV can receive by performing time-dimensional superposition and averaging, as well as transformation mapping.

[0078] Step 4: Design a threshold-adaptive brain-like model that combines spiking neural networks and deep reinforcement learning. The model consists of a spiking dynamic threshold Actor network and a deep Critic network. The spiking dynamic threshold Actor network contains four fully connected layers. Unlike the static fixed threshold of spiking neurons, each neuron deploys the designed dynamic threshold adaptive module, which dynamically and appropriately adjusts the threshold size to regulate the spiking trigger rate. The deep Critic network consists of four fully connected layers composed of traditional artificial neurons. The adaptive brain-like model structure is as follows: Figure 4 As shown.

[0079] Specifically, LIF neurons are used as neurons in the spiking dynamic threshold Actor network, and the dynamic threshold adaptation module is deployed to each spiking neuron. The dynamic threshold change of the spiking neuron is as follows: Figure 5 As shown, the membrane current, membrane voltage, and pulse triggering are expressed as follows:

[0080]

[0081]

[0082]

[0083]

[0084] in, and These represent the membrane current and membrane voltage of a spiking neuron, respectively. and These represent the synaptic weight matrix and the synaptic bias vector, respectively. and These represent the current decay factor and the voltage decay factor, respectively. Indicates an update to the door. Indicates the pulse delivery gate. This refers to the dynamic adaptive threshold.

[0085] Step 5: Input the original observation state into the trained threshold adaptive brain-like model, update the weight information, and obtain action commands for the underwater autonomous vehicle to avoid obstacles and navigate in underwater dynamic obstacle environments.

[0086] The original observation state information is converted into a pulse state sequence by the pulse encoder and then input into the pulse dynamic threshold Actor network. The pulse decoder then converts the output action information. A reward value is obtained based on the interaction between the AUV and the underwater environment. To ensure stable convergence of the network training, the original observation state, output action information, reward value, and the original observation state at the next moment are stored as a sequence in an experience pool. A small batch of sequences is then randomly selected for network training. A pulse dynamic threshold Actor target network and a deep Critic target network are constructed respectively. A TD loss function is constructed based on the deep Critic network, the deep Critic target network, and the reward value. The deep Critic network is updated using the TD loss function, and the pulse dynamic threshold Actor network is updated using the Q loss function. The specific details are as follows:

[0087]

[0088]

[0089]

[0090] in, The loss function of a deep Critic network is represented. This represents the loss function of the impulse dynamic threshold Actor network. Indicates the reward value. This represents the Q value at the current moment. Represents the original state information. This indicates the output action information. and These represent the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the corresponding target networks. Indicates the discount factor. and represent the parameters of the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the parameters of the corresponding target network.

[0091] Repeat the above steps, using a soft update method to update the impulse dynamic threshold Actor target network and the deep Critic target network. Representing constants, as shown below:

[0092] .

[0093] Embodiments of the present invention:

[0094] The threshold-adaptive neuromorphic model, trained using the ROS-Gazebo simulator on Ubuntu 18.04, employs a course-based learning approach, sequentially training in four underwater environments of increasing complexity. The model first learns simple obstacle avoidance strategies in simple environments, then learns more complex strategies in complex environments, thus completing multi-scenario, multi-stage dynamic obstacle avoidance strategy training. Two different underwater testing environments are set up: a dynamic obstacle environment and a mixed dynamic and static obstacle environment. The speed and distance of the dynamic obstacles ensure that the autonomous underwater vehicle has sufficient space and time to pass smoothly.

[0095] The test was conducted using 200 randomly generated starting and target points. Figure 6 and Figure 7 These are the visualization results of the success and failure trajectories of the present invention and the static threshold comparison method in a dynamic testing environment. Figure 8 and Figure 9 The images show the visualization results of successful and failed trajectories of the present invention and the static threshold comparison method in a mixed dynamic and static test environment. By comparing with the static threshold method, it can be found that the threshold-adaptive brain-like model of the present invention has more successful trajectories and fewer collisions. The complexity of obstacle distribution differs between the two test environments, with success rates of 85% and 89.5% respectively, higher than the static threshold comparison method. By dynamically adjusting the pulse trigger rate through threshold changes, the model maintains steady state and improves generalization, enabling the underwater autonomous vehicle to possess efficient underwater dynamic obstacle avoidance capabilities.

[0096] In particular, in some preferred embodiments of the present invention, a computer device is also provided, including a memory and a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the underwater autonomous vehicle dynamic obstacle avoidance method based on a threshold adaptive brain-like model described in any of the above embodiments.

[0097] In some other preferred embodiments of the present invention, a computer-readable storage medium is also provided, on which a computer program / instruction is stored, wherein when the computer program is executed by a processor, the steps of the underwater autonomous vehicle dynamic obstacle avoidance method based on a threshold adaptive brain-like model described in any of the above embodiments are implemented.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above embodiments of the dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold adaptive brain-like model, which will not be repeated here.

[0099] Computer-readable storage media encompass a variety of types, including persistent and non-persistent, portable and fixed. These media store information using different technologies, and the content can be machine instructions, data structures, program modules, or other types of data. Some typical examples of computer storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), various types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory and other storage technologies, optical storage media such as CD-ROM and digital video disc (DVD), magnetic storage devices such as magnetic tape and disks, and other non-transferable media used to store information accessible to computing devices. It is important to note that the computer-readable media described herein do not include temporary storage media, such as modulated data signals and carrier waves.

[0100] Those skilled in the art will further recognize that the operation of the module can be achieved using existing technical protocols or programs, without relying on new computer programs themselves. The units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0101] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dynamic obstacle avoidance of underwater autonomous vehicle based on threshold adaptive brain-inspired model, characterized in that, Includes the following steps: Step 1: Obtain the raw observation state obtained from the interaction between the underwater autonomous vehicle and the underwater environment, and set the reward function; Step 2: Construct a dynamic threshold adaptive module to reasonably adjust the pulse trigger rate according to the dynamic adaptive changes of the threshold; Step 3: Construct a pulse encoder and a pulse decoder. The pulse encoder is used to convert continuous information into pulse sequence information, and the pulse decoder is used to convert pulse sequence information into continuous information. Step 4: Construct a threshold-adaptive brain-like model, including a spiking dynamic threshold Actor network and a deep Critic network, and deploy the dynamic threshold adaptation module in each spiking neuron of the spiking dynamic threshold Actor network; Step 5: Input the original observation state into the trained threshold adaptive brain-like model, and the pulse dynamic threshold Actor network outputs the action command converted by the pulse decoder to control the underwater autonomous vehicle to perform dynamic obstacle avoidance and navigation tasks.

2. The dynamic obstacle avoidance method for underwater autonomous vehicle based on threshold adaptive brain-inspired model according to claim 1, characterized in that, The original observation state described in step 1 includes: the distance and angle between the autonomous underwater vehicle and the target point, the linear velocity and angular velocity of the underwater vehicle, and obstacle ranging information returned by forward-looking sonar; The reward function is: , wherein in, Indicates the reward value. This represents the speed reward function. This represents the reward value for reaching the target point. This indicates a penalty for colliding with an obstacle. and These represent the minimum distances from the center of the autonomous underwater vehicle to the target point and the obstacle, respectively. and These represent the arrival threshold and collision threshold, respectively; d represents the scaling factor; and t represents time. , and It is a constant. and These represent the linear velocity and angular velocity of the autonomous underwater vehicle, respectively.

3. The dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold-adaptive brain-like model according to claim 1, characterized in that, The dynamic threshold adaptive module described in step 2, at time t+1, represents the j-th neuron in the n-th layer as follows: ,in, This indicates a dynamic threshold adaptive module. Indicates the adaptive energy threshold. Indicates the dynamic depolarization threshold. and Indicates hyperparameters; The adaptive energy threshold It is positively correlated with the membrane potential of spiking neurons and is used to adjust the pulse trigger rate based on the neuron membrane potential level. The dynamic depolarization threshold It shows a negative correlation with the membrane potential changes of spiking neurons over the past two moments, and is used to adjust the pulse trigger rate based on the trend of membrane potential changes. in, and Let represent the membrane potential and threshold of the j-th neuron in the n-th layer at time t, respectively. and These represent the average membrane potential and the average trigger threshold of all neurons in the same layer, respectively. and It's a hyperparameter. This represents the change in membrane potential of the j-th neuron in the n-th layer at time t+1 and time t. and It is a constant.

4. The dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive brain-like model according to claim 1, characterized in that, The pulse encoder in step 3 encodes the original observation state information into a pulse state sequence through normalization and Poisson sampling operations; the pulse decoder obtains the action command information of the underwater autonomous vehicle by averaging the network output results in the time dimension and then transforming and mapping them.

5. The dynamic obstacle avoidance method for underwater autonomous vehicles based on a threshold-adaptive brain-like model according to claim 1, characterized in that, The spiking dynamic threshold Actor network described in step 4 includes multiple fully connected layers, and each spiking neuron in the fully connected layer deploys the dynamic threshold adaptive module described above; the deep Critic network includes multiple fully connected layers composed of traditional artificial neurons.

6. The dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive brain-like model according to claim 5, characterized in that, The neurons in the pulse dynamic threshold Actor network are LIF neurons, and the dynamic threshold adaptive module is deployed in each pulse neuron to dynamically adjust membrane current, membrane voltage, and pulse firing process. in, and These represent the membrane current and membrane voltage of a spiking neuron, respectively. and These represent the synaptic weight matrix and the synaptic bias vector, respectively. and These represent the current decay factor and the voltage decay factor, respectively. Indicates an update to the door. Indicates the pulse delivery gate. This refers to the dynamic adaptive threshold.

7. The dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive brain-like model according to claim 1, characterized in that, Step 5 describes the training process of the threshold-adaptive brain-like model, which includes: The original observation state information is converted into a pulse state sequence by the pulse encoder and then input into the pulse dynamic threshold Actor network. The pulse decoder then converts and outputs action information. A reward value is obtained based on the interaction between the AUV and the underwater environment. The network training is stabilized and converged, and a pulse dynamic threshold Actor target network and a deep Critic target network are constructed. A TD loss function is constructed based on the deep Critic network, the deep Critic target network, and the reward value. The deep Critic network is updated using the TD loss function, and the pulse dynamic threshold Actor network is updated using the Q loss function. in, The loss function of a deep Critic network is represented. This represents the loss function of the impulse dynamic threshold Actor network. Indicates the reward value. This represents the Q value at the current moment. Represents the original state information. This indicates the output action information. and These represent the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the corresponding target networks. Indicates the discount factor. and represent the parameters of the impulse dynamic threshold Actor network and the deep Critic network, respectively. and These represent the parameters of the corresponding target network.

8. The dynamic obstacle avoidance method for an underwater autonomous vehicle based on a threshold-adaptive brain-like model according to claim 7, characterized in that, The impulsive dynamic threshold Actor network uses STBP for policy gradient backpropagation, while the deep Critic network is updated using a normal backpropagation algorithm. The parameters of the impulsive dynamic threshold Actor network and the deep Critic network are periodically used to softly update the parameters of the target network. in, It is a constant.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

Images