USV Swarm Collision Avoidance Using LSTM Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing collision avoidance methods for swarms of unmanned surface vehicles are inefficient due to high calculation requirements and poor decision-making efficiency, making rapid collision avoidance difficult.
Innovation Solution
An intelligent collision avoidance method based on deep reinforcement learning, utilizing a vehicle coordinate system, motion modeling, environmental feature representation, reward and punishment functions, and LSTM neural networks to train USVs for effective collision avoidance in compliance with COLREGS rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing collision avoidance methods are used for unmanned surface vehicles, then collision avoidance can be achieved, but the calculation and analysis amount becomes huge, leading to low calculation efficiency
Solution Approach 1:
The patent transforms the continuous state space of collision avoidance into a discrete state space by defining specific state parameters (relative position, relative velocity, bearing angle) with quantized ranges. This parameter discretization enables the use of pre-computed lookup tables and reduces the computational complexity from continuous optimization to discrete state transitions, thereby improving calculation efficiency while maintaining collision avoidance reliability
Solution Approach 2:
The patent pre-computes and stores optimal avoidance maneuvers in lookup tables based on predefined state parameters and COLREGS rules. During actual operation, the system only needs to query the pre-computed results based on current state, avoiding real-time complex calculations. This preliminary action significantly reduces online computational burden while ensuring reliable collision avoidance
2Reliability
If existing collision avoidance methods are used for swarms of unmanned surface vehicles, then collision avoidance can be achieved, but the calculation and analysis amount becomes huge, making rapid decision difficult
Solution Approach 1:
The patent segments the swarm collision avoidance problem into individual vehicle decision-making units, where each USV independently determines its avoidance maneuver based on local state information and pre-computed policies. This segmentation eliminates the need for centralized coordination and complex inter-vehicle communication, enabling rapid independent decisions while maintaining overall swarm safety through standardized state-parameter frameworks
Solution Approach 2:
The patent pre-computes avoidance maneuvers for all possible state combinations and stores them in lookup tables. During runtime, each USV quickly queries the appropriate maneuver based on current state parameters, avoiding real-time complex calculations. This preliminary action reduces decision-making time from seconds to milliseconds while ensuring reliable collision avoidance according to COLREGS rules
3Adaptability or versatility
If deep reinforcement learning is used for collision avoidance training, then adaptability and decision-making efficiency are improved, but training complexity and computational resources increase
Solution Approach 1:
The patent defines a structured state space with specific parameters (relative position, velocity, bearing angle) and action space with discrete maneuvers (turn left, turn right, maintain course). This parameterization enables the reinforcement learning agent to generalize across different scenarios while keeping the state and action spaces manageable. The structured parameter definition balances adaptability with computational feasibility during training
Data Source
AI summary
Disclosed is an intelligent collision avoidance method for a swarm of unmanned surface vehicles based on deep reinforcement learning; firstly, a theoretical framework of autonomous learning collision avoidance of a swarm of unmanned surface vehicles based on deep reinforcement learning is proposed, and the LSTM neural network memory ability is integrated to realize the continuity of collision avoidance actions; then, according to the USV environment in the framework, the characterization method is obtained, and the USV collision avoidance reward and punishment function is proposed to evaluate the collision avoidance effect; finally, an intelligent collision avoidance training system for a swarm of unmanned surface vehicles is formed. The simulation and verification of this disclosure show that the USV trained in this disclosure can navigate safely in the collision avoidance environment with a swarm of unmanned surface vehicles and realize intelligent collision avoidance.