An intelligent collision avoidance method for unmanned surface vehicle based on trajectory prediction

Abstract

The application provides an unmanned ship intelligent collision avoidance method based on trajectory prediction, and belongs to the field of unmanned ship collision avoidance, and comprises the following steps: S1, acquiring motion state data of the ship and surrounding target ships and preprocessing; S2, constructing an interaction graph with the target ships as nodes, extracting graph convolution features of the nodes, and performing dynamic screening to obtain sparse attention features; performing time sequence feature extraction on the spliced features of the graph convolution features and the sparse attention features through a multi-layer convolution network, introducing a hierarchical dynamic weighting mechanism to weight and fuse the multi-layer convolution network outputs, decoding the obtained space-time fusion features to predict the future trajectories of the target ships; S3, jointly constructing a state space of the motion state of the ship and the future trajectories of the target ships, inputting the state space into a collision avoidance decision model, and outputting collision avoidance actions; S4, converting the collision avoidance actions into control instructions of the unmanned ship and executing the control instructions; and S5, repeating S1-S4 until the collision avoidance task is completed. The application improves the collision avoidance success rate and safety of the unmanned ship.

Description

Technical Field

[0001] This invention belongs to the field of unmanned surface vessel (USV) collision avoidance, specifically relating to an intelligent collision avoidance method for USVs based on trajectory prediction. Background Technology

[0002] With the continuous and in-depth development and exploration of marine resources, unmanned driving technology provides crucial technical support for marine development. As a new type of marine equipment, unmanned surface vessels (USVs) play a vital role in improving the efficiency of maritime operations and ensuring maritime safety through their autonomous navigation technology. The motion of target vessels at sea is often uncertain, posing risks to the safe navigation of USVs. Therefore, dynamic collision avoidance has become a crucial condition for ensuring their stable navigation. In recent years, deep reinforcement learning technology has demonstrated significant advantages in the field of USV collision avoidance. Its strong environmental adaptability and end-to-end decision-making characteristics provide innovative solutions to the challenges of collision avoidance in complex dynamic environments.

[0003] In existing research on unmanned surface vessel (USV) collision avoidance, the state space is constructed in three main ways: 1. State representation based on relative motion, which constructs the interaction relationship between the USV and the target vessel through the relative distance, relative speed, and relative heading. This method is simple to implement, but it can only reflect instantaneous geometric and motion differences and is difficult to capture the complex dynamic interactions between multiple target vessels; 2. State representation based on risk functions, which quantifies the potential threat of the target vessel into state input using collision risk indicators (such as DCPA and TCPA). This method can directly reflect collision risk, but it relies on thresholds and empirical formulas, making it difficult to take into account the complex environmental characteristics of multiple targets and multiple scenarios, and it lacks stability under rapidly changing dynamic sea conditions; 3. State representation based on trajectory prediction, which uses the historical or predicted trajectory of the target vessel as state input for USV collision avoidance decision-making. This method can use historical trajectories to extrapolate future behavior, enabling the decision network to avoid potential collision risks at an earlier stage, improving the lead time of collision avoidance actions and overall safety. While the above methods have improved the collision avoidance performance of unmanned surface vessels to some extent, the following problems still exist: relying solely on the representation of relative motion or risk functions makes it difficult to capture the complex interaction relationships and temporal dependencies between multiple target vessels; and if the prediction model lacks adaptive update capabilities, the prediction accuracy will drop rapidly when encountering sudden scenarios or abnormal motion patterns, resulting in insufficient reliability of collision avoidance decisions.

[0004] While trajectory prediction-based state representation methods can incorporate historical motion information of target ships and predict their future trajectories during the modeling process, they are better able to reflect the temporal dependencies and potential interaction trends between ships in complex and abrupt scenarios. However, traditional trajectory prediction-based state representation methods mostly use fixed neighborhood or simple weighting methods to model the interaction relationships between target ships. This fails to effectively screen key interaction objects and easily introduces a large amount of redundant information in high-density navigation environments, leading to state space dimensionality expansion and noise interference, thereby reducing the accuracy of trajectory prediction and adversely affecting the reliability of collision avoidance decisions. In addition, traditional reinforcement learning collision avoidance methods usually rely on prediction models trained offline, lacking the ability to quickly adapt and update during actual navigation. When there are differences between the actual navigation scenario and the distribution of training data, the model's insufficient generalization ability often leads to a decrease in trajectory prediction accuracy and significantly limits the performance of collision avoidance decisions.

[0005] To address the aforementioned issues, this invention proposes an improved PPO (Point of Collision Avoidance) intelligent collision avoidance method for unmanned surface vessels (USVs) based on a sparse attention mechanism and hierarchical dynamic weighted trajectory prediction. This method introduces a sparse attention mechanism to filter and aggregate the interaction features of multiple target vessels, and combines this with a hierarchical dynamic weighting strategy to fully utilize the temporal information of convolutional layers at different depths. This enables higher-precision modeling of the future motion trends of target vessels during trajectory prediction. The prediction results are used as input to the state space and integrated into the improved PPO decision framework. Simultaneously, a trajectory prediction accuracy index is introduced into the reward function, feeding the prediction error back to the reinforcement learning decision process. This allows the collision avoidance strategy to dynamically favor more predictable actions, thereby improving the reliability and real-time performance of USVs in complex dynamic environments. Summary of the Invention

[0006] The purpose of this invention is to propose an intelligent collision avoidance method for unmanned surface vessels (USVs) based on trajectory prediction. This method addresses the shortcomings of existing solutions that directly use trajectory prediction results as state inputs, lacking constraints on prediction accuracy and failing to achieve good collision avoidance performance for USVs in complex dynamic scenarios or sudden motion patterns. By constructing a dynamic state space representation that integrates trajectory prediction, multi-dimensional encoding of environmental information is achieved. Furthermore, by designing a trajectory prediction accuracy reward function, the prediction module and decision network can synchronously adapt to environmental changes. This effectively improves the decision reliability, environmental adaptability, and real-time response of the unmanned system in complex dynamic environments, thereby enhancing the collision avoidance success rate and safety of USVs in high-density and dynamically uncertain scenarios.

[0007] To achieve the above objectives, the technical solution of the present invention is: an intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction, comprising the following steps:

[0008] Step S1: Acquire and preprocess the motion status data of the ship and multiple surrounding target ships to align the data in the time series dimension; the motion status data of the ship includes position coordinates, speed and heading, and the motion status data of the target ships includes position coordinates, speed, heading, relative speed and relative bearing with the ship.

[0009] Step S2: Construct an interaction graph with target ships as nodes. Extract graph convolutional features of nodes through a graph convolutional network, and introduce a sparse attention mechanism to dynamically filter features highly relevant to the current prediction task to obtain sparse attention features. Extract temporal features from the spliced ​​features of graph convolutional features and sparse attention features in the temporal dimension through a multi-layer convolutional network, and introduce a hierarchical dynamic weighting mechanism to weight and fuse the outputs of the multi-layer convolutional network to obtain spatiotemporal fusion features. Decode the spatiotemporal fusion features to predict the future trajectories of each target ship.

[0010] Step S3: Construct a state space by combining the motion state of the current vessel with the predicted future trajectory of the target vessel, input the collision avoidance decision model based on the PPO algorithm, and output the collision avoidance action;

[0011] Step S4: Convert the collision avoidance maneuver into thrust and steering torque control commands for the unmanned surface vessel and execute them;

[0012] Step S5: Repeat steps S1 to S4 until the collision avoidance task is completed.

[0013] Preferably, the construction of the interaction graph with the target ship as the node is as follows:

[0014] Each target ship is defined as a graph node, and each node is assigned a feature vector representing the target ship's past performance. Position sequence within a step time:

[0015] ,

[0016] In the formula, For the first The feature vectors of the nodes corresponding to the target ships. It is the first The target ship at the moment Location coordinates, It is the first The target ship at the moment Location coordinates, It is a real number;

[0017] Define the interaction relationships between target ships as edge weights:

[0018]

[0019] In the formula, For the first The target ship and the first The edge weights between the nodes corresponding to the target ships. and Representing the first The target ship and the first The target ship at the moment Location, It is the first The target ship and the first The Euclidean distance between the target ships and It is the first The target ship and the first The speed of the target ship.

[0020] Preferably, the step of extracting graph convolutional features of nodes through a graph convolutional network and introducing a sparse attention mechanism to dynamically filter features highly relevant to the current prediction task to obtain sparse attention features is as follows:

[0021] Construct an adjacency matrix based on edge weights. After adding self-loops, normalization is performed to obtain the normalized adjacency matrix. ;in, This represents the number of target ships around this vessel. It is the identity matrix. It is a degree matrix;

[0022] With the normalized adjacency matrix As input, graph convolutional features are extracted through a graph convolutional network. :

[0023]

[0024] Among them, graph convolution features The first in The row corresponds to the first Spatial interaction characteristics between the target ship's corresponding node and its neighboring nodes. For activation functions; for Feature matrix of nodes corresponding to each target ship For the first The feature vectors of the nodes corresponding to the target ships; The weight matrix is ​​a learnable matrix;

[0025] Convolution features of the graph Linear mapping to query matrix Key matrix Sum matrix Sparse attention features are obtained through sparse attention mechanisms. :

[0026]

[0027] in, Let be the dimension of the key matrix; This represents the sparsification Softmax operation; This is a sparse matrix used to retain only the weights corresponding to the keys most relevant to each query, while resetting the weights of the rest to zero; superscript This indicates transpose.

[0028] Preferably, the introduction of a hierarchical dynamic weighting mechanism to weight and fuse the outputs of multi-layer convolutional networks yields spatiotemporal fusion features, as detailed below:

[0029] pass One-dimensional convolution pairs of stacked layers for graph convolution features With sparse attention features Temporal feature extraction is performed by concatenating features along the temporal dimension; let the first... Frame number The output of the convolutional layer is ,in For the feature dimension, frame It is a discrete index of time-series features in the time dimension;

[0030] The first The convolutional output features of each layer in a frame are concatenated layer by layer to obtain a matrix. :

[0031]

[0032] Calculate the first Weight scores for each layer corresponding to the frame :

[0033]

[0034] in, , ; Solve for the mean; These are trainable projection vectors used to project each layer. 3D feature mapping to scalar scores; Indicates a trainable bias term;

[0035] Scoring by weight of each layer The weighted sum of the outputs of each convolutional layer is used to obtain the first... Spatiotemporal fusion features of frames :

[0036]

[0037] in, ; Indicates the first The weights of the layers are determined by The The components are obtained after normalization. Spatiotemporal fusion features of all frames This constitutes a spatiotemporal fusion feature matrix.

[0038] Preferably, the decoding of the spatiotemporal fusion features to predict the future trajectories of each target ship is as follows:

[0039] For the The target ship uses the corresponding eigenvector in the spatiotemporal fusion feature matrix as the initial input to the decoding module. ,pass One-dimensional convolutions stacked layer by layer are used for decoding to obtain decoded features; the decoding module... One-dimensional convolution is calculated as follows:

[0040]

[0041] in, , This represents the number of historical observation frames. Indicates the first The decoding module corresponding to the target ship Convolutional layer output, , To predict the number of frames; Indicates the decoding module number One-dimensional convolution kernel of the layer, , For width, For input and output channel dimensions; Indicates the decoding module number Layer bias, ;

[0042] Output of the last convolutional layer Apply on Convolution, which converts each time step's... Mapping a dimensional vector to two-dimensional planar coordinates yields the 1st... Predicted future trajectory of the target ship :

[0043]

[0044] in, , These represent the convolution kernel and bias, respectively, used to... Channel mapping as aisle, Future trajectory prediction middle, Indicates the first The target ship in the future Frames Axis predicted coordinates .

[0045] Preferably, the process of constructing a state space by combining the current motion state of the ship with the predicted future trajectory of the target ship is as follows:

[0046]

[0047] in, for Time-state space; express The planar position coordinates of the ship in the global coordinate system at any given moment; for The pitching speed, rolling speed, and angular velocity of the ship in the ship's coordinate system at any given moment; for The ship's heading angle at any given moment; for The target ship's predicted future Frame 2D coordinate sequence, the first The target ship's predicted future Frame 2D coordinate sequence , Indicates the first The target ship in the future Frames Axis predicted coordinates; The minimum Euclidean distance between all predicted points and the ship's current position:

[0048]

[0049] in, To indicate the first The target ship in the future The two-dimensional coordinates of the frame express The coordinates of the ship's planar position in the global coordinate system at any given time. Index for future frames.

[0050] Preferably, the reward function used to train the collision avoidance decision model Including distance bonus Heading rewards Risk penalties Collision penalty and predicted rewards :

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057] in, , , , , These are the weight coefficients of each reward function, and all weight coefficients of each reward function are greater than 0; express The coordinates of the ship's planar position in the global coordinate system at any given time. This represents the coordinates of the target point in the global coordinate system. The formula for calculating the distance between two points on the plane representing the current position of the ship and the predicted position of the target ship; for At any given time, the ship's heading angle, The bearing of the target as the ship points; This represents the minimum predicted distance between this ship and any target ship in the future. The collision threshold, This is the danger threshold. This is a safety threshold; No. The ship in the future The actual coordinates observed in the frame.

[0058] Preferably, the collision avoidance decision model based on the PPO algorithm adopts an Actor-Critic architecture;

[0059] The Actor network is a policy network, and its input is the state space. The output is the Gaussian distribution parameters of the action. and The original action variables were obtained from the Gaussian distribution using a reparameterized sampling method. The actual collision avoidance action is obtained by performing a normalization operation. , , To control longitudinal thrust, To control the torque;

[0060] The action space is constructed using a centrally symmetric linear mapping, which converts the normalized motion quantities output by the Actor network into the actual thrust of the unmanned surface vessel. and torque :

[0061]

[0062] In the formula, These represent the minimum and maximum thrust of the propulsion unit. Minimum and maximum torque provided to the servo motor;

[0063] Critic networks are value networks, with the state space as their input. The output is a state value estimate. .

[0064] Preferably, the collision avoidance decision model training phase uses a shearing objective function. Optimize the Actor network:

[0065]

[0066] in, For time step-based Empirical expectations of the sampled data; For strategy ratio, , Indicates the current state New policy network parameters Decide to take action The probability, Indicates the state old policy network parameters Decide to take action The probability of; For generalized advantage estimation; This is a hyperparameter for the cropping range. For strategy pruning;

[0067] The Critic network updates its parameters by minimizing the mean square error between the predicted Q-value and the target Q-value.

[0068]

[0069]

[0070] in, It is the loss function of the Critic network. This refers to the number of samples in the batch. Indicates the first in the batch One sample index; For the target Q value, by the first Individual sample reward and the next state Corresponding state value constitute; Discount factor;

[0071] During training, an empirical replay pool is used to store sample data, and a soft update method is used to smoothly update the parameters of the target Critic network.

[0072]

[0073] in, These are the parameters of the target Critic network, i.e., the network parameters used to calculate the target Q value; These are the parameters of the current Critic network. This is the soft update coefficient. This is an assignment operation.

[0074] Preferably, the training environment used in the collision avoidance decision model training phase includes the following preset parameters: the spatial range of the simulation scene, the number of randomly generated target ships and the navigation state of each target ship, and the randomly set target point positions; and a three-degree-of-freedom MMG model is used to describe the dynamic behavior of the unmanned surface vessel in the plane and around the longitudinal and yaw axes of the hull:

[0075]

[0076] in, These are the ship's longitudinal velocity, lateral velocity, and bow roll rate, respectively. These are the pitch, roll, and bow velocities of the unmanned surface vessel in the ship's coordinate system, respectively. These are the longitudinal force, lateral force, and yaw moment of the hull, respectively. These are the relevant component forces and torques generated by the rudder; For thruster thrust; For the mass of the unmanned surface vessel; Let be the moment of inertia about the vertical axis;

[0077] Training termination conditions include: the target ship intruding into the collision zone of the ship, the ship reaching the target point, or the current number of steps reaching the maximum training step length.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] This invention introduces a sparse attention mechanism and a hierarchical dynamic weighting mechanism to adaptively filter and aggregate key features in complex interactive environments with multiple target vessels, avoiding interference from invalid information in state space modeling and effectively improving the accuracy and stability of trajectory prediction. Based on the PPO framework, the predicted future trajectory of the target vessel and the current vessel's navigation state are jointly used to construct the state space, enabling the collision avoidance strategy to perceive potential risks in advance and generate more reasonable control actions. At the same time, by designing a prediction accuracy reward function, multi-dimensional constraints and optimizations are achieved on collision avoidance behavior, ensuring that the unmanned surface vessel has stronger environmental adaptability and decision reliability in high-density and uncertain navigation environments. Attached Figure Description

[0080] Figure 1 This is a flowchart illustrating the collision avoidance decision-making process for an unmanned surface vessel (USV) according to an embodiment of the present invention.

[0081] Figure 2 This is a structural diagram of the intelligent collision avoidance algorithm for unmanned surface vessels based on trajectory prediction and PPO in an embodiment of the present invention;

[0082] Figure 3 This is a flowchart illustrating the fusion of sparse attention mechanism and hierarchical weighting mechanism in an embodiment of the present invention. Detailed Implementation

[0083] The following is in conjunction with the appendix Figure 1-3 The technical solution of the present invention will be described in detail below.

[0084] This invention proposes an intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction. This method uses a sparse attention mechanism to filter and fuse the interaction features of multiple target vessels, and integrates the temporal representations of different depths of convolutional networks through a hierarchical dynamic weighting mechanism, thereby completing the prediction of the future motion trajectory of the target vessels. At the same time, the trajectory prediction module inputs the prediction information into the state space for reinforcement learning algorithms to train the collision avoidance model.

[0085] Unmanned surface vessel collision avoidance decision-making process reference Figure 1 The steps are as follows:

[0086] Step S1: Sensor data collection and processing;

[0087] Step S2, Interaction graph construction and trajectory prediction;

[0088] Step S3: Collision avoidance decision model calculation;

[0089] Step S4: The unmanned surface vessel control system executes a collision avoidance maneuver.

[0090] Step S5: Repeat steps S1-S4 until the collision avoidance task is completed.

[0091] For step S1, in the unmanned surface vessel's trajectory prediction-based collision avoidance process, the main focus is on processing the position information of the target ships (TS). First, the owner ship (OS) needs to acquire the motion status of each surrounding target ship through a multi-sensor fusion system (using millimeter-wave radar, AIS, visual sensors, and other multi-source sensing devices), including position information, speed, heading, relative speed, and relative bearing. The owner ship then acquires its own position information, speed, and heading through the Global Navigation Satellite System (GNSS). Before input, the sensing data undergoes filtering, time synchronization, and interpolation processing to form a data structure of length [length missing]. The historical trajectory sequence ensures that the status data of each target ship are aligned in the time dimension.

[0092] For step S2, constructing the interaction graph and calculating the trajectory prediction are core to obtaining the target ship's navigation intention and are also the foundation for collision avoidance model calculation. The goal is to model multiple target ships and their dynamic behaviors as a graph structure, capturing the spatiotemporal dependencies between ships. Through Graph Convolutional Networks (GCNs) and sparse attention mechanisms, the mutual influence between target ships can be effectively captured, and future trajectories can be calculated through trajectory prediction. The specific implementation steps are as follows:

[0093] Step S2.1, for each TS, denoted as the th A TS is defined as a graph node, and each node is assigned a feature vector representing the past performance of that TS. The position sequence within a time step. Specifically, the feature vector of each node is derived from the ship's past... The trajectory coordinates at each moment are as follows:

[0094]

[0095] In the formula, For the first The feature vectors of the nodes corresponding to the target ships. It is the first The ship at the moment Location coordinates, and They represent the first TS ships in and Coordinates in direction It is the first The target ship at the moment Location coordinates, It is a real number.

[0096] Step S2.2: Calculate the interaction relationships between the target ships, that is, represent the interactions between ships using edge weights. Edge weights... It is measured by the spatial distance and relative speed between ships. The edge weight formula is as follows:

[0097]

[0098] In the formula, For the first The target ship and the first The edge weights between the nodes corresponding to the target ships. and Representing the first The target ship and the first The target ship at the moment Location, It is the first The target ship and the first The Euclidean distance between the target ships and It is the first The target ship and the first The speed of the target ship.

[0099] Step S2.3, constructing an adjacency matrix This is normalized to ensure that each node receives information from its neighbors equally during graph convolution computation. The normalization formula is as follows:

[0100]

[0101] In the formula, It is an adjacency matrix. This represents the number of target ships around this vessel. It is the identity matrix. It is a degree matrix, and its elements are... Normalized adjacency matrix Used for graph convolution operations to better propagate information.

[0102] Step S2.4 utilizes a Graph Convolutional Network (GCN) to combine the features of each node with the features of its neighboring nodes for information propagation. This helps capture the spatial interaction relationships between ships. The graph convolution operation is implemented using the following formula:

[0103]

[0104] Among them, graph convolution features The first in The row corresponds to the first Spatial interaction characteristics between the target ship's corresponding node and its neighboring nodes. This is the normalized adjacency matrix; for Feature matrix of nodes corresponding to each target ship; It is a learnable weight matrix, with each column corresponding to an output feature dimension. During training, it is updated through backpropagation and learns how to extract the combination of features that are most helpful in predicting future motion patterns from the original trajectory and the influence of neighbors. The activation function introduces nonlinearity into each node, enabling the network to fit complex spatiotemporal relationships. It is the first The feature vectors of the nodes corresponding to each target ship. This operation combines the features of each node in the graph with the information of its neighboring nodes to obtain a new feature representation of each TS after integrating the influence of all surrounding ships. .

[0105] Step S2.5: Considering that some areas or ships in actual sea conditions may not participate in the collision avoidance decision of the unmanned surface vessel at the current moment, a sparse attention mechanism is introduced based on the node state features extracted by GCN to dynamically focus on key interaction information. In this way, sparse attention can adaptively highlight key target vessel features and reduce redundant information in the interaction information, thereby achieving dynamic filtering of features highly relevant to the current prediction task and effectively improving the discriminative ability of trajectory representation. Specifically, the node features output by GCN are transformed through convolutional layers and mapped to a query matrix (Q), a key matrix (K), and a value matrix (V), respectively. Then, a sparsity constraint is introduced when calculating the attention weights, ensuring that each target vessel only interacts with its first few most relevant neighbors, and normalized attention weights are generated through Softmax. Finally, the weighted aggregate value (V) yields the dynamically weighted interaction feature output. This enhances the discriminative power of trajectory representation. The specific formula is as follows:

[0106]

[0107] in, , , The query matrix, key matrix, and value matrix are all obtained by linear mapping of the TS features extracted by GCN; It is the dimension of the key vector; This is a sparse matrix used to retain only the keys that are most relevant to each query, while the remaining weights are reset to zero. This is represented by calculating and weighting the similarity between nodes. This approach helps the network focus its attention on the target story (TS) relevant to the current prediction task, weakening irrelevant or noisy information, thereby generating more discriminative dynamic interaction feature representations and improving the effectiveness of trajectory prediction.

[0108] Step S2.6, by... and After concatenating the temporal data, temporal patterns at different scales are extracted through multi-layer one-dimensional convolutions. To achieve a balance among features at different levels, a hierarchical dynamic weighting mechanism is introduced to fuse the multi-layer outputs. Specifically, the feature map of each convolutional output is non-linearly mapped into a scalar score, and the average of the scores from each layer is used to obtain the hierarchical weights. The weight calculation formula is:

[0109]

[0110] in For the first The weight scores of each layer corresponding to the frame, frame It is a discrete index of time-series features in the time dimension; Indicates the first Frame in Output of each layer in a multilayer convolutional network The matrix obtained by concatenating layers; Solve for the mean; These are trainable projection vectors used to project each layer. 3D feature mapping to scalar scores; This represents a trainable bias term that affects the score across all layers.

[0111] Step S2.7, by all The features of the layer are summed by weight according to the layer weights to obtain the first layer. Final spatiotemporal fusion features of the frame This allows us to focus on the most information-rich levels. The specific formula is:

[0112]

[0113] In the formula Indicates the weighted fusion of the first... Frame output features are used for trajectory decoding; Stack layers for the CNN; Indicates the first The weights of the layers are determined by The Each component Extract and normalize to obtain; Indicates the first Convolutional networks in the first layer Frame generation 3D features; spatiotemporal fusion features from all frames This constitutes a spatiotemporal fusion feature matrix.

[0114] Step S2.8, for the first The target ship uses the corresponding eigenvectors of the spatiotemporal fusion feature matrix as the initial input to the decoding module. This is to complete the prediction of future trajectories. Specifically, the decoding module consists of... The system consists of one-dimensional convolutional layers. Each layer aggregates the local temporal information from the previous layer in a sliding window manner along the temporal dimension, and maps the channel number dimension layer by layer, while extracting deeper temporal features to obtain the final temporal features. . No. The formula for calculating the one-dimensional convolution of a layer is:

[0115]

[0116] in, , This represents the number of historical observation frames. Indicates the first The decoding module corresponding to the target ship Convolutional layer output, , To predict the number of frames; Indicates the first The width of the one-dimensional convolution kernel of the layer The input and output channel dimensions are ; Indicates the first Layer bias.

[0117] Step S2.9, output the last convolutional layer. Apply a Convolution, which converts each time step's... A dimensional vector can be directly mapped to two-dimensional planar coordinates to obtain the 1st dimension vector. Predicted future trajectory of the target ship :

[0118] ,

[0119] in, This represents the kernel and bias of the last convolutional layer, used to... Channel mapping as aisle. Indicates the first The ship in the future Frames Predict coordinates .

[0120] For step S3, the collision avoidance model adopts an Actor-Critic structure based on the PPO algorithm. Both the Actor and Critic are multi-layer fully connected neural networks, containing 4 hidden layers with 256 neurons per layer, and using the ReLU activation function. At each decision time t, a state vector is constructed by fusing the generated target ship's predicted trajectory with the current ship's state. Input the values ​​into the Actor and Critic, and calculate and output the collision avoidance actions according to the following process:

[0121] The Actor network processes the input. Perform forward propagation to obtain the parameters of the action distribution, i.e., the mean vector. with standard deviation vector ;

[0122] The original action variables are obtained by reparameterized sampling. The final action is obtained by normalizing the data using tanh mapping and scaling transformation. ;

[0123] state space Inputting the Critic network yields state value estimates. ;

[0124] Actions The decoding is into propulsion force and control torque.

[0125] Constructing a training environment is crucial for unmanned surface vessels (USVs) to train collision avoidance models through reinforcement learning. A well-designed training environment can improve the robustness and generalization ability of USV collision avoidance algorithms. Simultaneously, the generated multi-TS trajectories will be... The state space, fused with the OS's own state, constitutes the state space required for reinforcement learning training, defined as... .

[0126] Step S3.1: Set the size of the simulation scene, the number of randomly generated TSs and their navigation status, and the location of random target points.

[0127] Step S3.1.1: To accurately simulate the kinematic characteristics of the unmanned surface vessel (USV) in the training environment, the dynamic model of the USV used in training is a 7-meter model scaled down from the KVLCC2 hull type. The classic three-degree-of-freedom MMG motion model is used to describe its dynamic behavior in the plane and around the longitudinal and transverse axes of the hull. The specific model is as follows:

[0128]

[0129] In the formula, These are the ship's longitudinal velocity, lateral velocity, and bow roll rate, respectively. These are the pitch, roll, and bow velocities of the unmanned surface vessel in the ship's coordinate system, respectively. These are the longitudinal forces, lateral forces, and yaw moment of the hull. The relevant components of force and torque generated by the rudder; For thruster thrust; For the quality of USV; Let be the moment of inertia about the vertical axis.

[0130] Step S3.1.2, design the training termination conditions, which consist of three points: first, when the collision domain of OS is invaded by TS; second, when OS reaches the target point; and finally, the current step number. Reach the maximum stride length during training.

[0131] Step S3.2 involves fusing the multi-target trajectory prediction information generated by the previous prediction module with the OS's own state to form the high-dimensional state space required for reinforcement learning. Specifically, the state space... It can be represented as:

[0132]

[0133] in, express The planar position of OS in the global coordinate system at any given moment; for The pitch, roll, and angular velocities of OS in the ship's coordinate system at any given moment; for The heading angle of the OS at any given moment; for The target ship's predicted future Frame two-dimensional coordinate sequence, For the first The target ship's predicted future Frame two-dimensional coordinate sequence; Indicates the first The target ship in the future Frames Axis predicted coordinates; The minimum Euclidean distance between all predicted points and the current position of OS. To indicate the first The target ship in the future The two-dimensional coordinates of the frame express The coordinates of the ship's planar position in the global coordinate system at any given time. Index for future frames.

[0134] For step S4, the high-dimensional state space constructed based on step S3... The collision avoidance strategy employs the PPO algorithm within the Actor-Critic architecture of deep reinforcement learning for collision avoidance decision calculation.

[0135] Step S4.1 involves constructing the action space using a centrally symmetric linear mapping, directly converting the normalized action quantities output by the Actor network into the actual thrust and torque commands of the unmanned surface vessel. Specifically, the policy network outputs a normalized action vector at each time step. .in Controlling longitudinal thrust, Control torque. The thrust that maps to the actual control quantity. and torque The specific formula is as follows:

[0136]

[0137] In the formula, These represent the minimum and maximum thrust of the propulsion unit. The minimum and maximum torques provided to the servo motor.

[0138] The unmanned surface vessel's control system consists of a thruster and a rudder mechanism. Its purpose is to convert the collision avoidance parameters output in step S3 into actual control commands to implement collision avoidance. The specific process is as follows:

[0139] Based on the expected propulsion output of the decision network Converted into propeller speed command The calculation formula is as follows:

[0140]

[0141] in, The density of seawater, The diameter of the propeller. This is the propeller thrust coefficient. To ensure safety, it is necessary to... Apply saturation constraints:

[0142]

[0143] in, These are the minimum and maximum safe speeds of the propeller, respectively.

[0144] Based on the expected steering torque output by the decision network Calculate the corresponding rudder angle command :

[0145]

[0146] in, This is the proportionality coefficient between the rudder angle and the lateral moment. The rudder angle must also satisfy physical constraints:

[0147]

[0148] in, This is the maximum permissible deflection angle of the servo motor.

[0149] Step S4.2, in order to more comprehensively guide the unmanned surface vessel to learn a good collision avoidance strategy in multiple dimensions such as navigation efficiency, course accuracy, collision risk perception and prediction accuracy, the reward function is... Subdivided into distance rewards Heading rewards Risk penalties Collision penalty With predicted rewards The final reward is then calculated by weighted summation. The reward function formula is as follows:

[0150]

[0151] in, , , , , These are the weighting coefficients for each reward function, all of which are greater than 0. The specific meanings of each sub-item are as follows:

[0152] 1. Distance Rewards This reward is primarily used to encourage unmanned surface vessels to move towards the target point, and its calculation formula is as follows:

[0153]

[0154] in, The current coordinates of the OS; The coordinates of the target point, The formula for calculating the distance between two points on the plane representing the current position of the ship and the predicted position of the target ship.

[0155] 2. Course Rewards This reward is primarily intended to encourage unmanned surface vessels to face the target point, thereby reducing unnecessary lateral drift. The calculation formula is as follows:

[0156]

[0157] in The heading angle of OS; The cosine angle is the azimuth of the target pointed to by OS; the closer the cosine value is to 1, the higher the reward.

[0158] 3. Risk Penalty This penalty term is mainly based on continuously penalizing the minimum safe distance predicted for the future trajectory of TS, and its calculation formula is as follows:

[0159]

[0160] in For the first The obstacle ship in the future The predicted coordinates of the frame; the smaller the distance, the heavier the penalty.

[0161] 4. Collision Penalty This penalty primarily follows the principle that the closer the OS and TS are, the heavier the penalty; the farther apart they are, the lighter the penalty. The calculation formula is as follows:

[0162]

[0163] in, Let be the minimum predicted distance between OS and any TS in the future; This is the collision threshold, i.e., when OS collides with TS; This is the danger threshold, i.e., the security radius within which the OS has intruded into the TS; This is a safety threshold, meaning the OS and TS maintain a safe distance.

[0164] 5. Predict Rewards This reward is primarily used to directly feed back the prediction error of the TS's future trajectory to the policy, thereby guiding the OS to consider not only the current collision avoidance effect but also the impact of the action on the accuracy of the TS trajectory prediction when selecting actions. The calculation formula is as follows:

[0165]

[0166] in, These correspond to the observed true coordinates. By incorporating trajectory prediction accuracy into the reward function, the model is guided to optimize trajectory prediction performance while learning collision avoidance strategies.

[0167] Step S4.3 mainly focuses on the design, training, and updating process of the Actor network and the Critic network.

[0168] Step S4.3.1: The Actor network generates actions by learning a policy function, where the input is the state. The network output is the Gaussian distribution parameters of the actions. and This network employs a multilayer perceptron architecture, containing 256 hidden layers, each with 256 neurons, and uses the ReLU activation function. To improve the smoothness and robustness of the policy output, a reparameterization sampling method is used to obtain the actual actions from this Gaussian distribution. This method allows the model to balance exploration and exploitation during optimization, avoiding the generation of overly sharp or unstable actions. To maximize expected reward and smooth exploration, a shearing objective function is employed. Optimize:

[0169]

[0170] in, For time step-based Empirical expectations of the sampled data; For strategy ratio, Indicates the current state New policy network parameters Decide to take action The probability, Indicates the state old policy network parameters Decide to take action The probability of; Generalized advantage estimation (GAE); This is a hyperparameter for the cropping range. For strategy tailoring.

[0171] Step S4.3.2: The Critic network evaluates the value of the action output by the Actor network. It also uses a multilayer perceptron structure with four hidden layers, each containing 256 neurons, and employs the ReLU activation function. The input includes state features. Output state value estimation This refers to the expected long-term reward of choosing this action under the current state. Simultaneously, it's necessary to minimize the mean squared error between the predicted Q-value and the target Q-value. The parameters used to update and optimize the Critic network are... , can be represented as:

[0172]

[0173]

[0174] in, This refers to the number of samples in the batch. Indicates the first in the batch One sample index; For the target Q value, by the first Individual sample reward and the next state Corresponding state value constitute; This is the discount factor.

[0175] Step S4.3.3: During training, the experience replay pool is used to store sample data obtained from the environment, including the current state. Actions performed ,award Next state and termination mark Batch data is randomly sampled from the experience replay pool to update the parameters of the Actor and Critic networks.

[0176] The Critic network is updated by drawing batch data from the experience replay pool, processing it with sparse attention and hierarchical weighting mechanisms, and then using the Adam optimizer to update the Critic network parameters. The Actor network is updated using the Adam optimizer to ensure a balance between high expected reward and policy entropy. To enhance training stability, a soft update method is used to smoothly update the parameters of the target Critic network.

[0177]

[0178] in, These are the parameters of the target Critic network, i.e., the network parameters used to calculate the target Q value; These are the parameters of the current Critic network. This is the soft update coefficient, and a smaller value is taken, usually 0.005; This is an assignment operation.

[0179] Step S4.3.4: To maintain policy exploration capability, the entropy temperature coefficient needs to be adaptively adjusted during training. This objective function... for:

[0180]

[0181] in, is the entropy regularization weight coefficient.

[0182] For step S5, repeat steps S1 to S4, sampling new data from the experience replay pool in small batches according to the sampling frequency to update the network, and adapting to scene changes in real time; until the task objective is achieved or the system terminates.

[0183] In actual navigation, to ensure that the unmanned surface vessel (USV) can continuously avoid collision risks in dynamic environments, a cyclic control mechanism is set up, that is, after completing one action, the entire process of steps S1 to S4 is repeated. Specifically, the vessel at any given time... Environmental observation data is obtained, data is collected and predicted trajectory is calculated in steps S1–S2, collision avoidance decision calculation is performed in step S3, and the result is converted into specific execution instructions in step S4. Acting on the propulsion and steering systems; subsequently at time Then, new observation data are collected again, and the above process is repeated until the target point is reached.

[0184] Figure 1 The collision avoidance decision-making process for unmanned surface vessels corresponds to steps S1-S5.

[0185] Figure 2 The flowchart shows the algorithm for collision avoidance decision-making of unmanned surface vessels, in which trajectory prediction, integrated with the PPO algorithm, is the core design module of this invention.

[0186] First, the historical information and motion state of the target ship are acquired and feature extraction is performed. Then, the interaction graph relationship is constructed (steps S2.2-S2.4). Finally, the predicted trajectory is generated through sparse attention and hierarchical weighting mechanism (steps S2.5-S2.9).

[0187] Next, the predicted trajectory obtained in step S2 and the ship's state information are input together into the Actor network and Critic network to construct the state space (step S3.2).

[0188] During the training phase, the model performs reinforcement learning by guiding the policy network parameters through the reward function and optimizing the entropy regularization, thus maintaining the policy's exploration capability (steps S4.3.1-S4.3.4).

[0189] During the deployment phase, the Actor network outputs collision avoidance actions in real time after being guided by the objective function (step S4.1).

[0190] Figure 3 This module integrates the sparse attention mechanism and the hierarchical weighting mechanism. First, the TS state information observed by the OS is input into a graph convolutional network to extract interaction features between TSs and obtain an encoded spatiotemporal representation (step S2.4). Then, these features are mapped to Q, K, and V in the sparse attention mechanism. Features of Q and K are transformed through convolutional layers, followed by ReLU activation and further convolution to enhance feature representation. Next, Softmax is used to weight the importance of different targets, ultimately outputting the fused attention features (step S2.5). Then, the fused features are encoded using a convolutional neural network (CNN) to encode historical trajectory information, forming a trajectory representation through layer-by-layer residual accumulation, extracting the motion patterns of TSs in the time series (step S2.6). Finally, the attention-encoded interaction features and the trajectory representation extracted by the CNN are fused in a unified space to obtain a spatiotemporal feature representation that can simultaneously characterize the interaction relationships between multiple ships and future motion trends (steps S2.7-S2.9), providing input for subsequent trajectory prediction and collision avoidance decisions.

[0191] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for intelligent collision avoidance of unmanned surface vessels based on trajectory prediction, characterized in that, Includes the following steps: Step S1: Acquire and preprocess the motion status data of the ship and multiple surrounding target ships to align the data in the time series dimension; the motion status data of the ship includes position coordinates, speed and heading, and the motion status data of the target ships includes position coordinates, speed, heading, relative speed and relative bearing with the ship. Step S2: Construct an interaction graph with target ships as nodes. Extract graph convolutional features of nodes through a graph convolutional network, and introduce a sparse attention mechanism to dynamically filter features highly relevant to the current prediction task to obtain sparse attention features. Extract temporal features from the spliced ​​features of graph convolutional features and sparse attention features in the temporal dimension through a multi-layer convolutional network, and introduce a hierarchical dynamic weighting mechanism to weight and fuse the outputs of the multi-layer convolutional network to obtain spatiotemporal fusion features. Decode the spatiotemporal fusion features to predict the future trajectories of each target ship. Step S3: Construct a state space by combining the motion state of the current vessel with the predicted future trajectory of the target vessel, input the collision avoidance decision model based on the PPO algorithm, and output the collision avoidance action; Step S4: Convert the collision avoidance maneuver into thrust and steering torque control commands for the unmanned surface vessel and execute them; Step S5: Repeat steps S1 to S4 until the collision avoidance task is completed.

2. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 1, characterized in that, The construction of the interaction graph with the target ship as the node is as follows: Each target ship is defined as a graph node, and each node is assigned a feature vector representing the target ship's past performance. Position sequence within a step time: ， In the formula, For the first The feature vectors of the nodes corresponding to the target ships. It is the first The target ship at the moment Location coordinates, It is the first The target ship at the moment Location coordinates, It is a real number; Define the interaction relationships between target ships as edge weights: In the formula, For the first The target ship and the first The edge weights between the nodes corresponding to the target ships. and Representing the first The target ship and the first The target ship at the moment Location, It is the first The target ship and the first The Euclidean distance between the target ships and It is the first The target ship and the first The speed of the target ship.

3. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 2, characterized in that, The graph convolution features of nodes are extracted through a graph convolutional network, and a sparse attention mechanism is introduced to dynamically filter features that are highly relevant to the current prediction task, resulting in sparse attention features, as detailed below: Construct an adjacency matrix based on edge weights. After adding self-loops, normalization is performed to obtain the normalized adjacency matrix. ;in, This represents the number of target ships around this vessel. It is the identity matrix. It is a degree matrix; With the normalized adjacency matrix As input, graph convolutional features are extracted through a graph convolutional network. : Among them, graph convolution features The first in The row corresponds to the first Spatial interaction characteristics between the target ship's corresponding node and its neighboring nodes. For activation functions; for Feature matrices of nodes corresponding to each target ship For the first The feature vectors of the nodes corresponding to the target ships; The weight matrix is ​​a learnable matrix; Convolution features of the graph Linear mapping to query matrix Key matrix Sum matrix Sparse attention features are obtained through sparse attention mechanisms. : in, Let be the dimension of the key matrix; This represents the sparsification Softmax operation; This is a sparse matrix used to retain only the weights corresponding to the keys most relevant to each query, while resetting the weights of the rest to zero; superscript This indicates transpose.

4. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 3, characterized in that, The introduced hierarchical dynamic weighting mechanism performs weighted fusion of the outputs of multi-layer convolutional networks to obtain spatiotemporal fusion features, as detailed below: pass One-dimensional convolution pairs of stacked layers for graph convolution features With sparse attention features Temporal feature extraction is performed by concatenating features along the temporal dimension; Record No. Frame number The output of the convolutional layer is ,in For the feature dimension, frame It is a discrete index of time-series features in the time dimension; The first The convolutional output features of each layer in a frame are concatenated layer by layer to obtain a matrix. : Calculate the first Weight scores for each layer corresponding to the frame : in, , ; Solve for the mean; These are trainable projection vectors used to project each layer. 3D feature mapping to scalar scores; Indicates a trainable bias term; Scoring by weight of each layer The weighted sum of the outputs of each convolutional layer is used to obtain the first... Spatiotemporal fusion features of frames : in, ; Indicates the first The weights of the layers are determined by The The components are obtained after normalization. Spatiotemporal fusion features of all frames This constitutes a spatiotemporal fusion feature matrix.

5. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 4, characterized in that, The decoding of spatiotemporal fusion features to predict the future trajectories of each target ship is as follows: For the The target ship uses the corresponding eigenvector in the spatiotemporal fusion feature matrix as the initial input to the decoding module. ,pass One-dimensional convolutions stacked layer by layer are used for decoding to obtain decoded features; Decoding module One-dimensional convolution is calculated as follows: in, , This represents the number of historical observation frames. Indicates the first The decoding module corresponding to the target ship Convolutional layer output, , To predict the number of frames; Indicates the decoding module number One-dimensional convolution kernel of the layer, , For width, For input and output channel dimensions; Indicates the decoding module number Layer bias, ; Output of the last convolutional layer Apply on Convolution, which converts each time step's... Mapping a dimensional vector to two-dimensional planar coordinates yields the 1st... Predicted future trajectory of the target ship : in, , These represent the convolution kernel and bias, respectively, used to... Channel mapping as aisle, Future trajectory prediction middle, Indicates the first The target ship in the future Frames Axis predicted coordinates .

6. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 1, characterized in that, The process of constructing a state space by combining the current motion state of the ship with the predicted future trajectory of the target ship is as follows: in, for Time-state space; express The planar position coordinates of the ship in the global coordinate system at any given moment; for The pitching speed, rolling speed, and angular velocity of the ship in the ship's coordinate system at any given moment; for The ship's heading angle at any given moment; for The target ship's predicted future Frame 2D coordinate sequence, the first The target ship's predicted future Frame 2D coordinate sequence , Indicates the first The target ship in the future Frames Axis predicted coordinates; The minimum Euclidean distance between all predicted points and the ship's current position: in, To indicate the first The target ship in the future The two-dimensional coordinates of the frame express The coordinates of the ship's planar position in the global coordinate system at any given time. Index for future frames.

7. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 6, characterized in that, Reward function used to train the collision avoidance decision model Including distance bonus Heading rewards Risk penalties Collision penalty and predicted rewards : in, , , , , These are the weight coefficients of each reward function, and all weight coefficients of each reward function are greater than 0; express The coordinates of the ship's planar position in the global coordinate system at any given time. This represents the coordinates of the target point in the global coordinate system. The formula for calculating the distance between two points on the plane representing the current position of the ship and the predicted position of the target ship; for At any given time, the ship's heading angle, The bearing of the target as the ship points; This represents the minimum predicted distance between this ship and any target ship in the future. The collision threshold, This is the danger threshold. This is a safety threshold; No. The ship in the future The actual coordinates observed in the frame.

8. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 6, characterized in that, The collision avoidance decision model based on the PPO algorithm adopts an Actor-Critic architecture; The Actor network is a policy network, and its input is the state space. The output is the Gaussian distribution parameters of the action. and The original action variables were obtained from the Gaussian distribution using a reparameterized sampling method. The actual collision avoidance action is obtained by performing a normalization operation. , , To control longitudinal thrust, To control the torque; The action space is constructed using a centrally symmetric linear mapping, which converts the normalized motion quantities output by the Actor network into the actual thrust of the unmanned surface vessel. and torque : In the formula, These represent the minimum and maximum thrust of the propulsion unit. Minimum and maximum torque provided to the servo motor; Critic networks are value networks, with the state space as their input. The output is a state value estimate. .

9. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 8, characterized in that, The collision avoidance decision model uses a shearing objective function during the training phase. Optimize the Actor network: in, For time step-based Empirical expectations of the sampled data; For strategy ratio, , Indicates the current state New policy network parameters Decide to take action The probability, Indicates the state old policy network parameters Decide to take action The probability of; For generalized advantage estimation; This is a hyperparameter for the cropping range. For strategy pruning; The Critic network updates its parameters by minimizing the mean square error between the predicted Q-value and the target Q-value. in, It is the loss function of the Critic network. This refers to the number of samples in the batch. Indicates the first in the batch One sample index; For the target Q value, by the first Individual sample reward and the next state Corresponding state value constitute; Discount factor; During training, an empirical replay pool is used to store sample data, and a soft update method is used to smoothly update the parameters of the target Critic network. in, These are the parameters of the target Critic network, i.e., the network parameters used to calculate the target Q value; These are the parameters of the current Critic network. This is the soft update coefficient. This is an assignment operation.

10. The intelligent collision avoidance method for unmanned surface vessels based on trajectory prediction according to claim 1, characterized in that, The training environment used in the collision avoidance decision model training phase includes the following preset parameters: the spatial range of the simulation scene, the number of randomly generated target ships and the navigation state of each target ship, and the randomly set target point positions; and a three-degree-of-freedom MMG model is used to describe the dynamic behavior of the unmanned surface vessel in the plane and around the longitudinal and yaw axes of the hull: in, These are the ship's longitudinal velocity, lateral velocity, and bow roll rate, respectively. These are the pitch, roll, and bow velocities of the unmanned surface vessel in the ship's coordinate system, respectively. These are the longitudinal force, lateral force, and yaw moment of the hull, respectively. These are the relevant component forces and torques generated by the rudder; For thruster thrust; For the mass of the unmanned surface vessel; Let be the moment of inertia about the vertical axis; Training termination conditions include: the target ship intruding into the collision zone of the ship, the ship reaching the target point, or the current number of steps reaching the maximum training step length.

Images