An Expandable Cooperative Obstacle Avoidance Decision-making Method for Unmanned Surface Vehicle Swarms

Abstract

The present invention discloses a scalable method for collaborative obstacle avoidance decision-making for a swarm of unmanned boats. This method establishes a collaborative penetration mission scenario for a swarm of unmanned boats, determines the joint state space, action space, and reward function of the attacking swarm of unmanned boats based on a Markov game model, and establishes a prediction network for the attacking swarm of unmanned boats to obtain the acceleration and angular velocity of the attacking swarm of unmanned boats. This method completes the collaborative obstacle avoidance decision-making for the swarm of unmanned boats. The prediction model of the present invention can predict the acceleration and angular velocity of the attacking swarm of unmanned boats based on multi-dimensional feature inputs. Combined with an attention mechanism, the prediction model is generalizable and extensible, solving the problem of the remaining attacking swarms being unable to continue their penetration mission when the swarm size changes due to a collision between unmanned boats.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned boat swarms, and in particular to an expandable unmanned boat swarm collaborative obstacle avoidance decision-making method. Background Art

[0002] With the advancement of science and technology and the increasing level of informatization, unmanned equipment is playing an increasingly important role in both civilian and military fields. Unmanned boats, with their small size, strong concealment, and high flexibility, offer unique advantages in information gathering, close-range reconnaissance, and raids on key targets. During mission execution, swarms of unmanned boats often encounter an unknown number of non-cooperative vessels, facing threats such as collisions and interceptions. This requires the unmanned boats to possess high penetration capabilities and form effective collaborative relationships with other swarm members. Currently, research on unmanned equipment swarm confrontations primarily focuses on unmanned vehicles, drones, and aviation, while research on unmanned boat swarm confrontations is relatively limited.

[0003] When performing a penetration mission, unmanned aerial vehicles (UAVs) treat the defending UAVs as intelligent, dynamic obstacles. Unlike conventional UAV swarm formation and navigation tasks, the greatest difficulty for the attacking UAVs in performing a penetration mission lies in how to reach the target area while avoiding interception by the defenders. This requires the attacking UAV swarm to possess a high degree of autonomous decision-making capabilities to quickly and accurately respond to various situations. Current applications of deep reinforcement learning to solve similar problems typically use fixed-dimensional feature inputs, and the decision-making model lacks generalizability and scalability. Therefore, when an UAV collision causes the swarm size to change, the remaining attacking UAVs cannot continue the penetration mission. Summary of the Invention

[0004] The present invention discloses an expandable unmanned boat cluster collaborative obstacle avoidance decision-making method to overcome the above technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A scalable unmanned watercraft swarm collaborative obstacle avoidance decision-making method includes the following steps:

[0007] S1: Establish a coordinated penetration mission scenario for the unmanned boat cluster, including the radius of the target area R1, the radius of the defense area of the defender's unmanned boat R2, the radius of the penetration mission area of the attacking unmanned boat R3, and the safety radius of the unmanned boats of both sides R safe , the distance between the mth attacking unmanned boat and the target center The position and perception information of the attacking UAV; the center of the defense area and the penetration mission area are both the center of the target area;

[0008] S2: Based on the radius R1 of the target area, the radius R2 of the defense area of the defender's unmanned boat, the radius R3 of the breakthrough mission area of the attacker's unmanned boat, and the safety radius R of the unmanned boats of both sides. safe , the distance between the mth attacking unmanned boat and the target center Establish the joint state space, action space, and reward function of the attacking UAV swarm based on the Markov game model;

[0009] S3: Based on the joint state space, action space, and reward function of the attacking UAV swarm, a prediction network for the attacking UAV is established and trained; the prediction network includes a policy network and an evaluation network based on an attention mechanism;

[0010] S4: Based on the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the coordinated penetration decision of the unmanned boat cluster.

[0011] Furthermore, the network structure of the policy network includes: an input layer, a fully connected layer and an output layer;

[0012] The input layer includes a first encoding layer, a query layer, a key layer, a value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat;

[0013] The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, Key layer and Value layer respectively;

[0014] The Query layer is used to obtain the Q matrix based on the pose encoding features;

[0015] The Key layer is used to obtain the K matrix according to the pose encoding features;

[0016] The Value layer is used to obtain the V matrix based on the pose encoding features;

[0017] The first multiplication module is used to obtain a correlation matrix between the Q matrix and the K matrix according to the Q matrix and the K matrix;

[0018] The softmax module is used to obtain a normalized correlation matrix according to the correlation matrix;

[0019] The second multiplication module is used to obtain an attention matrix according to the normalized correlation matrix and the V matrix;

[0020] The mean module is used to obtain the feature vector of the attacking unmanned boat's posture information after being converted by the attention mechanism according to the attention matrix;

[0021] The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat;

[0022] The fully connected layer is used to obtain the acceleration and angular velocity of the attacking unmanned boat based on the feature vector and perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, and output it through the output layer.

[0023] Furthermore, the joint state space of the attacking unmanned boat cluster is established as follows:

[0024]

[0025] in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; U represents the heading of the mth attacking unmanned boat; max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection range of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

[0026] Furthermore, the action space is established as follows:

[0027]

[0028] in, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

[0029] Furthermore, the reward function is established as follows:

[0030]

[0031] Among them, r srepresents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat;

[0032]

[0033] r g =r single +r group

[0034] Where: represents the distance between the mth attacking UAV and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; r single Indicates the breakthrough reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking UAV;

[0035]

[0036]

[0037] in: represents the distance between the mth attacking UAV and the target center; represents the distance between the mth attacking UAV and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking UAV; i represents the index number of the ranging information; c and m both represent the index numbers of the attacking UAV, and c≠m; represents the relative azimuth between the cth attacking unmanned boat and the mth attacking unmanned boat.

[0038] Beneficial Effects: The present invention provides a scalable unmanned boat swarm collaborative obstacle avoidance decision-making method. By establishing a collaborative penetration mission scenario based on an unmanned boat swarm, determining the joint state space, action space, and reward function of the attacking unmanned boat swarm based on the Markov game model, and establishing a prediction network for the attacking unmanned boats, the acceleration and angular velocity of the attacking unmanned boats are obtained, and the collaborative obstacle avoidance decision of the attacking unmanned boat swarm is completed. The prediction model of the present invention can predict the acceleration and angular velocity of the attacking unmanned boats based on multi-dimensional feature inputs. Combined with the attention mechanism, the prediction model has generalization and scalability, solving the problem that when the unmanned boats collide and the cluster size changes, the remaining attacking unmanned boats cannot continue to perform the penetration mission. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 This is a flow chart of the unmanned boat swarm collaborative obstacle avoidance decision-making method of the present invention;

[0041] Figure 2 Schematic diagram of a penetration mission scenario in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the strategy network and evaluation network structure in an embodiment of the present invention;

[0043] Figure 4 : is a curve illustrating the success rate of the attacking unmanned boat swarm mission in an embodiment of the present invention;

[0044] Figure 5 Schematic diagram of the unmanned boat trajectory during the simulation process in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] This embodiment introduces a scalable unmanned boat swarm collaborative obstacle avoidance decision-making method, such as Figure 1As shown, the following steps are included:

[0047] S1: Establish a coordinated penetration mission scenario for an unmanned boat swarm, including the radius of the target area R1, the coordinated penetration scenario for the unmanned boat swarm; the radius of the defense area of the defender's unmanned boat R2, the radius of the penetration mission area of the attacking unmanned boat R3; the safety radius of the unmanned boats of both sides R safe , the distance between the mth attacking unmanned boat and the target center The position and perception information of the attacking UAV; the centers of the defense area and the penetration mission area are both the center of the target area; that is, the defense area, the penetration mission area and the target area are concentric;

[0048] Specifically, the unmanned boat cluster coordinated penetration mission scenario is as follows Figure 2 As shown, in a limited open sea area, there are N A The attacking UAV and N D The performance of the two sides' unmanned boats is the same, and the number of the defending side's unmanned boats is greater than the number of the attacking side's unmanned boats. m=1,2,...,N A represents the number of the attacking UAV, k = 1, 2, ..., N D represents the number of the defending party's unmanned boat. The target center is located in the center of the sea area. The radius of the target area is R1, the radius of the defending area is R2, and the radius of the attacking party's breakthrough mission area is R3. safe Indicates the safety radius of the unmanned boats of both sides, represents the distance between the mth attacking UAV and the target center. The attacking UAV uses its onboard communication equipment to obtain position information of other attacking UAVs and uses its onboard lidar to sense the relative distances of surrounding UAVs. The attacker's mission objective is to cooperate with the defender and ultimately reach the target area. The defender's mission objective is to destroy the attacker within the defense range through a suicide collision, rendering both sides incapacitated. This example provides two mission-ending conditions: the mission ends when any attacking UAV safely reaches the target area, and the attacker wins; the mission ends when no attacking UAV is capable of operating within the penetration mission area, and the defender wins.

[0049] S2: Establish the joint state space, action space and reward function of the attacking UAV swarm based on the Markov game model of the attacking UAV swarm’s coordinated penetration decision-making;

[0050] Specifically, based on the existing technology, this embodiment models the attacking party's unmanned boat swarm coordinated penetration decision-making as a Markov game: <N A ,S,A,H,R,γ>. Among them, N Arepresents the number of attacking unmanned boats, S and A represent the joint state space and joint action space of the attacking unmanned boat cluster, respectively, H represents the state transfer function, R represents the joint reward function of the attacking unmanned boat cluster, and γ represents the reward discount factor.

[0051] Preferably, the joint state space of the attacking unmanned boat cluster is established as follows:

[0052] Specifically, the joint state space of the attacking unmanned boat cluster is composed of the state spaces of each attacking unmanned boat. Taking the mth attacking unmanned boat as an example, its state space is as follows:

[0053]

[0054] in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; U represents the heading of the mth attacking unmanned boat; max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection range of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

[0055] Specifically, in this embodiment, the perception information of the attacking unmanned boat is the distance to other unmanned boats measured by the laser radar divided by the detection distance of the radar, which is obtained by the laser radar.

[0056] Preferably, the action space is established as follows:

[0057] Specifically, the joint action space of the attacking unmanned boat cluster is composed of the action spaces of each attacking unmanned boat, and the action space of the mth attacking unmanned boat is as follows:

[0058]

[0059] in, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

[0060] Preferably, the reward function is established as follows:

[0061] Specifically, the joint reward function of the attacking unmanned boat cluster is composed of the reward functions of each attacking unmanned boat. The reward function designed in this embodiment includes two parts: sparse reward and guidance reward. When the mission is completed, the attacking unmanned boat will receive a one-time reward. During the execution of the mission, the attacking unmanned boat will receive a guidance reward at each time step, thereby guiding it to avoid threats and approach the target. In addition, the attacking unmanned boats need to cooperate with each other and penetrate from different directions to avoid conflicts in the penetration route. The reward function of the mth attacking unmanned boat is The design is as follows:

[0062]

[0063] Among them, r s represents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat;

[0064]

[0065] Where: represents the distance between the mth attacking UAV and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; Represents the ranging information of the laser radar carried by the mth attacking unmanned boat;

[0066] Specifically, when When , it means that the mth attacking unmanned boat arrives at the target area, the mission is completed, and a positive reward is obtained; Represents the minimum value of the laser radar ranging information carried by the attacking unmanned boat. When , it means that the mth attacking UAV collides with other UAVs, loses its ability to move, and obtains a negative reward.

[0067] r g =r single +r group

[0068] Where: r single Indicates the breakthrough reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking UAV;

[0069]

[0070] in: represents the distance between the mth attacking UAV and the target center; represents the distance between the mth attacking UAV and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking unmanned boat; i represents the index number of the ranging information;

[0071]

[0072] Where: c, m both represent the index number of the attacking unmanned boat, and c≠m; represents the relative azimuth between the cth attacking unmanned boat and the mth attacking unmanned boat.

[0073] S3: Establishing a prediction network for the attacking UAV based on the joint state space, action space, and reward function of the attacking UAV cluster, wherein the prediction network includes a policy network and an evaluation network based on an attention mechanism, and training the prediction network; wherein the policy network takes the position information and perception information of the attacking UAV as input and the acceleration and angular velocity of the attacking UAV as output;

[0074] Specifically, each attacking UAV has its own independent policy network and evaluation network. The policy network outputs the attacking UAV's acceleration and angular velocity based on its state space. The evaluation network outputs a state-action value based on the attacking UAV's state space and action space. The higher the state-action value, the better the policy network's output.

[0075] Considering the mission characteristics of the coordinated penetration of the UAV cluster, the size of the cluster changes dynamically. There are two reasons for the change. One is that when the number of the defending UAV is large, the number of the attacking UAV needs to be increased, resulting in an increase in the number of clusters. The other is that during the mission, the attacking UAV may be crashed by the defending UAV, resulting in a decrease in the cluster size. The change in the size of the attacking cluster results in the set of the attacking UAV cluster pose information in the state space of the attacking UAV s. usv The dynamic changes in the dimensions of the attacking UAV require the attacking UAV's policy network and evaluation network to be highly scalable. However, conventional neural networks require fixed input dimensions. To address this issue, this embodiment designs a scalable network suitable for inputs of uncertain scale features based on the self-attention mechanism, converting the input information into a fixed dimension.

[0076] Specifically, taking the mth attacking unmanned boat as an example, the network of the strategy network of this embodiment is as follows: Figure 3 As shown, it includes: input layer, fully connected layer and output layer.

[0077] The input layer includes a first encoding layer, a Query layer, a Key layer, a Value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat.

[0078] The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, Key layer and Value layer respectively;

[0079] The Query layer is used to obtain the Q matrix based on the posture coding features; the Key layer is used to obtain the K matrix based on the posture coding features; the Value layer is used to obtain the V matrix based on the posture coding features; wherein, obtaining the Q matrix, K matrix and V matrix respectively through the Query layer, Key layer and Value layer is a conventional technology in the field and will not be described in detail here.

[0080] The first multiplication module is used to multiply the Q matrix and the K matrix to obtain a correlation matrix between the Q matrix and the K matrix;

[0081] The softmax module is used to process the correlation matrix through a softmax function to obtain a normalized correlation matrix;

[0082] The second multiplication module is used to multiply the normalized correlation matrix and the V matrix to obtain an attention matrix;

[0083] The mean module is used to obtain the feature vector of the attacking unmanned boat's posture information after being converted by the attention mechanism according to the attention matrix, that is, the A vector used to characterize the features of the attacking unmanned boat's posture information after being converted by the attention mechanism;

[0084] The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat;

[0085] The fully connected layer is used to splice the A vector and the perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, obtain the acceleration and angular velocity of the attacking unmanned boat, and output it through the output layer.

[0086] Specifically, the position information of all attacking UAVs is input into the first encoding layer. The output of the first encoding layer is input into the query layer, key layer, and value layer, respectively, to generate the Q matrix, K matrix, and V matrix. The Q matrix is multiplied by the K matrix and processed through the softmax function. The result is multiplied by the V matrix and averaged to obtain the A vector. The attacking UAV's own perception information is input into the second encoding layer, and the output of the second encoding layer is concatenated with the A vector to form the output of the input layer.

[0087] The fully connected layer includes a first fully connected layer and a second fully connected layer. Specifically, the first fully connected layer inputs the output of the input layer and performs fully connected calculations. The output of the first fully connected layer is processed by an activation function and then input into the second fully connected layer for full connection calculations and activation processing as the output of the fully connected layer.

[0088] The output layer takes the output of the fully connected layer as input and outputs the acceleration and angular velocity of the attacking unmanned boat.

[0089] S4: Based on the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the coordinated penetration decision of the unmanned boat cluster.

[0090] Specifically, the attacking UAV's evaluation network has a similar structure to the policy network. The difference is that the evaluation network's input layer includes information about the attacking UAV's actions, and the output layer outputs the state-action value. The evaluation network's results guide the policy network's parameter updates, a common technique in the field and not detailed here.

[0091] In this embodiment, the attacking UAV cluster uses a distributed training approach, where each attacking UAV independently trains its own policy network parameters and evaluation network parameters based on SAC deep reinforcement learning theory. When the specified number of training cycles is reached, the policy network parameters and evaluation network parameters of each attacking UAV are saved.

[0092] Specifically, in this embodiment, a Cartesian coordinate system is established with the center of the target area as the origin. In the simulation environment, the position update formula of the unmanned boat during the training of the prediction network is:

[0093]

[0094] Among them, x, y, U, ψ represent the lateral coordinate, longitudinal coordinate, speed, and heading of the unmanned boat at the current moment respectively; x′, y′, U′, ψ′ represent the lateral coordinate, longitudinal coordinate, speed, and heading of the unmanned boat at the next moment respectively; v represents the acceleration of the unmanned boat; represents the angular velocity of the unmanned boat; Δt represents the sampling time interval.

[0095] This embodiment conducts physical prototype experiments on the decision-making method of cooperative penetration of unmanned boat swarms;

[0096] The trained attacking unmanned boat strategy network and the defending unmanned boat interception strategy are deployed to two host computers respectively, which are called the attack host computer and the defense host computer. Both the attack host computer and the defense host computer establish communication connections with the physical prototypes of each unmanned boat through serial communication. The physical prototype of the unmanned boat executes the control instructions issued by the host computer, including the expected heading and expected speed, and sends its own longitude and latitude, actual speed and actual heading to the host computer. The target area location is selected according to the area and shape of the sea area, and the longitude and latitude of the center of the target area are recorded. Based on this, the longitude and latitude of each unmanned boat are converted into horizontal and vertical coordinates in the Cartesian coordinate system. Due to the small scale of the physical prototype and limited resources, it is impossible to deploy a lidar. In the experiment, we simulate the ranging information of the lidar based on the positions of all unmanned boats. The experimental steps are as follows:

[0097] 1) All UAVs are divided into two groups: one group serves as the attacking UAVs and the other as the defending UAVs. The attacking UAVs establish a communication connection with the attacking host computer, while the defending UAVs establish a communication connection with the defending host computer. The attacking host computer and the defending host computer establish a communication connection to exchange the position information of their UAVs.

[0098] 2) The experimenters remotely deployed the defending UAVs to the periphery of the target area and the attacking UAVs to the periphery of the defense area, and then switched all UAVs to automatic mode.

[0099] 3) The experimenters simultaneously activated the attacking and defending decision-making programs. Each UAV executed control commands from the host computer every 0.5 seconds and returned its own position information until the mission was completed. It should be noted that the control commands issued by the attacking host computer were the desired speed and heading obtained by adding the acceleration and angular velocity output by the policy network to the actual speed and heading of the attacking UAV, respectively. During the experiment, if a UAV collided, it would become incapable of movement, and the experimenters would manually disconnect it from the decision-making program.

[0100] 4) After the experiment is completed, the position information of all unmanned boats during the entire mission is saved.

[0101] Specifically, the expected speed and expected heading of the attacking unmanned boat in this embodiment are obtained by designing an interception strategy for the defending unmanned boat cluster, wherein the defending unmanned boat adopts a two-layer interception strategy, the upper layer is the target allocation layer, and the lower layer is the interception guidance layer.

[0102] Among them, the target allocation layer is to assign specific interception targets to each defending unmanned boat. Specifically, based on linear programming theory, an allocation model is established, which includes three parts: design variables, objective functions and constraints.

[0103] The design variable Z is a (N A ×N D ), when the interception target of the k-th defender's unmanned boat is the m-th attacker's unmanned boat, Z mk =1, otherwise Z mk =0, where Z mk Represents the element in the mth row and kth column of the design variable.

[0104] The objective function F is as follows:

[0105]

[0106] Among them, d mk Z represents the distance between the kth defender's unmanned boat and the mth attacker's unmanned boat; mk Represents the element in the mth row and kth column of the design variable.

[0107] The constraints are:

[0108]

[0109]

[0110] Among them, N max Indicates the maximum number of defending UAVs that can intercept the same attacking UAV.

[0111] Specifically, when the sum of the distances between all matched offensive and defensive UAVs is minimized, the objective function reaches its maximum value. The constraint condition indicates that each defensive UAV can intercept at most one offensive UAV, and each offensive UAV can match at least one defensive UAV, with a maximum of N matched UAVs. max A defending unmanned boat.

[0112] The interception guidance layer is to calculate the expected speed of the defender's unmanned boat when performing the interception mission and heading Take the kth defender's unmanned boat as an example, its interception target is the mth attacker's unmanned boat. At this time, the expected speed and expected heading of the kth defender's unmanned boat when performing the interception mission are: The calculation formula is as follows:

[0113]

[0114] in, represents the expected speed of the kth defender's unmanned boat when performing the interception mission; represents the expected heading of the k-th defender's unmanned boat when performing the interception mission; represents the speed of the mth attacking unmanned boat; represents the heading of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the lateral coordinate of the kth defending unmanned boat; represents the longitudinal coordinate of the kth defending unmanned boat; c1 and c2 are interception parameters.

[0115] Figure 4 This is a curve showing the success rate of the attacking UAV swarm during a penetration mission during training. Algorithm 1 is the algorithm proposed in this example, and Algorithm 2 is a basic algorithm in the field. As training progresses, the attacking UAV swarm's penetration success rate gradually improves. Ultimately, in a simulation environment, the attacking UAV swarm trained using the algorithm proposed in this example achieves a success rate exceeding 90%. Figure 5 Figure 1 shows the trajectory of each unmanned boat during the experiment. The experiment involved a physical prototype of an unmanned boat with a single rudder and single propeller. There were four attacking unmanned boats and 12 defending unmanned boats. As can be seen in the figure, at 46 seconds, attacking unmanned boat No. 4 collided with a defending unmanned boat and was disabled, while the remaining attacking unmanned boats were able to continue their mission. At 58 seconds, attacking unmanned boat No. 2 entered the target area without colliding, marking the victory for the attacking team.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scalable unmanned boat swarm collaborative obstacle avoidance decision-making method, characterized by: The steps include: S1: Establish a coordinated penetration mission scenario for the unmanned boat cluster, including the radius of the target area R1, the radius of the defense area of the defender's unmanned boat R2, the radius of the penetration mission area of the attacking unmanned boat R3, and the safety radius of the unmanned boats of both sides R safe , the distance between the mth attacking unmanned boat and the target center The position and perception information of the attacking UAV; the center of the defense area and the penetration mission area are both the center of the target area; S2: Based on the radius R1 of the target area, the radius R2 of the defense area of the defender's unmanned boat, the radius R3 of the breakthrough mission area of the attacker's unmanned boat, and the safety radius R of the unmanned boats of both sides. safe , the distance between the mth attacking unmanned boat and the target center Establish the joint state space, action space, and reward function of the attacking UAV swarm based on the Markov game model; S3: Based on the joint state space, action space, and reward function of the attacking UAV swarm, a prediction network for the attacking UAV is established and trained; the prediction network includes a policy network and an evaluation network based on an attention mechanism; The reward function is established as follows: Among them, r s represents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat; r g ＝r single +r group Where: represents the distance between the mth attacking UAV and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; r single Indicates the breakthrough reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking UAV; in: represents the distance between the mth attacking UAV and the target center; represents the distance between the mth attacking UAV and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking UAV; i represents the index number of the ranging information; c and m both represent the index numbers of the attacking UAV, and c≠m; represents the relative azimuth angle between the cth attacking unmanned boat and the mth attacking unmanned boat; N A Indicates the total number of attacking unmanned boats; S4: Based on the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the collaborative obstacle avoidance decision of the unmanned boat cluster.

2. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The network structure of the policy network includes: an input layer, a fully connected layer and an output layer; The input layer includes a first encoding layer, a query layer, a key layer, a value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat; The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, Key layer and Value layer respectively; The Query layer is used to obtain the Q matrix based on the pose encoding features; The Key layer is used to obtain the K matrix according to the pose encoding features; The Value layer is used to obtain the V matrix based on the pose encoding features; The first multiplication module is used to obtain a correlation matrix between the Q matrix and the K matrix according to the Q matrix and the K matrix; The softmax module is used to obtain a normalized correlation matrix according to the correlation matrix; The second multiplication module is used to obtain an attention matrix according to the normalized correlation matrix and the V matrix; The mean module is used to obtain the feature vector of the attacking UAV’s posture information after being transformed by the attention mechanism based on the attention matrix; The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat; The fully connected layer is used to obtain the acceleration and angular velocity of the attacking unmanned boat based on the feature vector and perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, and output it through the output layer.

3. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The joint state space of the attacking unmanned boat swarm is established as follows: in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; U represents the heading of the mth attacking unmanned boat; max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection range of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

4. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The action space is established as follows: Among them, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

Images