Expandable unmanned ship cluster collaborative obstacle avoidance decision-making method

Abstract

The invention discloses an expandable unmanned ship cluster cooperative obstacle avoidance decision-making method, and the method comprises the steps: determining a joint state space, an action space and a reward function of an attacker unmanned ship cluster based on a Markov game model through building an unmanned ship cluster cooperative penetration task scene, and building a prediction network of an attacker unmanned ship, therefore, the accelerated speed and the angular speed of the unmanned ship of the attacking side are obtained, and the cooperative obstacle avoidance decision of the unmanned ship cluster of the attacking side is completed. The prediction model can predict the acceleration and the angular velocity of the unmanned ship of the attacking side based on multi-dimensional feature input, the prediction model has generalization and expansibility in combination with an attention mechanism, and the problem that the cluster scale changes due to collision of the unmanned ship is solved. And the remaining unmanned ships of the attacking party cannot continue to execute the defense penetration task.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned boat clusters, and in particular to an expandable unmanned boat cluster collaborative obstacle avoidance decision-making method. Background Art

[0002] With the development of science and technology, the level of informatization is getting higher and higher, and unmanned equipment is playing an increasingly important role in the civil and military fields. Unmanned boats have the characteristics of small size, strong concealment, and high flexibility. They have unique advantages in information collection, close reconnaissance, and raids on important targets. In the process of mission execution, unmanned boat clusters usually encounter an unknown number of non-cooperative ships and encounter threats such as collision and interception. Unmanned boats need to have a high penetration capability and form an effective collaborative relationship with other cluster members. At present, the research on unmanned equipment cluster confrontation is mostly concentrated in the fields of unmanned vehicles, drones, and aviation, and the research on unmanned boat cluster confrontation is relatively small.

[0003] When the unmanned boat performs the penetration mission, it regards the defending unmanned boat as an intelligent dynamic obstacle. Unlike the conventional unmanned boat cluster formation and navigation mission, the biggest difficulty for the offensive unmanned boat to perform the penetration mission is how to avoid the interception of the defender while heading to the target area. This requires the offensive unmanned boat cluster to have a high degree of autonomous decision-making ability to respond to various situations quickly and accurately. At present, when deep reinforcement learning is applied to solve similar problems, fixed-dimensional feature inputs are usually used, and the decision model does not have generalization and scalability. Therefore, when the unmanned boat collides and causes the cluster size to change, the remaining offensive unmanned boats cannot continue to perform the penetration mission. Summary of the invention

[0004] The present invention discloses an expandable unmanned boat cluster collaborative obstacle avoidance decision-making method to overcome the above technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A scalable unmanned boat swarm collaborative obstacle avoidance decision-making method includes the following steps:

[0007] S1: Establish a coordinated penetration mission scenario for unmanned boat clusters, including the radius of the target area R1, the radius of the defense area of ​​the defending unmanned boat R2, the radius of the penetration mission area of ​​the attacking unmanned boat R3, and the safety radius of the unmanned boats of both sides R safe , the distance between the mth attacking unmanned boat and the target center The position information and perception information of the attacking unmanned boat; the center of the defense area and the penetration mission area are both the center of the target area;

[0008] S2: Based on the radius R1 of the target area, the radius R2 of the defense area of ​​the defender's unmanned boat, the radius R3 of the penetration mission area of ​​the attacker's unmanned boat, and the safety radius R safe , the distance between the mth attacking unmanned boat and the target center Establish the joint state space, action space and reward function of the attacking unmanned boat swarm based on the Markov game model;

[0009] S3: Based on the joint state space, action space and reward function of the attacking unmanned boat cluster, a prediction network of the attacking unmanned boat is established, and the prediction network is trained; the prediction network includes a strategy network and an evaluation network based on an attention mechanism;

[0010] S4: According to the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the coordinated penetration decision of the unmanned boat cluster.

[0011] Furthermore, the network structure of the strategy network includes: an input layer, a fully connected layer and an output layer;

[0012] The input layer includes a first encoding layer, a Query layer, a Key layer, a Value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat;

[0013] The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, the Key layer and the Value layer respectively;

[0014] The Query layer is used to obtain the Q matrix according to the pose encoding features;

[0015] The Key layer is used to obtain the K matrix according to the pose encoding features;

[0016] The Value layer is used to obtain the V matrix according to the pose encoding features;

[0017] The first multiplication module is used to obtain a correlation matrix between the Q matrix and the K matrix according to the Q matrix and the K matrix;

[0018] The softmax module is used to obtain a normalized correlation matrix according to the correlation matrix;

[0019] The second multiplication module is used to obtain an attention matrix according to the normalized relevance matrix and the V matrix;

[0020] The mean module is used to obtain the feature vector of the position information of the attacking unmanned boat after being transformed by the attention mechanism according to the attention matrix;

[0021] The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat;

[0022] The fully connected layer is used to obtain the acceleration and angular velocity of the attacking unmanned boat according to the feature vector and perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, and output it through the output layer.

[0023] Furthermore, the joint state space of the attacking unmanned boat cluster is established as follows:

[0024]

[0025] in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; represents the heading of the mth attacking unmanned boat; U max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection distance of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

[0026] Furthermore, the action space is established as follows:

[0027]

[0028] in, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

[0029] Furthermore, the reward function is established as follows:

[0030]

[0031] Among them, r srepresents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat;

[0032]

[0033] r g =r single +r group

[0034] Where: represents the distance between the mth attacking unmanned boat and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; r single Indicates the penetration reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking unmanned boat;

[0035]

[0036]

[0037] in: represents the distance between the mth attacking unmanned boat and the target center; represents the distance between the mth attacking unmanned boat and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking unmanned boat; i represents the index number of the ranging information; c, m both represent the index number of the attacking unmanned boat, and c≠m; Represents the relative azimuth between the cth attacking unmanned boat and the mth attacking unmanned boat.

[0038] Beneficial effects: The present invention provides an expandable unmanned boat cluster collaborative obstacle avoidance decision-making method, which establishes a collaborative penetration mission scenario based on an unmanned boat cluster, determines the joint state space, action space and reward function of the attacking unmanned boat cluster based on the Markov game model, and establishes a prediction network for the attacking unmanned boat to obtain the acceleration and angular velocity of the attacking unmanned boat, and completes the collaborative obstacle avoidance decision of the attacking unmanned boat cluster. The prediction model of the present invention can predict the acceleration and angular velocity of the attacking unmanned boat based on multi-dimensional feature inputs, and combined with the attention mechanism, the prediction model has generalization and extensibility, which solves the problem that when the unmanned boats collide and cause the cluster size to change, the remaining attacking unmanned boats cannot continue to perform the penetration mission. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 It is a flow chart of the unmanned boat swarm collaborative obstacle avoidance decision method of the present invention;

[0041] Figure 2 Schematic diagram of a penetration mission scenario in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the strategy network and evaluation network structure in an embodiment of the present invention;

[0043] Figure 4 It is a curve showing the success rate of the attacking unmanned boat swarm mission in an embodiment of the present invention;

[0044] Figure 5 Schematic diagram of the unmanned boat trajectory during the simulation process in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] This embodiment introduces a scalable unmanned boat swarm collaborative obstacle avoidance decision-making method, such as Figure 1As shown, the following steps are included:

[0047] S1: Establish the unmanned boat cluster coordinated penetration mission scenario, including the radius of the target area R1, the unmanned boat cluster coordinated penetration scenario; the radius of the defense area of ​​the defending unmanned boat R2, the radius of the penetration mission area of ​​the attacking unmanned boat R3; the safety radius R of the unmanned boats of both sides safe , the distance between the mth attacking unmanned boat and the target center The position information and perception information of the attacking unmanned boat; the centers of the defense area and the penetration mission area are both the centers of the target area; that is, the defense area, the penetration mission area and the target area are concentric;

[0048] Specifically, the unmanned boat cluster coordinated penetration mission scenario is as follows Figure 2 As shown, in a limited open sea area, there are N A The attacking drone and N D The unmanned boats of the two sides have the same performance, and the number of unmanned boats of the defending side is greater than the number of unmanned boats of the attacking side. m=1,2,...,N A represents the number of the attacking unmanned boat, k = 1, 2, ..., N D represents the number of the defending unmanned boat. The target center is located in the center of the sea area. The radius of the target area is R1, the radius of the defending area is R2, and the radius of the attacking area is R3. safe Indicates the safety radius of the unmanned boats of both sides, Represents the distance between the mth attacking unmanned boat and the target center. Among them, the attacking unmanned boat obtains the position information of other attacking unmanned boats through its own communication equipment, and senses the relative distance information of the surrounding unmanned boats through its own laser radar. The mission objective of the attacking party is to cooperate to avoid the defending party and finally reach the target area. The mission objective of the defending party is to destroy the attacking party within the defense range through a suicide collision, and both parties in the collision lose the ability to move. This example gives two conditions for the end of the mission, that is, if any attacking unmanned boat safely reaches the target area, the mission ends and the attacking party wins; if there is no attacking unmanned boat with the ability to move in the penetration mission area, the mission ends and the defending party wins.

[0049] S2: Establish the joint state space, action space and reward function of the attacking unmanned boat swarm based on the Markov game model of the coordinated penetration decision of the attacking unmanned boat swarm;

[0050] Specifically, based on the prior art, this embodiment models the coordinated penetration decision of the attacking unmanned boat cluster as a Markov game: <N A ,S,A,H,R,γ>. Among them, N Arepresents the number of attacking unmanned boats, S and A represent the joint state space and joint action space of the attacking unmanned boat cluster respectively, H represents the state transfer function, R represents the joint reward function of the attacking unmanned boat cluster, and γ represents the reward discount factor.

[0051] Preferably, the joint state space of the attacking unmanned boat cluster is established as follows:

[0052] Specifically, the joint state space of the attacking unmanned boat cluster is composed of the state spaces of each attacking unmanned boat. Taking the mth attacking unmanned boat as an example, its state space as follows:

[0053]

[0054] in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; represents the heading of the mth attacking unmanned boat; U max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection distance of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

[0055] Specifically, in this embodiment, the perception information of the attacking unmanned boat is the distance to other unmanned boats measured by the laser radar divided by the detection distance of the radar, which is obtained by the laser radar.

[0056] Preferably, the action space is established as follows:

[0057] Specifically, the joint action space of the attacking unmanned boat cluster is composed of the action spaces of each attacking unmanned boat. The action space of the mth attacking unmanned boat is as follows:

[0058]

[0059] in, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

[0060] Preferably, the reward function is established as follows:

[0061] Specifically, the joint reward function of the attacking unmanned boat cluster is composed of the reward functions of each attacking unmanned boat. The reward function designed in this embodiment includes two parts: sparse reward and guidance reward. When the mission is completed, the attacking unmanned boat receives a one-time reward. During the mission, the attacking unmanned boat will receive a guidance reward at each time step to guide it to avoid threats and approach the target. In addition, the attacking unmanned boats need to cooperate with each other and penetrate from different directions to avoid conflicts in the penetration routes. The reward function of the mth attacking unmanned boat is The design is as follows:

[0062]

[0063] Among them, r s represents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat;

[0064]

[0065] Where: represents the distance between the mth attacking unmanned boat and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; Represents the ranging information of the laser radar carried by the mth attacking unmanned boat;

[0066] Specifically, when When , it means that the mth attacking unmanned boat arrives at the target area, the mission is completed, and a positive reward is obtained; Represents the minimum value of the laser radar ranging information carried by the attacking unmanned boat. When , it means that the mth attacking unmanned boat collides with other unmanned boats, loses its ability to move, and obtains a negative reward.

[0067] r g =r single +r group

[0068] Where: r single Indicates the penetration reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking unmanned boat;

[0069]

[0070] in: represents the distance between the mth attacking unmanned boat and the target center; represents the distance between the mth attacking unmanned boat and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking unmanned boat; i represents the index number of the ranging information;

[0071]

[0072] Where: c, m both represent the index number of the attacking unmanned boat, and c≠m; Represents the relative azimuth between the cth attacking unmanned boat and the mth attacking unmanned boat.

[0073] S3: Establishing a prediction network of the attacking unmanned boat based on the joint state space, action space and reward function of the attacking unmanned boat cluster, wherein the prediction network includes a strategy network and an evaluation network based on an attention mechanism and the prediction network is trained; wherein the strategy network takes the position information and perception information of the attacking unmanned boat as input and takes the acceleration and angular velocity of the attacking unmanned boat as output;

[0074] Specifically, each attacking unmanned boat has an independent strategy network and evaluation network. The strategy network outputs the acceleration and angular velocity of the attacking unmanned boat according to the state space of the attacking unmanned boat. The evaluation network outputs the state action value according to the state space and action space of the attacking unmanned boat. The higher the state action value, the better the output of the strategy network.

[0075] Considering the mission characteristics of the coordinated penetration of unmanned boat clusters, the size of the cluster changes dynamically. There are two reasons for the change. One is that when the number of unmanned boats on the defending side is large, the number of unmanned boats on the attacking side needs to be increased, resulting in an increase in the number of clusters. The other is that during the mission, the unmanned boats on the attacking side may be crashed by the unmanned boats on the defending side, resulting in a decrease in the size of the cluster. The change in the size of the attacking cluster results in the set of the attacking unmanned boat cluster pose information in the state space of the attacking unmanned boats s usv The dimension of the attacking unmanned boat changes dynamically, which requires the strategy network and evaluation network of the attacking unmanned boat to have good scalability. However, conventional neural networks require the input dimension to be fixed. To address this problem, this embodiment designs a scalable network suitable for uncertain scale feature input based on the self-attention mechanism to convert the input information into a fixed dimension.

[0076] Specifically, taking the mth attacking unmanned boat as an example, the network of the strategy network of this embodiment is as follows: Figure 3 As shown, it includes: input layer, fully connected layer and output layer.

[0077] The input layer includes a first encoding layer, a Query layer, a Key layer, a Value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat.

[0078] The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, the Key layer and the Value layer respectively;

[0079] The Query layer is used to obtain the Q matrix according to the posture coding features; the Key layer is used to obtain the K matrix according to the posture coding features; the Value layer is used to obtain the V matrix according to the posture coding features; wherein, obtaining the Q matrix, K matrix and V matrix respectively through the Query layer, Key layer and Value layer is a conventional technology in the field and will not be described in detail here.

[0080] The first multiplication module is used for multiplying the Q matrix and the K matrix to obtain a correlation matrix between the Q matrix and the K matrix;

[0081] The softmax module is used to process the correlation matrix through a softmax function to obtain a normalized correlation matrix;

[0082] The second multiplication module is used to multiply the normalized relevance matrix and the V matrix to obtain an attention matrix;

[0083] The mean module is used to obtain the feature vector of the position information of the attacking unmanned boat after being transformed by the attention mechanism according to the attention matrix, that is, the A vector used to characterize the features of the position information of the attacking unmanned boat after being transformed by the attention mechanism;

[0084] The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat;

[0085] The fully connected layer is used to concatenate the A vector and the perceptual coding features according to the feature vector and the perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, obtain the acceleration and angular velocity of the attacking unmanned boat, and output it through the output layer.

[0086] Specifically, the position information of all attacking unmanned boats is input into the first coding layer, and the output of the first coding layer is input into the query layer, key layer, and value layer to obtain the Q matrix, K matrix, and V matrix. The Q matrix is ​​multiplied by the K matrix and then processed by the softmax function. The processed result is multiplied by the V matrix and then averaged to obtain the A vector. The perception information of the attacking unmanned boat itself is input into the second coding layer, and the output of the second coding layer is concatenated with the A vector as the output of the input layer.

[0087] The fully connected layer includes a first fully connected layer and a second fully connected layer. Specifically, the first fully connected layer inputs the output of the input layer, performs fully connected calculations, and the output of the first fully connected layer is processed by an activation function and then input to the second fully connected layer for fully connected calculations and activation processing as the output of the fully connected layer.

[0088] The output layer takes the output of the fully connected layer as input, and outputs the acceleration and angular velocity of the attacking unmanned boat.

[0089] S4: According to the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the coordinated penetration decision of the unmanned boat cluster.

[0090] Specifically, the evaluation network of the attacking unmanned boat is similar to the strategy network structure, except that the action information of the attacking unmanned boat is added to the input layer of the evaluation network, and the output of the output layer is the state action value. Based on the evaluation results of the evaluation network, the parameters of the strategy network are updated, which is a conventional technology in the field and will not be described here.

[0091] In this embodiment, the attacking unmanned boat cluster adopts a distributed training method, that is, each attacking unmanned boat independently trains its own strategy network parameters and evaluation network parameters based on the SAC deep reinforcement learning theory. When the specified number of training times is reached, the strategy network parameters and evaluation network parameters of each attacking unmanned boat are saved.

[0092] Specifically, in this embodiment, a Cartesian coordinate system is established with the center of the target area as the origin. In the simulation environment, the position update formula of the unmanned boat during the training of the prediction network is:

[0093]

[0094] Among them, x, y, U, ψ respectively represent the lateral coordinate, longitudinal coordinate, speed and heading of the unmanned boat at the current moment; x′, y′, U′, ψ′ respectively represent the lateral coordinate, longitudinal coordinate, speed and heading of the unmanned boat at the next moment; v represents the acceleration of the unmanned boat; represents the angular velocity of the unmanned boat; Δt represents the sampling time interval.

[0095] This embodiment conducts physical prototype experiments on the decision-making method of cooperative penetration of unmanned boat clusters;

[0096] The trained attacking unmanned boat strategy network and the defending unmanned boat interception strategy are deployed to two host computers, called the attack host computer and the defense host computer. Both the attack host computer and the defense host computer establish communication connections with the physical prototypes of each unmanned boat through serial communication. The unmanned boat physical prototype executes the control instructions issued by the host computer, including the expected heading and expected speed, and sends its own longitude and latitude, actual speed and actual heading to the host computer. The target area location is selected according to the area and shape of the sea area, and the longitude and latitude of the center of the target area are recorded. Based on this, the longitude and latitude of each unmanned boat are converted into horizontal and vertical coordinates in the Cartesian coordinate system. Due to the small scale of the physical prototype and limited resources, it is impossible to deploy a lidar. In the experiment, we simulate the ranging information of the lidar based on the positions of all unmanned boats. The experimental steps are as follows:

[0097] 1) All unmanned boats are divided into two groups, one group is the attacking unmanned boat, and the other group is the defending unmanned boat. The attacking unmanned boat establishes a communication connection with the attacking host computer, and the defending unmanned boat establishes a communication connection with the defending host computer. The attacking host computer establishes a communication connection with the defending host computer to exchange the position information of their own unmanned boats.

[0098] 2) The experimenters remotely deployed the defending unmanned boats to the periphery of the target area and the attacking unmanned boats to the periphery of the defense area, and then switched all the unmanned boats to automatic mode.

[0099] 3) The experimenter starts the decision-making procedures for both the attacker and defender at the same time. Each unmanned boat executes the control instructions issued by the host computer every 0.5 seconds and returns its own position information until the mission is completed. It should be noted that the control instructions issued by the attacking host computer are the expected speed and expected heading obtained by adding the acceleration and angular velocity output by the strategy network to the actual speed and actual heading of the attacker's unmanned boat. During the experiment, the unmanned boat will lose its ability to move after a collision, and the experimenter will manually disconnect it from the decision-making procedure.

[0100] 4) After the experiment, save the position information of all unmanned boats during the entire mission.

[0101] Specifically, the expected speed and expected heading of the attacking unmanned boat in this embodiment are obtained by designing an interception strategy for the defending unmanned boat cluster, wherein the defending unmanned boat adopts a double-layer interception strategy, with the upper layer being the target allocation layer and the lower layer being the interception guidance layer.

[0102] Among them, the target allocation layer is to assign specific interception targets to each unmanned boat of the defending party. Specifically, based on the linear programming theory, an allocation model is established, which includes three parts: design variables, objective functions and constraints.

[0103] The design variable Z is a (N A ×N D ), when the interception target of the k-th defender’s unmanned boat is the m-th attacker’s unmanned boat, Z mk =1, otherwise Z mk =0, where Z mk Represents the element in the mth row and kth column of the design variable.

[0104] The objective function F is as follows:

[0105]

[0106] Among them, d mk represents the distance between the kth defending unmanned boat and the mth attacking unmanned boat; Z mk Represents the element in the mth row and kth column of the design variable.

[0107] The constraints are:

[0108]

[0109]

[0110] Among them, N max Indicates the maximum number of defending unmanned boats that can intercept the same attacking unmanned boat.

[0111] Specifically, when the sum of the distances between all matched offensive and defensive unmanned boats is the smallest, the objective function reaches its maximum value. The constraint condition indicates that each defensive unmanned boat can intercept at most one offensive unmanned boat, and each offensive unmanned boat can match at least one defensive unmanned boat, with a maximum of N matches. max A defending unmanned boat.

[0112] The interception guidance layer is to calculate the expected speed of the defender's unmanned boat when performing the interception mission. and heading Taking the kth defender’s unmanned boat as an example, its interception target is the mth attacker’s unmanned boat. At this time, the expected speed and expected heading of the kth defender’s unmanned boat when performing the interception mission are: The calculation formula is as follows:

[0113]

[0114] in, represents the expected speed of the kth defender’s unmanned boat when performing the interception mission; represents the expected heading of the kth defender’s unmanned boat when performing the interception mission; represents the speed of the mth attacking unmanned boat; represents the heading of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the lateral coordinate of the kth defending unmanned boat; represents the longitudinal coordinate of the kth defending unmanned boat; c1 and c2 are interception parameters.

[0115] Figure 4 The curve of the success rate of the attacking unmanned boat cluster when performing the penetration mission during the training process is shown in Figure 1, where Algorithm 1 is the algorithm proposed in this embodiment, and Algorithm 2 is the basic algorithm in the field. As the training progresses, the success rate of the attacking unmanned boat cluster gradually increases. Finally, in the simulation environment, the success rate of the attacking unmanned boat cluster trained using the algorithm proposed in this embodiment can reach more than 90%. Figure 5 The motion trajectory of each unmanned boat during the experiment, in which the physical prototype of the unmanned boat with a single rudder and a single propeller drive mode was used, with 4 unmanned boats on the attacking side and 12 unmanned boats on the defending side. It can be seen from the figure that at 46 seconds, the No. 4 unmanned boat of the attacking side collided with the unmanned boat of the defending side and lost its ability to move, while the remaining unmanned boats of the attacking side were still able to continue to perform their tasks. At 58 seconds, the No. 2 unmanned boat of the attacking side entered the target area without collision, and the attacking side won.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scalable unmanned boat swarm collaborative obstacle avoidance decision-making method, characterized in that: The steps include: S1: Establish a coordinated penetration mission scenario for unmanned boat clusters, including the radius of the target area R1, the radius of the defense area of ​​the defending unmanned boat R2, the radius of the penetration mission area of ​​the attacking unmanned boat R3, and the safety radius of the unmanned boats of both sides R safe , the distance between the mth attacking unmanned boat and the target center The position information and perception information of the attacking unmanned boat; the center of the defense area and the penetration mission area are both the center of the target area; S2: Based on the radius R1 of the target area, the radius R2 of the defense area of ​​the defender's unmanned boat, the radius R3 of the penetration mission area of ​​the attacker's unmanned boat, and the safety radius R safe , the distance between the mth attacking unmanned boat and the target center Establish the joint state space, action space and reward function of the attacking unmanned boat swarm based on the Markov game model; S3: Based on the joint state space, action space and reward function of the attacking unmanned boat cluster, a prediction network of the attacking unmanned boat is established, and the prediction network is trained; the prediction network includes a strategy network and an evaluation network based on an attention mechanism; S4: According to the position information and perception information of the attacking unmanned boat and the trained prediction network, the acceleration and angular velocity of the attacking unmanned boat are obtained to complete the collaborative obstacle avoidance decision of the unmanned boat cluster.

2. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The network structure of the strategy network includes: an input layer, a fully connected layer and an output layer; The input layer includes a first encoding layer, a Query layer, a Key layer, a Value layer, a first multiplication module, a softmax module, and a second multiplication module, which are used to input the position information and perception information of the attacking unmanned boat; The first coding layer is used to obtain the posture coding features according to the input posture information of the attacking unmanned boat, and input the posture coding features into the Query layer, the Key layer and the Value layer respectively; The Query layer is used to obtain the Q matrix according to the pose encoding features; The Key layer is used to obtain the K matrix according to the pose encoding features; The Value layer is used to obtain the V matrix according to the pose encoding features; The first multiplication module is used to obtain a correlation matrix between the Q matrix and the K matrix according to the Q matrix and the K matrix; The softmax module is used to obtain a normalized correlation matrix according to the correlation matrix; The second multiplication module is used to obtain an attention matrix according to the normalized relevance matrix and the V matrix; The mean module is used to obtain the feature vector of the position information of the attacking unmanned boat after being transformed by the attention mechanism according to the attention matrix; The second coding layer is used to obtain perception coding features based on the perception information of the attacking unmanned boat; The fully connected layer is used to obtain the acceleration and angular velocity of the attacking unmanned boat according to the feature vector and perceptual coding features after the posture information of the attacking unmanned boat is converted by the attention mechanism, and output it through the output layer.

3. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The joint state space of the attacking unmanned boat cluster is established as follows: in, represents the state space of the mth attacking unmanned boat; s usv Represents the set of position information of the attacking unmanned boat cluster; represents the perception information of the mth attacking unmanned boat; Represents the position information of the mth attacking unmanned boat; represents the lateral coordinate of the mth attacking unmanned boat; represents the longitudinal coordinate of the mth attacking unmanned boat; represents the speed of the mth attacking unmanned boat; represents the heading of the mth attacking unmanned boat; U max represents the maximum speed of the mth attacking unmanned boat; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; l max Represents the maximum detection distance of the laser radar; R3 represents the penetration mission area radius of the attacking unmanned boat.

4. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The action space is established as follows: in, represents the action space of the mth attacking unmanned boat; represents the acceleration of the mth attacking unmanned boat; represents the angular velocity of the mth attacking unmanned boat.

5. The scalable unmanned boat swarm collaborative obstacle avoidance decision-making method according to claim 1, characterized in that: The reward function is established as follows: Among them, r s represents sparse rewards, r g Indicates guidance reward; represents the reward function of the mth attacking unmanned boat; r g ＝r single +r group Where: represents the distance between the mth attacking unmanned boat and the target center; R1 represents the radius of the target area; R safe Indicates the safety radius of the unmanned boats of both sides; represents the ranging information of the laser radar carried by the mth attacking unmanned boat; r single Indicates the penetration reward of a single attacking unmanned boat; r group Indicates the coordination bonus of the attacking unmanned boat; in: represents the distance between the mth attacking unmanned boat and the target center; represents the distance between the mth attacking unmanned boat and the target center at the next moment; P represents the dimension of the ranging information; Represents the i-th distance information in the ranging information; ξ i Indicates the collision threat coefficient between the unmanned boat and other detected unmanned boats; represents the perception information of the mth attacking unmanned boat; i represents the index number of the ranging information; c, m both represent the index number of the attacking unmanned boat, and c≠m; Represents the relative azimuth between the cth attacking unmanned boat and the mth attacking unmanned boat.

Images