A multi-ship cooperative target search method and system for complex search scenarios

Abstract

This invention discloses a multi-ship cooperative target search method and system for complex search scenarios. The multi-ship cooperative target search system is installed on the ship's hull and includes an information acquisition module, a top-level task coordination and allocation module, a main control module, a communication module, a ship motion decision-making module, and a shore-based interactive control center. This invention proposes a multi-ship cooperative collision avoidance motion planning method based on the MADDPG algorithm combined with the APF algorithm. An APF-MADDPG model is constructed and trained using an LSTM network structure. The trained deep network adaptively designs parameters, enabling each intelligent ship to efficiently generate navigation and collision avoidance routes. Furthermore, a regularly symmetrical elliptical safety zone is defined for each ship, divided into 18 regions. The distance information and collision risk of each ship to other intelligent ships within each refined safety detection region are added to the state space of each ship, enhancing the model's robustness and generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of ship technology, specifically relating to a multi-ship collaborative target search method and system for complex search scenarios. Background Technology

[0002] Given the increasingly complex maritime environment, discussing only a single ship performing a specific task has some limitations. Multi-ship collaborative mission planning, as an important application of multi-agent collaborative control, has attracted widespread attention due to its enormous potential in areas such as locating marine pollution sources, maritime search and rescue missions, and military reconnaissance.

[0003] The requirements of distributed, autonomous, and robust multi-ship cooperative control are similar to the dispersed and self-organizing characteristics of biological groups such as ant colonies, bee colonies, flocks of birds, and schools of fish. Inspired by the self-organizing mechanisms of biological groups, multi-ship cooperative search significantly improves task execution efficiency through resource coordination and proximity-based searching. In multi-ship cooperative target search, several target points represent several sub-tasks, which need to be assigned to various intelligent ships. Each intelligent ship, based on its own sensory data and information shared with other intelligent ships, jointly completes the task assignment. Besides task assignment, motion planning is also a crucial step in multi-ship cooperative target search. Each intelligent ship, after knowing the target point it needs to reach, needs to find an optimal collision-avoidance navigation path to maximize search efficiency. Currently, mature and commonly used methods include the A* algorithm, the RRT algorithm, the artificial potential field method, the ant colony algorithm, and reinforcement learning-based algorithms.

[0004] Therefore, how to more accurately and efficiently solve problems such as multi-target point cooperative search and multi-ship cooperative collision avoidance path planning in complex scenarios remains a pressing challenge. Current technologies lack research on cooperative target search for multiple ships. In cooperative target search tasks, several target points constitute several sub-tasks, which need to be assigned to each intelligent ship. Each intelligent ship, based on its own perception data and information shared with other intelligent ships, jointly completes the task assignment. Besides task assignment, motion planning is also a crucial step in multi-ship cooperative target search. Each intelligent ship, after knowing the target point it needs to reach, must simultaneously find an optimal collision avoidance navigation path to maximize search efficiency. Therefore, how to more accurately and efficiently solve multi-ship cooperative target search in complex scenarios remains a pressing challenge. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-ship cooperative target search method and system for complex search scenarios, which realizes the allocation of target points in multi-ship cooperative target search tasks and ensures the navigation safety of each intelligent ship during the target search process.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is: a multi-ship collaborative target search system for complex search scenarios. This system is installed on the ship's hull and includes: an information acquisition module, a top-level task collaborative allocation module, a main control module, a communication module, a ship motion decision-making module, and a shore-based interactive control center; wherein,

[0007] The information acquisition module is used to acquire perception information of the mission area environment. Several intelligent ships are set up in the mission area. The perception information includes at least: obstacles in the environment, the position of intelligent ships and weather information, and the position, speed and heading information of the ship itself and other intelligent ships.

[0008] The top-level task coordination and allocation module is used for perception information calculation and target point search task allocation. Specifically, perception information calculation involves acquiring perception information through the information acquisition module and calculating the motion parameters between ships, the encounter situation, and the collision risk between ships. Specifically, target point search task allocation involves allocating each target point to each smart ship based on the distance, turning direction, and obstacle density of each smart ship to the target point using the Hungarian algorithm, and sending the task allocation information to the main control module.

[0009] The main control module is used to integrate the perception information and task allocation information collected by each smart ship, and jointly control each smart ship to complete the target point search task.

[0010] The communication module is used for information transmission and coordination between the smart ship and the shore-based interactive control center, as well as among the smart ships, to ensure real-time sharing of mission execution status, sensing information, and the action information of other ships.

[0011] The ship motion decision-making module includes an intelligent ship collision avoidance module and an intelligent ship maneuvering motion control module. Each intelligent ship is equipped with a corresponding intelligent ship collision avoidance module, which is used to make real-time decisions based on environmental perception data and the behavior of other intelligent ships to ensure that each intelligent ship can safely avoid obstacles and other intelligent ships and generate feasible target point search paths.

[0012] The ship maneuvering motion control module is used to convert the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, controlling each intelligent ship to smoothly reach each target point and complete the target search task.

[0013] The objective function of the Hungarian algorithm in the top-level task collaborative allocation module is expressed as:

[0014]

[0015]

[0016] In the above formula, Z(t) refers to the total cost after the target point is allocated at time t, and the cost matrix C(t) = [C ij (t)]n×m D refers to the cost required for the i-th smart ship to select target point j at time t; ij (t) = 1 indicates that the j-th target point at time t corresponds to the i-th intelligent ship, if D ij If (t) = 0, it means that the j-th target point failed to match any smart ship; after calculation using the Hungarian algorithm, an allocation matrix D(t) = [D ij (t)] n×m .

[0017] The ship motion decision-making module integrates the Deep Deterministic Policy Gradient (MADDPG) algorithm and the Artificial Potential Field (APF) method to form the APF-MADDPG algorithm, enabling each intelligent ship to avoid obstacles and reach its target point. The formation steps of the APF-MADDPG algorithm are as follows:

[0018] Considering the continuous motion characteristics of ships in a multi-ship cooperative search scenario, a continuous state space is constructed using speed, heading, the relative position of the target point to itself, and the relative position of each ship to other ships or static obstacles. in, This represents the state space of the intelligent ship i at time step t. This represents the speed of intelligent ship i at time step t. This indicates the heading of the intelligent ship i at time step t. This represents the distance between the intelligent ship i and the target point at time step t. This represents the distance between the intelligent ship i and surrounding static obstacles at time step t. This indicates the collision risk assessment of intelligent ship i with other intelligent ships at time step t;

[0019] To better reflect real-world navigation scenarios, a continuous motion space is designed based on changes in the ship's rudder angle. This represents the action space of the intelligent ship i at time step t. This represents the rudder angle of the intelligent ship i at time step t;

[0020] The reward function is designed by taking into account the proximity to the target point and navigation safety performance. The distance between each ship and each target search point and the smooth curve generated during navigation are used as navigation rewards during multi-ship cooperative search. When collisions occur between smart ships or between smart ships and static obstacles, negative reward values ​​are given.

[0021] It also includes: constructing an LSTM-MADDPG training network for multi-ship cooperative route search in complex search scenarios using the APF-MADDPG algorithm; this step specifically involves:

[0022] Initialize the Actor network and Critic network with LSTM layers and their corresponding target networks; the LSTM layers are used to process state sequence information and generate actions or Q-values ​​through fully connected layers;

[0023] Initialize the experience replay pool, the gravity and repulsion gain coefficients of the APF algorithm, and the initial state information of each smart ship in the multi-ship cooperative system.

[0024] The MADDPG algorithm employs two sets of Actor-Critic structures, with each smart ship having its own separate Actor network and Critic network. At each time step, the Actor network is configured according to the smart ship's... i Current observation value Calculate actions That is, the rudder angle of each ship; where the Critic network output is the estimated Q value, used for evaluation. Good or bad;

[0025] After each vessel receives the action to be executed, it receives the vessel's status information for the next moment and the reward for the current moment. Then, by determining whether the ship has reached the target point or collided, it can be determined whether the ship's action is in a terminated state; if not, the obtained sampling data is then processed. Store in the experience recycling pool;

[0026] A small batch of samples is randomly drawn from the experience replay pool for Actor and Critic network updates, while the target network parameters are softly updated from the original network with tiny steps. The method for soft updating the target network parameters is expressed as follows:

[0027] θ μ’ ←τθ μ +(1-τ)θ μ’ θ Q’ ←τθ Q +(1-τ)θ Q’

[0028] Wherein, the step size τ is the soft update step size, usually between 0 and 1, meaning that the target network only makes minor adjustments to the parameters of the main network each time it is updated; θ μ These are the main network parameters of the Actor, θ μ’ θ represents the target network parameters of the Actor. Q These are the main network parameters of Critic, θ Q’ These are the target network parameters for Critic.

[0029] When calculating the collision insecurity level (CRI) between ships, the nearest encounter distance (DCPA) and the nearest encounter time (TCPA) are selected as pricing indicators by combining the dynamic intelligent ship's own safety domain model and safety distance.

[0030] The collision reward setting in the APF-MADDPG algorithm is based on fuzzy logic. First, the membership functions for the unsafety levels of the two indices, DCPA and TCPA, are determined. Then, the two indices are weighted and fused to obtain the degree of collision unsafety between ships.

[0031] The DCPA (Discretionary Density Assessment) membership function is used to represent the probability of a collision between two ships. The smaller the DCPA value, the greater the probability of a collision. This membership function u DCPA for:

[0032]

[0033] In the formula, n q n is the number of intersections between the collision hazard detection line and the boundary of the ship's domain. s R1 is the number of intersections between the collision hazard detection line and the extended boundary of the vessel's territory; R2 is the distance from the intersection of the DCPA line and the boundary of the vessel's territory to the vessel itself; R2 is the distance from the intersection of the extended DCPA line and the boundary of the extended vessel's territory to the vessel itself.

[0034] The DCPA value is calculated as follows:

[0035]

[0036] In the formula, β represents the coordinates of other intelligent ships in the ship's coordinate system; β represents the relative heading in the ship's coordinate system.

[0037] The TCPA unsafety membership function is used to represent the urgency of the collision risk between two ships. This membership function u TCPA for:

[0038]

[0039] In the formula, T0 is the TCPA value when other smart ships are located on the boundary of the extended ship domain. When TCPA = T0, u TCPA =0.5; n s The number of intersections between the collision hazard detection line and the expanded boundary of the vessel's domain;

[0040] The TCPA value is calculated as follows:

[0041]

[0042] In the formula, This indicates the relative speed of this ship to other intelligent ships at time t;

[0043] The membership degrees of DCPA and TCPA are calculated using an average weighted fuzzy operator to obtain the inter-ship collision hazard ratio (CRI), which is expressed as:

[0044] CRI=ω1.u DCPA +ω2.u TCPA

[0045] In the formula, ω1 represents the fuzzy weight of DCPA on CRI, and ω2 represents the fuzzy weight of TCPA on CRI.

[0046] The ship collision avoidance module defines a regular, symmetrical elliptical safety zone for the ship, dividing this zone into 18 regions to improve the generalization ability of the reinforcement learning algorithm. After region division, it obtains the ship's position coordinates, as well as the distance D and relative bearing of the ship to other intelligent ships. When the distance between the two ships is less than the elliptical safe distance d of the ship i Safe distance d from other smart ships j The sum of, i.e., D <d i +d j Then, based on the relative bearings of other intelligent ships to this ship... Record the distance D in the corresponding region, i.e., K. i =D; if no other intelligent ship intrudes into the elliptical security area of ​​this ship, it is denoted as K. i =d i (i = 1, 2, ..., 18);

[0047] The degree of collision hazard is recorded in the corresponding area, i.e., C, based on the relative positions of other smart ships to this ship. i =CRI; if there is no risk of collision, it is recorded as C. i =0;

[0048] Information regarding the collision danger status of this vessel with another intelligent vessel at time t. Represented as:

[0049]

[0050] In the formula, K i C represents the distance between the specified smart ship and the i-th region within the elliptical safe zone; i This indicates the degree of unsafety in a collision between the specified smart ship and the current ship within the i-th region of the elliptical safe zone.

[0051] The dynamic ship safety domain model uses the quaternions of the QSD as the coordinates of the four vertices of the ship's safety domain, represented as: (R fore ,0),(-R aft ,0),(0,Rstarb ),(0,-R port The parameters for the ship's safety domain are expressed as follows:

[0052]

[0053] In the formula, (R) fore R aft R starb R port These represent the radii in the four directions (forward, aft, left, and right) of a quaternary ship.

[0054] Therefore, the parametric equation of the ellipse can be expressed as:

[0055]

[0056] The QSD quaternion representation is as follows:

[0057]

[0058] Where, k AD Let k be the gain coefficient for the approach distance during the ship's turning circle. DT The gain coefficient with respect to the ship's turning diameter is expressed as:

[0059]

[0060] In the formula, v is the ship's speed in knots, and L is the ship's length.

[0061] The extended security domain of the intelligent ship is the result of scaling the specified security domain of the ship by a coefficient f(t); where f(t) = D / D S D S This represents the minimum safe distance for this ship, and D represents the distance between smart ships at different times.

[0062] A multi-ship cooperative target search method for complex search scenarios is also provided, characterized by the following steps:

[0063] Obtain environmental information of the search task area to determine the location of the target point and the environmental information of the task area;

[0064] The target points are assigned based on the analysis of the distance, turning direction, and obstacle density of each smart ship to the target point using the Hungarian algorithm.

[0065] After the target point search task is assigned, the ship motion decision module generates a multi-ship collaborative navigation collision avoidance path for each intelligent ship. Each intelligent ship monitors the surrounding environment in real time through the information acquisition module, makes dynamic local path adjustments, and replans the navigation route.

[0066] Each intelligent ship executes the task of searching for designated target points according to the route generated by the ship motion decision module. During the task execution, each ship perceives environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each intelligent ship.

[0067] Each ship reported its mission completion status to the shore-based interactive control center in real time, and the multi-ship collaborative target search mission was completed.

[0068] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0069] This invention utilizes the Hungarian algorithm to ensure the rationality of target point allocation in multi-ship cooperative target search tasks. For the ship motion decision-making module during the cooperative search process, it proposes a multi-ship cooperative collision avoidance motion planning method based on the MADDPG algorithm combined with the APF algorithm. An APF-MADDPG model is established, and the network structure is trained using LSTM. The trained deep network adaptively designs parameters, enabling each intelligent ship to efficiently generate navigation and collision avoidance routes. Furthermore, for the ship collision avoidance module in the multi-ship cooperative search process, it integrates ship safety domain and collision unsafety level detection to calculate the Collision Unsafety Index (CRI) between the ship and other intelligent ships. A regular symmetrical elliptical safety region is defined for the ship, and this safety region is divided into 18 areas. The distance information of the ship and other intelligent ships, as well as the collision risk level, in each refined safety detection area are added to the state space, enhancing the model's robustness and generalization ability. Attached Figure Description

[0070] Figure 1 This is a system framework diagram of an embodiment of the present invention;

[0071] Figure 2 This is a flowchart illustrating the method of an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the workflow of each module in an embodiment of the present invention;

[0073] Figure 4 This is an algorithm architecture diagram of the ship motion decision module in an embodiment of the present invention;

[0074] Figure 5 This is a logic diagram for judging the collision risk level of the ship in an embodiment of the present invention;

[0075] Figure 6 This is a schematic diagram illustrating the parameters for calculating the collision risk of two ships in an embodiment of the present invention.

[0076] Figure 7 This is a schematic diagram illustrating the division of security domains in an embodiment of the present invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0078] To address the problem of multi-ship cooperative target search in complex scenarios, this invention proposes a method and system for multi-ship cooperative target search in complex search scenarios, enabling dynamic collision avoidance path planning and target search tasks in complex environments. The proposed method and system offer advantages such as high inspection efficiency and a safe and efficient multi-ship cooperative target search process.

[0079] refer to Figures 1-7 The technical solution of the present invention is as follows:

[0080] Example 1:

[0081] A method and system for multi-ship cooperative target search in complex search scenarios are provided. The method includes:

[0082] Acquire environmental information about the search task area, determine the location of the target point, and information such as the environmental characteristics of the task area;

[0083] The target points are assigned based on the analysis of the distance, turning direction, and obstacle density of each smart ship to the target point using the Hungarian algorithm.

[0084] After the target point search task is assigned, the global path planning module generates the global planned path for each smart ship. Each smart ship monitors the surrounding environment in real time through the information acquisition module, makes dynamic local path adjustments, and replans the navigation route.

[0085] Each intelligent ship executes the task of searching for the designated target point according to the route generated by the global path planning module. During the task execution, each ship perceives environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each intelligent ship.

[0086] Each intelligent vessel reports its mission completion status to the shore-based interactive control center in real time, and the multi-vessel collaborative target search mission is completed.

[0087] The further technical solution is as follows: The objective function for allocating multiple target points based on the Hungarian algorithm is shown below:

[0088]

[0089]

[0090] In the above formula, Z(t) refers to the total cost after the target point is allocated at time t, and the cost matrix C(t) = [C ij (t)] n×m D refers to the cost required for the i-th smart ship to select target point j at time t; ij (t) = 1 indicates that the j-th target point at time t corresponds to the i-th intelligent ship, if D ij If (t) = 0, it means that the j-th target point failed to match any smart ship; after calculation using the Hungarian algorithm, an allocation matrix D(t) = [D ij (t)] n×m .

[0091] Example 2:

[0092] A multi-ship collaborative target search method and system for complex search scenarios is also proposed. The system includes: an information acquisition module, a top-level task collaborative allocation module, a main control module, a communication module, a ship motion decision module, and a shore-based interactive control center.

[0093] The information acquisition module mainly acquires the perception information of the mission area environment. Several intelligent ships are set up in the mission area. The perception information includes at least: obstacles in the environment, the position of the intelligent ships and meteorological information, and the position, speed and heading information of the ship itself and other intelligent ships.

[0094] The top-level task control module includes perception information calculation and target point search task allocation. Perception information calculation specifically involves calculating the inter-ship motion parameters, encounter situation, responsibility for avoiding collision, and degree of unsafety in inter-ship collision based on the perception information obtained by the information acquisition module. Target point search task allocation specifically involves assigning the search task for each target point to each intelligent ship based on the distance, turning direction, and obstacle density of each intelligent ship to the target point, and sending the task allocation information to the main control module.

[0095] The main control module integrates the perception information and task allocation information collected by each smart ship, and jointly controls each smart ship to complete the target point search task.

[0096] The communication module is responsible for information transmission and coordination between the smart ship and the shore-based interactive control center, as well as among the smart ships, to ensure real-time sharing of mission execution status, perception information, and the actions of other ships.

[0097] Each intelligent vessel collision avoidance decision module makes real-time decisions based on environmental perception data and the behavior of other intelligent vessels, ensuring that each intelligent vessel can safely avoid obstacles and other intelligent vessels, and generating feasible target point search paths.

[0098] The intelligent ship maneuvering motion control module is responsible for converting the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, controlling each intelligent ship to smoothly reach each target point and complete the target search task.

[0099] Its further technical solution is as follows: The global path planning module integrates the Multi-Ship Agent Deep Deterministic Policy Gradient (MADDPG) algorithm and the Artificial Potential Field (APF) method, proposing the APF-MADDPG algorithm to complete the global path planning for each intelligent ship to reach each target point. Specifically, this algorithm includes: adjusting the gravity gain coefficient η in the APF algorithm... all and repulsive gain coefficient η rep This serves as the continuous action space within the policy function of each intelligent ship. The perception modules within each intelligent ship acquire information such as the distance to surrounding static obstacles, the current position of each intelligent ship, the current bow roll rate, and the current bow angle, which together form the local state space information of each intelligent ship, i.e., s. t ={P s psi, d1, ..., d i P g The reward function should be designed considering factors such as approaching the designated point as quickly as possible, avoiding surrounding obstacles effectively, and the smoothness of the generated route.

[0100] Based on the three key elements of reinforcement learning algorithms mentioned above, an LSTM-MADDPG training network for multi-ship cooperative route search in complex search scenarios is constructed using the APF-MADDPG algorithm. First, the Actor and Critic networks with LSTM layers and their corresponding target networks are initialized. The LSTM layers process state sequence information, generating actions or Q-values ​​through fully connected layers. Simultaneously, the experience replay pool, the attraction and repulsion gain coefficients of the APF algorithm, and the initial state information of each intelligent ship in the multi-ship cooperative system are initialized. The MADDPG algorithm employs two sets of Actor-Critic structures, with each intelligent ship having its own separate Actor and Critic network. At each time step, the Actor network adjusts its current Shipi observations based on the intelligent ship's current observations. Calculate actions That is, the rudder angle of each ship. The Critic network output is the estimated Q-value, used to evaluate... The quality of the action. After each vessel receives the action to be executed, it receives the vessel's status information for the next moment and the reward for the current moment. Then, by determining whether the ship has reached the target point or collided, it can be determined whether the ship's action is in a terminated state; if not, the obtained sampling data is then processed. Store in the experience recycling pool.

[0101] Randomly select a small batch of samples from the experience replay pool to update the Actor network and Criti. c The network is updated, and the target network parameters are simultaneously soft-updated from the original network in small steps. The method for soft-updating the target network is expressed as follows.

[0102] θ μ’ ←τθ μ +(1-τ)θ μ' θ Q' ←τθ Q +(1-τ)θ Q'

[0103] Wherein, the step size τ is the soft update step size, usually between 0 and 1, meaning that the target network only makes minor adjustments to the parameters of the main network each time it is updated; θ μ These are the main network parameters of the Actor, θ μ′ θ represents the target network parameters of the Actor. Q These are the main network parameters of Critic, θ Q’ These are the target network parameters for Critic.

[0104] When calculating the collision insecurity level (CRI) between ships, the nearest encounter distance (DCPA) and the nearest encounter time (TCPA) are selected as pricing indicators by combining the dynamic intelligent ship's own safety domain model and safety distance.

[0105] Based on the principles of fuzzy theory, we first determine the membership functions of the two indicators, the nearest encounter distance (DCPA) and the nearest encounter time (TCPA), to determine the degree of unsafety in ship-to-ship collisions. Then, we perform a weighted fusion of the two indicators to obtain the degree of unsafety in ship-to-ship collisions.

[0106] The membership function of the DCPA index indicates that the smaller the DCPA value, the greater the probability of a collision between the two ships. Its membership function u... DCPA for:

[0107]

[0108] In the formula, n q n is the number of intersections between the collision hazard detection line and the boundary of the ship's domain. sR1 is the number of intersections between the collision hazard detection line and the extended vessel domain boundary; R2 is the distance from the intersection of the DCPA line and the vessel domain boundary to the target vessel; R3 is the distance from the intersection of the extended DCPA line and the extended vessel domain boundary to the target vessel.

[0109] The DCPA value is calculated as follows:

[0110]

[0111] In the formula: β represents the coordinates of the intelligent ship in the ship's coordinate system; β represents the relative heading in the ship's coordinate system.

[0112] The TCPA unsafety membership function indicates the urgency of the collision risk between two ships. This membership function u TCPA for:

[0113]

[0114] In the formula, T0 is the TCPA value when other smart ships are located on the boundary of the extended ship domain. When TCPA = T0, u TCPA =0.5; n s This refers to the number of intersections between the collision hazard detection line and the extended boundary of the vessel's domain.

[0115] The TCPA value is calculated as follows:

[0116]

[0117] In the formula, This represents the relative speed of this ship to other intelligent ships at time t.

[0118] The membership degrees of DCPA and TCPA are calculated using an average weighted fuzzy operator to obtain the collision insecurity level (CRI) between ships, which is expressed as:

[0119] CRI=ω1.u DCPA +ω2.u TCPA

[0120] In the formula, ω1 represents the fuzzy weight of DCPA on CRI, and ω2 represents the fuzzy weight of TCPA on CRI.

[0121] The ship collision avoidance module defines a regular, symmetrical elliptical safety zone for the ship, dividing this zone into 18 regions to improve the generalization ability of the reinforcement learning algorithm. After region division, it obtains the ship's position coordinates, as well as the distance D and relative bearing of the ship to other intelligent ships. When the distance between the two ships is less than the elliptical safe distance d of the ship i Safe distance d from other smart shipsj The sum of, i.e., D <d i +d j Then, based on the relative bearings of other intelligent ships to this ship... Record the distance D in the corresponding region, i.e., K. i =D; if no other intelligent ship intrudes into the elliptical security area of ​​this ship, it is denoted as K. i =d i (i = 1, 2, ..., 18);

[0122] The degree of collision hazard is recorded in the corresponding area, i.e., C, based on the relative positions of other smart ships to this ship. i =CRI; if there is no risk of collision, it is recorded as C. i =0;

[0123] Information regarding the collision danger status of this vessel with another intelligent vessel at time t. Represented as:

[0124]

[0125] In the formula, K i C represents the distance between the specified smart ship and the i-th region within the elliptical safe zone; i This indicates the degree of unsafety in a collision between the specified smart ship and the current ship within the i-th region of the elliptical safe zone.

[0126] The dynamic ship safety domain model uses the quaternions of the QSD as the coordinates of the four vertices of the ship's safety domain, represented as: (R fore ,0),(-R aft ,0), (0,R starb ), (0, -R port The parameters for the ship's safety domain are expressed as follows:

[0127]

[0128] In the formula, (R) fore R aft R starb R port ) represent the radii in the four directions of front, rear, left, and right in the quaternary shipping field.

[0129] Therefore, the parametric equation of the ellipse is:

[0130]

[0131] The QSD quaternion representation is as follows:

[0132]

[0133] Where, k ADLet k be the gain coefficient for the approach distance during the ship's turning circle. DT The gain coefficient with respect to the ship's turning diameter can be expressed as:

[0134]

[0135] In the formula, v is the ship's speed in knots, and L is the ship's length.

[0136] The extended security domain of the intelligent ship is the result of scaling the specified security domain of the ship by a coefficient f(t); where f(t) = D / D S D S This represents the minimum safe distance for this ship, and D represents the distance between smart ships at different times.

[0137] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-ship cooperative target search system for complex search scenarios, characterized in that, The system is mounted on the ship and includes: an information acquisition module, a top-level task coordination and allocation module, a main control module, a communication module, a ship motion decision-making module, and a shore-based interactive control center; among which, The information acquisition module is used to acquire perception information of the mission area environment. Several intelligent ships are set up in the mission area. The perception information includes at least: obstacles in the environment, the position of intelligent ships and weather information, and the position, speed and heading information of the ship itself and other intelligent ships. The top-level task coordination and allocation module is used for perception information calculation and target point search task allocation. Specifically, perception information calculation involves acquiring perception information through the information acquisition module and calculating the motion parameters between ships, the encounter situation, and the collision risk between ships. Specifically, target point search task allocation involves allocating each target point to each smart ship based on the distance, turning direction, and obstacle density of each smart ship to the target point using the Hungarian algorithm, and sending the task allocation information to the main control module. The main control module is used to integrate the perception information and task allocation information collected by each smart ship, and jointly control each smart ship to complete the target point search task. The communication module is used for information transmission and coordination between the smart ship and the shore-based interactive control center, as well as among the smart ships, to ensure real-time sharing of mission execution status, sensing information, and the action information of other ships. The ship motion decision-making module includes an intelligent ship collision avoidance module and an intelligent ship maneuvering control module. Each intelligent ship is equipped with a corresponding intelligent ship collision avoidance module, which is used to make real-time decisions based on environmental perception data and the behavior of other intelligent ships to ensure that each intelligent ship can safely avoid obstacles and other intelligent ships and generate feasible target point search paths. The ship motion decision-making module integrates the deep deterministic policy gradient algorithm MADDPG and the artificial potential field method APF to form the APF-MADDPG algorithm, which enables each intelligent ship to avoid obstacles and reach each target point. The collision reward setting in the APF-MADDPG algorithm is based on the principle of fuzzy theory. First, the membership functions of the unsafety degree of the two indicators DCPA and TCPA are determined, and then the two indicators are weighted and fused to obtain the collision risk between ships. The ship maneuvering motion control module is used to convert the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, so as to control each intelligent ship to smoothly reach each target point and complete the target search task. DCPA Unsafety Membership Function The DCPA value is used to represent the probability of a collision between two ships. The smaller the DCPA value, the greater the probability of a collision. for: In the formula, The number of intersections between the collision hazard detection line and the boundary of the ship's domain; The number of intersections between the collision hazard detection line and the expanded boundary of the vessel's domain; The distance from the intersection of the DCPA line and the boundary of the ship's domain to the ship itself; The distance from the point where the DCPA extension line intersects with the extended boundary of the vessel's territory to the vessel itself; The DCPA value is calculated as follows: In the formula, , The coordinates of other intelligent ships in this ship's coordinate system; The relative heading in the ship's coordinate system; TCPA Insecurity Membership Function Used to indicate the urgency of the danger of a collision between two ships. for: In the formula, The TCPA value is the size when other smart ships are located on the extended ship domain boundary, when hour, ; The TCPA value is calculated as follows: In the formula, This indicates that this ship and other intelligent ships are at the same time. The relative speed is denoted by D; D is the distance between this ship and other intelligent ships. Will and The collision risk index (CRI) between ships is calculated using the average weighted fuzzy operator and is expressed as: In the formula, This represents the fuzzy weight of DCPA on CRI. This represents the fuzzy weight of TCPA on CRI.

2. The multi-ship cooperative target search system for complex search scenarios according to claim 1, characterized in that, The objective function of the Hungarian algorithm in the top-level task collaborative allocation module is expressed as: The above formula Refers to the first t Total cost after target point allocation at each time point, cost matrix Refers to the first t Time of the first i The intelligent ship selected the target point. j The required costs; =1 indicates the first t Time of the first j The target point corresponds to the first i A smart ship, if This means the first j The target point failed to match any smart ship; the Hungarian algorithm will return an assignment matrix after calculation. .

3. The multi-ship cooperative target search system for complex search scenarios according to claim 1, characterized in that, The steps involved in forming the APF-MADDPG algorithm are as follows: Considering the continuous motion characteristics of ships in a multi-ship cooperative search scenario, a continuous state space is constructed using speed, heading, the relative position of the target point to itself, and the relative position of each ship to other ships or static obstacles. ,in, Indicating intelligent ships At time step The state space above, Indicating intelligent ships At time step The speed of time, Indicating intelligent ships At time step The direction of the bow at that time Indicating intelligent ships At time step Distance between time and target point Indicating intelligent ships At time step The distance between the object and surrounding static obstacles. Indicating intelligent ships At time step Determining the risk of collision with other intelligent ships; To better reflect real-world navigation scenarios, a continuous motion space is designed based on changes in the ship's rudder angle. , Indicating intelligent ships At time step The space of movement on top Indicating intelligent ships At time step The rudder angle at that time; The reward function is designed by taking into account the proximity to the target point and navigation safety performance. The distance between each ship and each target search point and the smooth curve generated during navigation are used as navigation rewards during multi-ship cooperative search. When collisions occur between smart ships or between smart ships and static obstacles, negative reward values ​​are given.

4. A multi-ship cooperative target search system for complex search scenarios according to claim 3, characterized in that, Also includes: An LSTM-MADDPG training network for multi-ship cooperative route search in complex search scenarios is constructed using the APF-MADDPG algorithm; the specific steps are as follows: Initialize the Actor network and Critic network with LSTM layers and their corresponding target networks; the LSTM layers are used to process state sequence information and generate actions or Q-values ​​through fully connected layers; Initialize the experience replay pool, the gravity and repulsion gain coefficients of the APF algorithm, and the initial state information of each smart ship in the multi-ship cooperative system. The MADDPG algorithm employs two sets of Actor-Critic structures, with each smart ship having its own separate Actor network and Critic network; at each time step, the Actor network is configured according to the smart ship's... Current observation value Calculate actions That is, the rudder angle of each ship; where the Critic network output is the estimated Q value, used for evaluation. Good or bad; After each vessel receives the action to be executed, it receives the vessel's status information for the next moment and the reward for the current moment. Furthermore, by determining whether the vessel has reached the target point or experienced a collision, it can be determined whether the vessel's action has ended. If it has not ended, the obtained sampled data is then processed. Store in the experience recycling pool; A small batch of samples is randomly drawn from the experience replay pool for Actor and Critic network updates, while the target network parameters are softly updated from the original network with tiny steps. The method for soft updating the target network parameters is expressed as follows: Among them, step size This is the soft update step size, usually between 0 and 1, meaning that the target network only makes minor adjustments to the parameters of the main network each time it is updated; These are the main network parameters of the Actor. These are the target network parameters of the Actor; These are the main network parameters of Critic. These are the target network parameters for Critic.

5. A multi-ship cooperative target search system for complex search scenarios according to claim 3, characterized in that, When calculating the collision risk index (CRI) between ships, the nearest encounter distance (DCPA) and the nearest encounter time (TCPA) are selected as pricing indicators by combining the dynamic intelligent ship's own safety domain model and safety distance.

6. A multi-ship cooperative target search system for complex search scenarios according to claim 3, characterized in that, The ship collision avoidance module defines a regular, symmetrical elliptical safety zone for the ship, dividing this zone into 18 regions to improve the generalization ability of the reinforcement learning algorithm. After region division, it obtains the ship's position coordinates and the distances from the ship to other intelligent ships. relative orientation ; When the distance between the two ships is less than the elliptical safe distance of this ship Safe distance from other smart ships The sum of Then, based on the relative bearings of other intelligent ships to this ship... Distance Recorded in the corresponding area, i.e. If no other intelligent vessel intrudes into the elliptical security area of ​​this vessel, it is denoted as... ; The risk of collision between ships is recorded in the corresponding area based on the relative positions of other intelligent ships to this ship. ; If there is no risk of collision, then record. ; This ship is at t Constantly monitored by information on the collision risk of other intelligent ships Represented as: In the formula, The first elliptical safe region Distance from the designated smart ship within a given area; The first elliptical safe region The risk of collision between a designated smart ship and this ship within a specific area.

7. A multi-ship cooperative target search system for complex search scenarios according to claim 5, characterized in that, The dynamic ship safety domain model uses the quaternions of QSD as the coordinates of the four vertices of the ship's safety domain, represented as follows: , , , The parameters obtained in the ship's safety field are expressed as follows: In the formula, These represent the radii in the four directions (forward, aft, left, and right) of a quaternary ship. Therefore, the parametric equation of the ellipse can be expressed as: The QSD quaternion representation is as follows: in, This is the gain coefficient for the approach distance during a ship's turning circle. The gain coefficient with respect to the ship's turning diameter is expressed as: In the formula, The speed of the ship is measured in knots. He is the captain; The expanded smart ship's own security domain is defined as the security domain of the designated ship itself, based on coefficients. The scaled result; where, This indicates the minimum safe distance for this vessel.

8. A search method implemented using the multi-ship cooperative target search system for complex search scenarios as described in any one of claims 1-7, characterized in that, Includes the following steps: Obtain environmental information of the search task area to determine the location of the target point and the environmental information of the task area; The target points are assigned based on the analysis of the distance, turning direction, and obstacle density of each smart ship to the target point using the Hungarian algorithm. After the target point search task is assigned, the ship motion decision module generates a multi-ship collaborative navigation collision avoidance path for each intelligent ship. Each intelligent ship monitors the surrounding environment in real time through the information acquisition module, makes dynamic local path adjustments, and replans the navigation route. Each intelligent ship executes the task of searching for designated target points according to the route generated by the ship motion decision module. During the task execution, each ship perceives environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each intelligent ship. Each ship reported its mission completion status to the shore-based interactive control center in real time, and the multi-ship collaborative target search mission was completed.

Images