Multi-ship collaborative target searching method and system for complex searching scene

Abstract

The invention discloses a multi-ship cooperative target searching method and system for a complex searching scene, and the system is disposed on a ship body, and comprises an information collection module, a top task cooperative distribution module, a main control module, a communication module, a ship motion decision module, and a shore end interaction control center. The invention provides a method for realizing multi-ship collaborative collision avoidance motion planning based on an MADDPG algorithm in combination with an APF algorithm. An APF-MADDPG model is constructed, a network structure is trained through LSTM, parameters are adaptively designed by using a trained deep network, so that each intelligent ship can efficiently generate a navigation collision avoidance route, in addition, a regular and symmetrical elliptical safety area is set for the ship, and the safety area is divided into 18 areas. The refined distance information between the ship and other intelligent ships in each safety detection area and the collision risk degree are added into the state space of each ship, and the robustness and generalization ability of the model are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of ships, and in particular relates to a multi-ship collaborative target search method and system for complex search scenarios. Background Art

[0002] In the face of increasingly complex maritime environments, only discussing the execution of a single ship on a mission has some limitations. As an important application in multi-agent collaborative control, multi-ship collaborative mission planning has attracted extensive attention due to its great potential in marine pollution source location, water search and rescue missions, and military reconnaissance.

[0003] The requirements of multi-ship collaborative control, such as distribution, autonomy, and robustness, are similar to the dispersion and self-organization characteristics of biological groups such as ant colonies, bee colonies, bird flocks, and fish schools in the biological world. Multi-ship collaborative search is inspired by the self-organization mechanism of biological groups. Through resource coordination and nearby search, it greatly improves the efficiency of task execution. In multi-ship collaborative target search, several target points are several subtasks, which need to be assigned to each smart ship. Each smart ship completes the task assignment operation based on its own perception data and information shared with other smart ships. In addition to task allocation, motion planning is also one of the key steps in multi-ship collaborative target search. After knowing the target point to be reached, each smart ship needs to find an optimal collision avoidance navigation path to maximize the search efficiency. At present, the more mature and commonly used methods include A* algorithm, RRT algorithm, artificial potential field method, ant colony algorithm, and reinforcement learning-based algorithm.

[0004] Therefore, how to more accurately and efficiently solve the problems of collaborative search of multiple target points and collaborative collision avoidance path planning of multiple ships in complex scenarios is still a difficult problem that needs to be solved urgently. The existing technology lacks research on collaborative target search for multiple ships. In the collaborative target search task, several target points are several subtasks, which need to be assigned to each smart ship. Each smart ship completes the task assignment task jointly based on its own perception data and information shared with other smart ships. In addition to task allocation, motion planning is also one of the key steps in the collaborative target search task of multiple ships. After knowing the target point to be reached, each smart ship needs to find an optimal collision avoidance navigation path at the same time to maximize the search efficiency. Therefore, how to more accurately and efficiently solve the collaborative target search of multiple ships in complex scenarios is still a difficult problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-ship collaborative target search method and system for complex search scenarios, which realizes the target point allocation in the multi-ship collaborative target search task and ensures the navigation safety of each intelligent ship during the target search process.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a multi-ship collaborative target search system for complex search scenarios, which is arranged on the hull and includes: an information acquisition module, a top-level task collaborative allocation module, a main control module, a communication module, a ship motion decision module and a shore-side interactive control center; wherein,

[0007] The information collection module is used to obtain the perception information of the mission area environment. Several smart ships are set up in the mission area. The perception information at least includes: obstacles in the environment, the location of smart ships and meteorological information, and the location, speed and heading information of the ship and other smart ships;

[0008] The top-level task collaborative allocation module is used for perception information solution and target point search task allocation; perception information solution is to obtain perception information through the information acquisition module, and solve the inter-ship motion parameters, encounter situation and inter-ship collision risk; target point search task allocation is to allocate each target point to each smart ship through the Hungarian algorithm according to the distance, turning direction and obstacle density of each smart ship to the target point, and send the task allocation information to the main control module;

[0009] The main control module is used to integrate the perception information and task allocation information collected by each smart ship, and jointly control each smart ship to complete the target point search task;

[0010] Communication module, used for information transmission and coordination between smart ships and the shore-side interactive control center, as well as between smart ships, to ensure real-time sharing of mission execution status, perception information, and action information of other ships;

[0011] Ship motion decision module, including smart ship collision avoidance module and smart ship maneuvering motion control module; each smart ship is equipped with a corresponding smart ship collision avoidance module, which is used to make real-time decisions based on environmental perception data and the behavior of other smart ships, to ensure that each smart ship can safely avoid obstacles and other smart ships, and generate a feasible target point search path;

[0012] The ship maneuvering motion control module is used to convert the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, control each intelligent ship to smoothly reach each target point, and complete the target search task.

[0013] The objective function of the Hungarian algorithm in the top-level task collaborative allocation module is expressed as:

[0014]

[0015]

[0016] The above formula Z(t) refers to the total cost after the target point is allocated at time t, and the cost matrix C(t) = [C ij (t)]n×m Refers to the cost that the i-th smart ship needs to spend to select the target point j at the t-th time; D ij (t) = 1 means that the jth target point at the tth time corresponds to the i-th smart ship. ij (t) = 0, which means that the jth target point has not been successfully matched with any smart ship; after calculation using the Hungarian algorithm, a distribution matrix D(t) = [D ij (t)] n×m .

[0017] The ship motion decision module integrates the deep deterministic policy gradient algorithm MADDPG and the artificial potential field method APF to form the APF-MADDPG algorithm, so that each smart ship can avoid obstacles and reach each target point; the formation steps of the APF-MADDPG algorithm are:

[0018] Considering the continuous motion characteristics of ships in the multi-ship collaborative task search scenario, the continuous state space is constructed using the speed, heading, the relative position of the target point and itself, and the relative position of each ship to other ships or static obstacles, that is, in, represents the state space of smart ship i at time step t, represents the speed of smart ship i at time step t, represents the heading of smart ship i at time step t, represents the distance between smart ship i and the target point at time step t, represents the distance between smart ship i and the surrounding static obstacles at time step t, It represents the collision risk judgment between smart ship i and other smart ships at time step t;

[0019] In order to better fit the actual navigation scene, the continuous action space is designed based on the change of the ship's rudder angle. represents the action space of smart ship i at time step t, represents the rudder angle of smart ship i at time step t;

[0020] The reward function is designed based on whether the target point is close to the navigation safety performance. The distance between each ship and each target search point and the smooth curve generated as much as possible during the navigation process are used as navigation rewards in the multi-ship collaborative search process. When a collision occurs between smart ships or between smart ships and static obstacles, a negative reward value is given.

[0021] It also includes: constructing an LSTM-MADDPG training network for multi-ship collaborative search routes in complex search scenarios through the APF-MADDPG algorithm; this step is specifically as follows:

[0022] Initialize the Actor network and Critic network with LSTM layers and their corresponding target networks; the LSTM layer is used to process state sequence information and generate actions or Q values ​​through the fully connected layer;

[0023] Initialize the experience replay pool, the gravitational and repulsive gain coefficients of the APF algorithm, and the initial state information of each smart ship in the multi-ship collaborative system;

[0024] The MADDPG algorithm uses two sets of Actor-Critic structures, and each smart ship has a separate Actor network and Critic network; at each time step, the Actor network i Current observations Calculate Action That is, the rudder angle of each ship; the output of the Critic network is the estimated Q value, which is used to evaluate The good or bad;

[0025] After obtaining the execution action, each ship will get the state information of the ship at the next moment and the reward at the current moment. Then, by judging whether the ship has reached the target point or collided, it is known whether the ship is in the end state after executing the action; if it is not the end state, the obtained sampling data is Deposit into the experience recycling pool;

[0026] A small batch of samples is randomly drawn from the experience replay pool to update the Actor network and the Critic network. At the same time, the target network parameters are soft-updated from the original network with a small step size. The method of soft updating the target network parameters is expressed as:

[0027] θ μ’ ←τθ μ +(1-τ)θ μ’ θ Q’ ←τθ Q +(1-τ)θ Q’

[0028] Among them, the step size τ is the soft update step size, which is usually between 0 and 1, which means that the target network will only make small adjustments to the parameters of the main network each time it is updated; θ μ is the main network parameter of Actor, θ μ’ is the target network parameter of the Actor; θ Q is the main network parameter of Critic, θ Q’ It is the target network parameter of Critic.

[0029] When calculating the CRI of collision unsafety between ships, the dynamic intelligent ship's own safety domain model and the safety distance are combined, and the closest encounter distance DCPA and the closest encounter time TCPA are selected as pricing indicators to calculate the collision unsafety degree.

[0030] The setting of collision reward in APF-MADDPG algorithm is based on the principle of fuzzy theory. First, the unsafe degree membership function of the two indicators of closest encounter distance DCPA and closest encounter time TCPA is determined, and then the two indicators are weighted and integrated to obtain the unsafe degree of collision between ships.

[0031] The DCPA unsafe degree membership function is used to represent the collision probability of the two ships. The smaller the DCPA value, the greater the probability of the two ships colliding. DCPA for:

[0032]

[0033] Where n q is the number of intersections between the collision risk detection line and the ship area boundary; n s is the number of intersections between the collision risk detection line and the boundary of the expanded ship area; R1 is the distance from the intersection of the DCPA line and the boundary of the ship area to the ship; R2 is the distance from the intersection of the DCPA extension line and the boundary of the expanded ship area to the ship;

[0034] The calculation method of DCPA value is expressed as:

[0035]

[0036] In the formula, is the coordinate of other smart ships in the coordinate system of own ship; β is the relative heading in the coordinate system of own ship;

[0037] The TCPA unsafe degree membership function is used to indicate the urgency of the collision risk between two ships. TCPA for:

[0038]

[0039] Where T0 is the TCPA value when other smart ships are located on the boundary of the expanded ship domain. When TCPA = T0, u TCPA =0.5; n s is the number of intersections between the collision risk detection line and the boundary of the expanded ship area;

[0040] The calculation method of TCPA value is expressed as:

[0041]

[0042] In the formula, Indicates the relative speed between the own ship and other smart ships at time t;

[0043] The membership of DCPA and TCPA is calculated using the average weighted fuzzy operator to obtain the collision risk index CRI, which is expressed as:

[0044] CRI=ω1.u DCPA +ω2.u TCPA

[0045] Where ω1 represents the fuzzy weight of DCPA on CRI, and ω2 represents the fuzzy weight of TCPA on CRI.

[0046] The ship collision avoidance module sets a regular symmetrical elliptical safety area for the ship and divides this safety area into 18 areas to improve the generalization ability of the reinforcement learning algorithm. After the area division is completed, the position coordinates of the ship and the distance D and relative position of the ship to other smart ships are obtained. When the distance between the two ships is less than the ship's elliptical safety distance d i Keep safe distance d from other smart ships j The sum of <d i +d j , then according to the relative position of other smart ships and own ship Record the distance D in the corresponding area, i.e. K i =D; if no other smart ship invades the area of ​​the ship's elliptical safety area, it is recorded as K i =d i (i=1,2,…,18);

[0047] According to the relative position of other smart ships and the own ship, the collision unsafe degree is recorded in the corresponding area, that is, C i = CRI; if there is no collision risk, it is recorded as C i =0;

[0048] The ship is in danger of collision with other smart ships at time t. It is expressed as:

[0049]

[0050] In the formula, K i is the distance between the i-th area of ​​the elliptical safety area and the designated smart ship; C i Indicates the collision unsafety degree between the specified smart ship and the own ship in the i-th area of ​​the elliptical safety area.

[0051] The dynamic ship safety domain model uses the quaternion of QSD as the coordinates of the four vertices of the ship's safety domain, expressed as: (R fore ,0),(-R aft ,0),(0,Rstarb ),(0,-R port ), the parameter of the ship's safety area is expressed as:

[0052]

[0053] In the formula, (R fore , R aft , R starb , R port ) represent the radius of the quaternary ship domain in the front, rear, left and right directions respectively;

[0054] Then the ellipse parametric equation is expressed as:

[0055]

[0056] QSD quaternion is represented as:

[0057]

[0058] Among them, k AD is the gain factor for the approach distance in the ship turning circle, k DT is the gain coefficient about the ship turning diameter, expressed as:

[0059]

[0060] Where v is the ship speed in knots and L is the ship length;

[0061] The safety domain of the smart ship after expansion is the result of scaling the safety domain of the designated ship by the coefficient f(t); where f(t) = D / D S , D S It indicates the minimum safe distance of the ship, and D indicates the distance between smart ships at different times.

[0062] A multi-vessel collaborative target search method for complex search scenarios is also provided, which is characterized by comprising the following steps:

[0063] Obtain environmental information of the search mission area, determine the location of the target point and the environmental information of the mission area;

[0064] According to the Hungarian algorithm, the distance, turning direction and obstacle density of each smart ship to the target point are analyzed, and the target point is allocated;

[0065] After the target point search task is assigned, the multi-ship collaborative navigation and collision avoidance paths of each smart ship are generated through the ship motion decision module. Each smart ship monitors the surrounding environment information in real time through the information collection module, makes dynamic local path adjustments, and replans the navigation route;

[0066] Each smart ship performs the designated target point search task according to the route generated by the ship motion decision module. During the task execution, each ship perceives the environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each smart ship;

[0067] Each ship will provide real-time feedback of the mission completion status to the shore-based interactive control center, and the multi-ship collaborative target search mission is completed.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] For the target point allocation in the multi-ship collaborative target search task, the present invention uses the Hungarian algorithm to ensure the rationality of the target point allocation; for the ship motion decision module in the collaborative search process, a multi-ship collaborative collision avoidance motion planning method based on the MADDPG algorithm combined with the APF algorithm is proposed; an APF-MADDPG model is established, and the LSTM training network structure is used to use the trained deep network adaptive design parameters to enable each smart ship to efficiently generate a navigation collision avoidance route; in addition, for the ship collision avoidance module in the multi-ship collaborative search process, the ship safety field and the collision unsafe degree detection are integrated to calculate the collision unsafe degree CRI of the ship and other smart ships. A regular and symmetrical elliptical safety area is set for the ship, and this safety area is divided into 18 areas. The distance information between the ship and other smart ships in each refined safety detection area and the collision risk are added to the state space, which enhances the robustness and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a system framework diagram of an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of a process of an embodiment of the present invention;

[0072] Figure 3 Schematic diagram of the workflow of each module in the embodiment of the present invention;

[0073] Figure 4 This is an algorithm architecture diagram of a ship motion decision module in an embodiment of the present invention;

[0074] Figure 5 The following is a logic diagram for judging the collision risk of the own ship in an embodiment of the present invention;

[0075] Figure 6 Schematic diagram of calculation parameters of collision risk between two ships in an embodiment of the present invention

[0076] Figure 7 The diagram is a schematic diagram of security domain division in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0078] Aiming at the problem of multi-ship collaborative target search in complex scenes, the present invention proposes a multi-ship collaborative target search method and system for complex search scenes to solve the path planning and target search tasks of dynamic collision avoidance in complex scenes. The method and system proposed by the present invention have the advantages of high inspection efficiency and safe and efficient multi-ship collaborative target search process.

[0079] refer to Figure 1-Figure 7 , the technical solution of the present invention is:

[0080] Embodiment 1:

[0081] A multi-ship collaborative target search method and system for complex search scenarios are provided, the method comprising:

[0082] Obtain environmental information of the search mission area, determine the location of the target point and environmental characteristics of the mission area;

[0083] According to the Hungarian algorithm, the distance, turning direction and obstacle density of each smart ship to the target point are analyzed, and the target point is allocated;

[0084] After the target point search task is assigned, the global path planning module generates the global planning path of each smart ship. Each smart ship monitors the surrounding environment information in real time through the information collection module, makes dynamic local path adjustments, and replans the navigation route.

[0085] Each smart ship performs the designated target point search task according to the route generated by the global path planning module. During the task execution, each ship perceives the environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each smart ship;

[0086] Each smart ship will provide real-time feedback on the mission completion status to the shore-side interactive control center, and the multi-ship collaborative target search mission is completed.

[0087] Its further technical solution is: the objective function of allocating multiple target points based on the Hungarian algorithm is as follows:

[0088]

[0089]

[0090] The above formula Z(t) refers to the total cost after the target point is allocated at time t, and the cost matrix C(t) = [C ij (t)] n×m Refers to the cost that the i-th smart ship needs to spend to select the target point j at the t-th time; D ij (t) = 1 means that the jth target point at the tth time corresponds to the i-th smart ship. ij (t) = 0, which means that the jth target point has not been successfully matched with any smart ship; after calculation using the Hungarian algorithm, a distribution matrix D(t) = [D ij (t)] n×m .

[0091] Embodiment 2:

[0092] A multi-ship collaborative target search method and system for complex search scenarios is also proposed. The system includes: an information acquisition module, a top-level task collaborative allocation module, a main control module, a communication module, a ship motion decision module and a shore-side interactive control center.

[0093] The information acquisition module mainly obtains the perception information of the mission area environment. Several smart ships are set up in the mission area. The perception information at least includes: obstacles in the environment, the location of the smart ships and meteorological information, and the location, speed and heading information of the ship and other smart ships.

[0094] The top-level task control module includes perception information solution and target point search task allocation; the perception information solution is specifically the perception information obtained by the information acquisition module, and the inter-ship motion parameters, encounter situation and collision avoidance responsibility and the degree of unsafe collision between ships are solved; the target point search task allocation is specifically, according to the distance of each smart ship to the target point, the turning and the obstacle density, the search task of each target point is allocated to each smart ship, and the task allocation information is sent to the main control module.

[0095] The main control module integrates the perception information and task allocation information collected by each smart ship, and jointly controls each smart ship to complete the target point search task.

[0096] The communication module is responsible for information transmission and coordination between the smart ship and the shore-side interactive control center, as well as between smart ships, to ensure real-time sharing of mission execution status, perception information and action information of other ships.

[0097] The collision avoidance decision modules of each intelligent ship make real-time decisions based on environmental perception data and the behaviors of other intelligent ships, ensuring that each intelligent ship can safely avoid obstacles and other intelligent ships and generate a feasible target point search path.

[0098] The intelligent ship maneuvering motion control module is responsible for converting the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, controlling each intelligent ship to smoothly reach each target point and completing the target search task.

[0099] The further technical solution is: the global path planning module integrates the deep deterministic policy gradient (MADDPG) algorithm and the artificial potential field (APF) method, and proposes the APF-MADDPG algorithm to complete the global planning path of each smart ship to each target point. The algorithm specifically includes: the gravity gain coefficient η in the APF algorithm is all and the repulsion gain coefficient η rep As the continuous action space in the strategy function of each smart ship. The perception module in each smart ship obtains the distance information to the surrounding static obstacles, the current position of each smart ship, the current bow angular velocity of each smart ship, the current heading angle of each smart ship and other information as the local state space information of each smart ship, that is, s t = {P s ,psi,d1,....,d i , P g The reward function is set by considering factors such as approaching the specified point as quickly as possible, avoiding surrounding obstacles, and the smoothness of the generated route.

[0100] Based on the setting of the three elements of the above reinforcement learning algorithm, the LSTM-MADDPG training network for multi-ship collaborative search routes in complex search scenarios is constructed through the APF-MADDPG algorithm. First, initialize the Actor and Critic networks with LSTM layers and their corresponding target networks. The LSTM layer is used to process state sequence information and generate actions or Q values ​​through the fully connected layer. At the same time, initialize the experience replay pool, the gravity and repulsion gain coefficients of the APF algorithm, and the initial state information of each smart ship in the multi-ship collaborative system; the MADDPG algorithm uses two sets of Actor-Critic structures, and each smart ship has a separate Actor network and Critic network. At each time step, the Actor network will be based on the current observation value of the smart ship Shipi. Calculation Action That is, the rudder angle of each ship. The output of the Critic network is the estimated Q value, which is used to evaluate After obtaining the execution action, each ship will get the state information of the ship at the next moment and the reward at the current moment. Then, by judging whether the ship has reached the target point or collided, it is known whether the ship is in the end state after executing the action; if it is not the end state, the obtained sampling data is Deposit into the experience recycling pool.

[0101] Randomly draw small batches of samples from the experience replay pool to update the Actor network and Criti c The network is updated, and the target network parameters are soft updated from the original network with a smaller step size. The method of soft updating of the target network is expressed as.

[0102] θ μ’ ←τθ μ +(1-τ)θ μ' θ Q' ←τθ Q +(1-τ)θ Q'

[0103] Among them, the step size τ is the soft update step size, which is usually between 0 and 1, which means that the target network will only make small adjustments to the parameters of the main network each time it is updated; θ μ is the main network parameter of Actor, θ μ′ is the target network parameter of the Actor; θ Q is the main network parameter of Critic, θ Q’ It is the target network parameter of Critic.

[0104] When calculating the CRI of collision unsafety between ships, the dynamic intelligent ship's own safety domain model and the safety distance are combined, and the closest encounter distance DCPA and the closest encounter time TCPA are selected as pricing indicators to calculate the collision unsafety degree.

[0105] According to the principle of fuzzy theory, the unsafe degree membership functions of the two indicators of closest approach distance DCPA and closest approach time TCPA are first determined, and then the two indicators are weightedly fused to obtain the unsafe degree of collision between ships.

[0106] The unsafe degree membership function of the DCPA index shows that the smaller the DCPA value, the greater the probability of collision between the two ships. DCPA for:

[0107]

[0108] Where n q is the number of intersections between the collision risk detection line and the ship area boundary; n sis the number of intersections between the collision risk detection line and the boundary of the expanded ship area; R1 is the distance from the intersection of the DCPA line and the boundary of the ship area to the target ship; R2 is the distance from the intersection of the DCPA extension line and the boundary of the expanded ship area to the target ship.

[0109] The DCPA value is calculated as:

[0110]

[0111] Where: is the coordinate of the specified smart ship in the own ship coordinate system; β is the relative heading in the own ship coordinate system.

[0112] The TCPA unsafe degree membership function indicates the urgency of the collision risk between the two ships. TCPA for:

[0113]

[0114] Where T0 is the TCPA value when other smart ships are located on the boundary of the expanded ship domain. When TCPA = T0, u TCPA =0.5; n s is the number of intersections between the collision risk detection line and the boundary of the expanded ship area.

[0115] The calculation method of TCPA value is:

[0116]

[0117] In the formula, Indicates the relative speed between the own ship and other smart ships at time t.

[0118] The membership of DCPA and TCPA is calculated using the average weighted fuzzy operator to obtain the ship collision insecurity level CRI, which is expressed as:

[0119] CRI=ω1.u DCPA +ω2.u TCPA

[0120] Where ω1 represents the fuzzy weight of DCPA on CRI, and ω2 represents the fuzzy weight of TCPA on CRI.

[0121] The ship collision avoidance module sets a regular symmetrical elliptical safety area for the ship and divides this safety area into 18 areas to improve the generalization ability of the reinforcement learning algorithm. After the area division is completed, the position coordinates of the ship and the distance D and relative position of the ship to other smart ships are obtained. When the distance between the two ships is less than the ship's elliptical safety distance d i Keep safe distance d from other smart shipsj The sum of <d i +d j , then according to the relative position of other smart ships and own ship Record the distance D in the corresponding area, i.e. K i =D; if no other smart ship invades the area of ​​the ship's elliptical safety area, it is recorded as K i =d i (i=1, 2, ..., 18);

[0122] According to the relative position of other smart ships and the own ship, the collision unsafe degree is recorded in the corresponding area, that is, C i = CRI; if there is no collision risk, it is recorded as C i =0;

[0123] The ship is in danger of collision with other smart ships at time t. It is expressed as:

[0124]

[0125] In the formula, K i is the distance between the i-th area of ​​the elliptical safety area and the designated smart ship; C i Indicates the collision unsafety degree between the specified smart ship and the own ship in the i-th area of ​​the elliptical safety area.

[0126] The dynamic ship safety domain model uses the quaternion of QSD as the coordinates of the four vertices of the ship's safety domain, expressed as: (R fore ,0),(-R aft ,0),(0,R starb ), (0, -R port ), the parameter of the ship's safety area is expressed as:

[0127]

[0128] In the formula, (R fore , R aft , R starb , R port ) represent the radius of the quaternary ship area in the front, rear, left and right directions respectively.

[0129] Then the parametric equation of the ellipse is:

[0130]

[0131] QSD quaternion is represented as:

[0132]

[0133] Among them, k ADis the gain factor for the approach distance in the ship turning circle, k DT is the gain coefficient about the ship turning diameter, which can be expressed as:

[0134]

[0135] Where v is the ship speed in knots and L is the ship length.

[0136] The safety domain of the smart ship after expansion is the result of scaling the safety domain of the designated ship by the coefficient f(t); where f(t) = D / D S , D S It indicates the minimum safe distance of the ship, and D indicates the distance between smart ships at different times.

[0137] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-vessel collaborative target search system for complex search scenarios, characterized in that: The system is installed on the ship and includes: information collection module, top-level task coordination allocation module, main control module, communication module, ship motion decision module and shore-side interactive control center; The information collection module is used to obtain the perception information of the mission area environment. Several smart ships are set up in the mission area. The perception information at least includes: obstacles in the environment, the location of smart ships and meteorological information, and the location, speed and heading information of the ship and other smart ships; The top-level task collaborative allocation module is used for perception information solution and target point search task allocation; perception information solution is to obtain perception information through the information acquisition module, and solve the inter-ship motion parameters, encounter situation and inter-ship collision risk; target point search task allocation is to allocate each target point to each smart ship through the Hungarian algorithm according to the distance, turning direction and obstacle density of each smart ship to the target point, and send the task allocation information to the main control module; The main control module is used to integrate the perception information and task allocation information collected by each smart ship, and jointly control each smart ship to complete the target point search task; Communication module, used for information transmission and coordination between smart ships and the shore-side interactive control center, as well as between smart ships, to ensure real-time sharing of mission execution status, perception information, and action information of other ships; Ship motion decision module, including smart ship collision avoidance module and smart ship maneuvering motion control module; each smart ship is equipped with a corresponding smart ship collision avoidance module, which is used to make real-time decisions based on environmental perception data and the behavior of other smart ships, to ensure that each smart ship can safely avoid obstacles and other smart ships, and generate a feasible target point search path; The ship maneuvering motion control module is used to convert the instructions generated by the intelligent ship collision avoidance module into actual maneuvering instructions, control each intelligent ship to smoothly reach each target point, and complete the target search task.

2. The multi-vessel cooperative target search system for complex search scenarios according to claim 1 is characterized in that: The objective function of the Hungarian algorithm in the top-level task collaborative allocation module is expressed as: The above formula Z(t) refers to the total cost after the target point is allocated at time t, and the cost matrix C(t) = [C ij (t)] n×m Refers to the cost that the i-th smart ship needs to spend to select the target point j at the t-th time; D ij (t) = 1 means that the jth target point at the tth time corresponds to the i-th smart ship. ij (t) = 0, which means that the jth target point has not been successfully matched with any smart ship; after calculation using the Hungarian algorithm, a distribution matrix D(t) = [D ij (T)] n×m .

3. The multi-vessel cooperative target search system for complex search scenarios according to claim 1 is characterized in that: The ship motion decision module integrates the deep deterministic policy gradient algorithm MADDPG and the artificial potential field method APF to form the APF-MADDPG algorithm, so that each smart ship can avoid obstacles and reach each target point; the formation steps of the APF-MADDPG algorithm are: Considering the continuous motion characteristics of ships in the multi-ship collaborative task search scenario, the continuous state space is constructed using the speed, heading, the relative position of the target point and itself, and the relative position of each ship to other ships or static obstacles, that is, in, represents the state space of smart ship i at time step t, represents the speed of smart ship i at time step t, represents the heading of smart ship i at time step t, represents the distance between smart ship i and the target point at time step t, represents the distance between smart ship i and the surrounding static obstacles at time step t, It represents the collision risk judgment between smart ship i and other smart ships at time step t; In order to better fit the actual navigation scene, the continuous action space is designed based on the change of the ship's rudder angle. represents the action space of smart ship i at time step t, represents the rudder angle of smart ship i at time step t; The reward function is designed based on whether the target point is close to the navigation safety performance. The distance between each ship and each target search point and the smooth curve generated as much as possible during the navigation process are used as navigation rewards in the multi-ship collaborative search process. When a collision occurs between smart ships or between smart ships and static obstacles, a negative reward value is given.

4. The multi-vessel cooperative target search system for complex search scenarios according to claim 3 is characterized in that: Also includes: The LSTM-MADDPG training network for multi-ship collaborative search routes in complex search scenarios is constructed through the APF-MADDPG algorithm; the specific steps are as follows: Initialize the Actor network and Critic network with LSTM layers and their corresponding target networks; the LSTM layer is used to process state sequence information and generate actions or Q values ​​through the fully connected layer; Initialize the experience replay pool, the gravitational and repulsive gain coefficients of the APF algorithm, and the initial state information of each smart ship in the multi-ship collaborative system; The MADDPG algorithm uses two sets of Actor-Critic structures, and each smart ship has a separate Actor network and Critic network; at each time step, the Actor network i Current observations Calculation Action That is, the rudder angle of each ship; the output of the Critic network is the estimated Q value, which is used to evaluate the good or bad; After obtaining the execution action, each ship will get the state information of the ship at the next moment and the reward at the current moment. Then, by judging whether the ship has reached the target point or collided, it is known whether the ship is in the end state after executing the action; if it is not the end state, the obtained sampling data is Deposit into the experience recycling pool; A small batch of samples is randomly drawn from the experience replay pool to update the Actor network and the Critic network. At the same time, the target network parameters are soft-updated from the original network with a small step size. The method of soft updating the target network parameters is expressed as: i μ' ←tth μ +(1-τ)θ μ' i Q' ←tth Q +(1-τ)θ Q' Among them, the step size τ is the soft update step size, which is usually between 0 and 1, which means that the target network will only make small adjustments to the parameters of the main network each time it is updated; θ μ is the main network parameter of Actor, θ μ' is the target network parameter of the Actor; θ Q is the main network parameter of Critic, θ Q' It is the target network parameter of Critic.

5. The multi-vessel cooperative target search system for complex search scenarios according to claim 3 is characterized in that: When calculating the CRI of collision unsafety between ships, the dynamic intelligent ship's own safety domain model and the safety distance are combined, and the closest encounter distance DCPA and the closest encounter time TCPA are selected as pricing indicators to calculate the collision unsafety degree.

6. The multi-vessel cooperative target search system for complex search scenarios according to claim 3 is characterized in that: The setting of collision reward in APF-MADDPG algorithm is based on the principle of fuzzy theory. First, the unsafe degree membership function of the two indicators of closest encounter distance DCPA and closest encounter time TCPA is determined, and then the two indicators are weighted and integrated to obtain the unsafe degree of collision between ships. The DCPA unsafe degree membership function is used to represent the collision probability of the two ships. The smaller the DCPA value, the greater the probability of the two ships colliding. DCPA for: Where n q is the number of intersections between the collision risk detection line and the ship area boundary; n s is the number of intersections between the collision risk detection line and the boundary of the expanded ship area; R1 is the distance from the intersection of the DCPA line and the boundary of the ship area to the ship; R2 is the distance from the intersection of the DCPA extension line and the boundary of the expanded ship area to the ship; The calculation method of DCPA value is expressed as: In the formula, is the coordinate of other smart ships in the coordinate system of own ship; β is the relative heading in the coordinate system of own ship; The TCPA unsafe degree membership function is used to indicate the urgency of the collision risk between two ships. TCPA for: Where T0 is the TCPA value when other smart ships are located on the boundary of the expanded ship domain. When TCPA = T0, u TCPA =0.5; n s is the number of intersections between the collision risk detection line and the boundary of the expanded ship area; The calculation method of TCPA value is expressed as: In the formula, Indicates the relative speed between the own ship and other smart ships at time t; The membership of DCPA and TCPA is calculated using the average weighted fuzzy operator to obtain the collision risk index CRI, which is expressed as: CRI=ω1.u DCPA +ω2.u TCPA Where ω1 represents the fuzzy weight of DCPA on CRI, and ω2 represents the fuzzy weight of TCPA on CRI.

7. The multi-vessel cooperative target search system for complex search scenarios according to claim 3 is characterized in that: The ship collision avoidance module sets a regular symmetrical elliptical safety area for the ship and divides this safety area into 18 areas to improve the generalization ability of the reinforcement learning algorithm. After the area division is completed, the position coordinates of the ship and the distance D and relative position of the ship to other smart ships are obtained. When the distance between the two ships is less than the ship's elliptical safety distance d i Keep safe distance d from other smart ships j The sum of <d i +d j , then according to the relative position of other smart ships and own ship Record the distance D in the corresponding area, i.e. K i =D; if no other smart ship invades the area of ​​the ship's elliptical safety area, it is recorded as K i =d i (i=1,2,…,18); According to the relative position of other smart ships and the own ship, the collision unsafe degree is recorded in the corresponding area, that is, C i = CRI; if there is no collision risk, it is recorded as C i =0; The ship is in danger of collision with other smart ships at time t. It is expressed as: In the formula, K i is the distance between the i-th area of ​​the elliptical safety area and the designated smart ship; C i Indicates the collision unsafety degree between the specified smart ship and the own ship in the i-th area of ​​the elliptical safety area.

8. The multi-vessel cooperative target search system for complex search scenarios according to claim 5 is characterized in that: The dynamic ship safety domain model uses the quaternion of QSD as the coordinates of the four vertices of the ship's safety domain, expressed as: (R fore ,0),(-R aft ,0),(0,R starb ),(0,-R port ), the parameter of the ship's safety area is expressed as: In the formula, (R fore , R aft , R starb , R port ) represent the radius of the quaternary ship domain in the front, rear, left and right directions respectively; Then the ellipse parametric equation is expressed as: QSD quaternion is represented as: Among them, k AD is the gain factor for the approach distance in the ship turning circle, k DT is the gain coefficient about the ship turning diameter, expressed as: Where v is the ship speed in knots and L is the ship length; The safety domain of the smart ship after expansion is the result of scaling the safety domain of the designated ship by the coefficient f(t); where f(t) = D / D S , D S It indicates the minimum safe distance of the ship, and D indicates the distance between smart ships at different times.

9. A multi-vessel collaborative target search method for complex search scenarios using any one of claims 1 to 6, characterized in that: The following steps are involved: Obtain environmental information of the search mission area, determine the location of the target point and the environmental information of the mission area; According to the Hungarian algorithm, the distance, turning direction and obstacle density of each smart ship to the target point are analyzed, and the target point is allocated; After the target point search task is assigned, the multi-ship collaborative navigation and collision avoidance paths of each intelligent ship are generated through the ship motion decision module. Each intelligent ship monitors the surrounding environment information in real time through the information collection module, makes dynamic local path adjustments, and replans the navigation route; Each smart ship performs the designated target point search task according to the route generated by the ship motion decision module. During the task execution, each ship perceives the environmental information in real time through the main control module, and then dynamically adjusts the target point allocation and the search route of each smart ship; Each ship will provide real-time feedback of the mission completion status to the shore-based interactive control center, and the multi-ship collaborative target search mission is completed.

Images