Unmanned aerial vehicle dynamic replacement control method and device for boat-aircraft cooperative interception and storage medium

Abstract

The invention belongs to the field of unmanned aerial vehicles, and particularly relates to an unmanned aerial vehicle dynamic replacement control method and device for boat-aircraft collaborative interception and a storage medium. The invention aims to solve the problem of poor tracking continuity of an existing unmanned aerial vehicle dynamic replacement control method for boat-aircraft cooperative interception. The method comprises the following steps: deploying an unmanned aerial vehicle and unmanned ship collaborative interception cluster; obtaining first real-time state data, second real-time state data and third real-time state data; inputting the first real-time state data, the second real-time state data and the third real-time state data into a trained reinforcement learning model for action generation to obtain an unmanned aerial vehicle replacement action; the dynamic replacement action comprises a return opportunity, a replacement scheme and task assignment; and controlling the unmanned aerial vehicle according to the replacement action of the unmanned aerial vehicle. The problem that an existing unmanned aerial vehicle dynamic replacement control method for boat-aircraft cooperative interception is poor in tracking continuity is solved.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicles (UAVs), and particularly relates to a dynamic takeover control method, device, and storage medium for UAVs used in ship-aircraft coordinated interception. Background Technology

[0002] With the escalating competition for maritime rights and the rapid development of unmanned warfare, unmanned surface vessels (USVs) and unmanned aerial vehicles (UAVs) have broad application prospects and significant research value in scenarios such as collaborative maritime defense and confrontation. UAVs possess wide-area detection capabilities and high maneuverability, making them suitable for large-scale area reconnaissance and patrol missions. USVs, on the other hand, offer advantages such as high payload capacity, long endurance, and precise surface operations, making them ideal platforms for physical interception missions. However, in efficient area interception missions, both USVs and UAVs, when deployed individually, have significant limitations in terms of area coverage, energy utilization, and interception efficiency. Therefore, constructing a three-dimensional, rapid-response area control system through USV-UAV collaboration has become an inevitable choice.

[0003] In existing technologies, most boat-unmanned surface vessel (USV) cooperative interception methods employ a simple, static patrol mode, designating a fixed patrol area for each USV. When a USV's battery is nearly depleted, it returns to base to recharge, and another USV takes over the patrol mission. While this method solves the basic handover problem, it lacks dynamic perception and optimized scheduling of the overall system's energy status, resulting in low energy utilization efficiency, poor system reliability, and resource waste. Therefore, a new boat-unmanned surface vessel (USV) cooperative interception method is needed. This method should not only achieve functional complementarity between USVs and USVs but also fundamentally address the endurance problem of USV swarms. Through intelligent task scheduling and energy management, it should improve the overall system's combat effectiveness and sustainability. Summary of the Invention

[0004] The purpose of this invention is to address the problem of poor tracking continuity in existing UAV dynamic succession control methods used for ship-aircraft coordinated interception. A UAV dynamic succession control method for ship-aircraft coordinated interception is provided, comprising:

[0005] Step 1: Deploy a coordinated interception swarm of drones and unmanned surface vessels;

[0006] Each unmanned surface vessel (USV) and its onboard drones are initialized and deployed at designated standby interception positions; the USVs function as both mobile charging platforms for drones and interception execution units.

[0007] Step 2: Obtain the first real-time status data, the second real-time status data, and the third real-time status data;

[0008] The first real-time status data includes: real-time status data of each UAV in the interception cluster; including the remaining battery power, real-time location, return distance to the mother ship, and flight path distance to the replacement UAV;

[0009] The second real-time status data includes: real-time status data of each unmanned surface vessel in the intercept cluster; including the location and mission status of the unmanned surface vessels;

[0010] The third real-time status data includes: the third real-time status data of the intercepted target in the interception area; including the target's position, heading, and speed;

[0011] Step 3: Input the first real-time state data, the second real-time state data, and the third real-time state data into the trained reinforcement learning model to generate actions and obtain the UAV takeover actions; the dynamic takeover actions include return timing, takeover plan, and task assignment.

[0012] Based on the drone takeover actions, control the drones; based on the dynamic takeover actions of the drones, control the drones to perform area patrol, target tracking, or return-to-base charging tasks, so as to achieve seamless takeover of the drone swarm and full-time coverage of the interception area.

[0013] The training process of the trained reinforcement learning model includes:

[0014] S1: Obtain the training set; the training set includes historical state data of UAVs, state data of unmanned surface vessels, and state data of intercepted targets, and generates enhanced data through a simulation environment;

[0015] S2: Construct a reinforcement learning model based on the PPO algorithm;

[0016] S3: Train the reinforcement learning model based on the PPO algorithm using the training set to obtain the trained reinforcement learning model;

[0017] The specific process of constructing the reinforcement learning model based on the PPO algorithm in S2 is as follows:

[0018] S2.1: Environment initialization; the specific process is as follows:

[0019] Discretize the interception area as A grid of squares of equal size;

[0020] The total number of grid cells in the region is expressed as: ; and All are positive integers;

[0021] S2.2: Define the state space and action space of the reinforcement learning model based on the PPO algorithm. The state space includes the distance between the UAV and the unmanned surface vessel, the distance between the UAV and other UAVs, the heading, the battery level, and the mission type (charging, patrolling, tracking, and takeover).

[0022] The action space includes the timing of the drone's return to home, the return location, and the selection of a replacement drone;

[0023] S2.3: Define the reward function of the reinforcement learning model based on the PPO algorithm to obtain the reinforcement learning model based on the PPO algorithm; the reward function is guided by the global system performance and integrates reward factors such as regional detection coverage, multi-machine collaboration success rate, battery health status and return efficiency.

[0024] A computer storage medium, characterized in that the storage medium stores at least one instruction, which is loaded and executed by a processor to implement the aforementioned dynamic takeover control method for UAVs used in submarine-aircraft cooperative interception.

[0025] A dynamic takeover control device for UAVs used in submarine-aircraft coordinated interception is characterized in that the device includes a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the dynamic takeover control method for UAVs used in submarine-aircraft coordinated interception.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention proposes a method for submarine-aircraft cooperative interception based on dynamic UAV replacement.

[0028] This invention utilizes the PPO algorithm to intelligently make decisions and schedule tasks for the entire system by considering information such as the real-time battery level of drones performing patrol or tracking missions, their distance from unmanned surface vessels (USVs) and other drones, thus realizing a dynamic drone replacement mechanism. This not only ensures that drones in the air are promptly replaced before their battery runs out, avoiding mission failure due to power depletion, but also improves the energy utilization rate of the entire drone swarm. Compared with traditional fixed-area patrols, this invention can achieve seamless patrol and tracking surveillance over larger areas for longer periods with less energy loss, ensuring that a greater number of enemy targets are detected and continuously tracked within the same energy budget, thus overcoming the limitation of drone endurance on the overall system's sustainability.

[0029] Meanwhile, this invention fully leverages the respective advantages of UAVs and unmanned surface vessels (USVs), achieving interception of enemy targets through refined task decomposition and collaborative cooperation. UAVs, with their wide-area detection range and high maneuverability, undertake aerial patrols and target tracking tasks, while USVs, with their strong payload capacity and precise surface operations, focus on providing charging support for UAVs and accurately intercepting enemy targets. This collaborative model enables the entire system to launch rapid and precise interception operations against multiple dispersed targets, far exceeding the number of friendly USVs. Simultaneously, under the continuous guidance of UAVs, USVs effectively overcome their limited detection range, achieving precise seizure of suspicious targets. This deeply integrated collaborative model transforms the system from a simple combination of independent platforms into an organically unified operational whole, significantly improving the ability to intercept multiple targets and the reliability of area control.

[0030] This invention combines high efficiency, continuity, adaptability, and robustness, and can effectively deal with multiple batches of suspicious targets with maneuvering trajectories in complex sea conditions, providing a better solution for submarine-aircraft coordinated interception. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the initial startup phase of the system of this invention;

[0032] Figure 2 This is a flowchart of the PPO algorithm of the present invention;

[0033] Figure 3 This is a design diagram of the state space, observation space, and reward function of the UAV during the handover process of the present invention;

[0034] Figure 4 This is a schematic diagram of the unmanned surface vessel of the present invention arriving at the designated standby interception position;

[0035] Figure 5 This is a schematic diagram of the drone of the present invention patrolling within an area;

[0036] Figure 6 This is a schematic diagram illustrating the patrol and handover of the drone within the area according to the present invention;

[0037] Figure 7 This is a schematic diagram illustrating how the UAV of the present invention detects enemy targets and transmits enemy information to the system;

[0038] Figure 8 This is a schematic diagram of the unmanned surface vessel of the present invention heading to the predicted location after receiving enemy information;

[0039] Figure 9 This is a schematic diagram of a drone patrolling near the unmanned surface vessel about to perform a locking mission according to the present invention;

[0040] Figure 10This is a schematic diagram of the UAV tracking enemy targets according to the present invention;

[0041] Figure 11 This is a schematic diagram of the unmanned surface vessel of the present invention starting to lock onto an enemy target;

[0042] Figure 12 This is a schematic diagram illustrating the successful locking and destruction of an enemy target according to the present invention. Detailed Implementation

[0043] Specific implementation method one: Combining Figures 1-12 This invention is described;

[0044] Step 1: Deploy a coordinated interception swarm of drones and unmanned surface vessels;

[0045] Each unmanned surface vessel (USV) and its onboard drones are initialized and deployed at designated standby interception positions; the USVs function as both mobile charging platforms for drones and interception execution units.

[0046] Step 2: Obtain the first real-time status data, the second real-time status data, and the third real-time status data;

[0047] The first real-time status data includes: real-time status data of each UAV in the interception cluster; including the remaining battery power, real-time location, return distance to the mother ship, and flight path distance to the replacement UAV;

[0048] The second real-time status data includes: real-time status data of each unmanned surface vessel in the intercept cluster; including the location and mission status of the unmanned surface vessels;

[0049] The third real-time status data includes: the third real-time status data of the intercepted target in the interception area; including the target's position, heading, and speed;

[0050] Step 3: Input the first real-time state data, the second real-time state data, and the third real-time state data into the trained reinforcement learning model to generate actions and obtain the UAV takeover actions; the dynamic takeover actions include return timing, takeover plan, and task assignment.

[0051] Based on the drone takeover actions, control the drones; based on the dynamic takeover actions of the drones, control the drones to perform area patrol, target tracking, or return-to-base charging tasks, so as to achieve seamless takeover of the drone swarm and full-time coverage of the interception area.

[0052] Specific Implementation Method Two: The difference between this implementation method and Specific Implementation Method One is that:

[0053] The training process of the trained reinforcement learning model includes:

[0054] S1: Obtain the training set; the training set includes historical state data of UAVs, state data of unmanned surface vessels, and state data of intercepted targets, and generates enhanced data through a simulation environment;

[0055] S2: Construct a reinforcement learning model based on the PPO algorithm;

[0056] S3: Train the reinforcement learning model based on the PPO algorithm using the training set to obtain the trained reinforcement learning model.

[0057] The other steps and parameters are the same as in Specific Implementation Method 1.

[0058] Specific Implementation Method Three: The difference between this implementation method and Specific Implementation Methods One and Two is that:

[0059] The specific process of constructing the reinforcement learning model based on the PPO algorithm in S2 is as follows:

[0060] S2.1: Environment initialization; the specific process is as follows:

[0061] Discretize the interception area as A grid of squares of equal size;

[0062] The total number of grid cells in the region is expressed as: ; and All are positive integers;

[0063] S2.2: Define the state space and action space of the reinforcement learning model based on the PPO algorithm. The state space includes the distance between the UAV and the unmanned surface vessel, the distance between the UAV and other UAVs, the heading, the battery level, and the mission type (charging, patrolling, tracking, and takeover).

[0064] The action space includes the timing of the drone's return to home, the return location, and the selection of a replacement drone;

[0065] S2.3: Define the reward function of the reinforcement learning model based on the PPO algorithm to obtain the reinforcement learning model based on the PPO algorithm; the reward function is guided by the global system performance and integrates reward factors such as regional detection coverage, multi-machine collaboration success rate, battery health status and return efficiency.

[0066] The other steps and parameters are the same as in one of the specific implementation methods one or two.

[0067] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that:

[0068] 4. The specific process of defining the state space and action space of the reinforcement learning model based on the PPO algorithm in S2.2 is as follows:

[0069] S2.2.1: Define the state space of the reinforcement learning model based on the PPO algorithm;

[0070] S2.2.2: Define the action space of the reinforcement learning model based on the PPO algorithm;

[0071] S2.2.3: Define the state transition function of the reinforcement learning model based on the PPO algorithm;

[0072] S2.2.4: Define the state-value function of the reinforcement learning model based on the PPO algorithm;

[0073] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0074] Specific Implementation Method Five: The difference between this implementation method and Specific Implementation Methods One to Four is that:

[0075] S2.2.1 defines the state space of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows:

[0076] Define the state at time step t. This can be expressed as a formula:

[0077]

[0078] In the formula, This represents the distance between the drone and the unmanned surface vessel at time step t. This represents the distance between the drone and other drones at time step t. This represents the speed of the drone at time step t. Indicates the drone's heading at time step t. This indicates the drone's battery level at time step t. Indicates the type of task performed by the UAV at time step t;

[0079] S2.2.2 defines the action space of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows:

[0080] Define the set of actions of the UAV at time step t. This can be expressed as a formula:

[0081]

[0082] In the formula, Represents the Cartesian product of the set of all drone actions within the environment. Represents the Cartesian product. This represents the first set of drone actions at time step t; This represents the second set of drone actions at time step t. This represents the set of actions of the Nth drone at time step t; N represents the number of drones in the drone and unmanned surface vessel cooperative interception swarm. This represents the set of actions performed by the i-th UAV at time step t; i = 1, 2, ..., N;

[0083] S2.2.3 defines the state transition function of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows:

[0084] Define the state transition function at time step t, expressed by the formula:

[0085]

[0086] In the formula, This represents the state transition function at time t. Represents the Cartesian product. This represents the mapping relationship of the state transition function. The output value of this function is between 0 and 1 (meaning needs to be added). It represents the probability of the environment transitioning to the next state in the current state.

[0087] This indicates the state at time step t. This indicates the state at time step t+1; This represents the drone's actions at time t;

[0088] Status includes: distance between the drone and the unmanned surface vessel, distance between the drone and other drones, heading, battery level, and mission type (charging, patrolling, tracking, takeover).

[0089] The state-value function of the reinforcement learning model based on the PPO algorithm is defined in S2.2.4, and the specific process is as follows:

[0090] Define the state-value function of the i-th UAV as follows:

[0091]

[0092] In the formula, E represents the expectation. The discount factor is set to 0.75. To represent time steps, The instantaneous reward for the i-th drone at time step t. The final learned drone swarm strategy; This indicates the state of the environment at time step t. express Receive status Then, press The probability distribution of random action selection S represents the state. This indicates the initial state; other steps and parameters are the same as in one of the specific implementation methods one to four.

[0093] Specific Implementation Method Six: The difference between this implementation method and Specific Implementation Methods One through Five is that:

[0094] The reward function of the PPO algorithm is defined in S2.3, and the specific process is as follows:

[0095] The reward function for the UAV at time step t is defined as follows:

[0096]

[0097] In the formula, This represents the instant reward for the first drone at time step t. Represents the Cartesian product. This represents the instantaneous reward for the Nth drone at time step t. The instantaneous reward for the i-th drone at time step t is expressed by the formula:

[0098]

[0099] In the formula, This represents the detection reward for the i-th drone. This represents the cooperative reward for the i-th drone. This represents the battery reward for the i-th drone. This represents the return reward for the i-th drone;

[0100] The other steps and parameters are the same as those in any of the specific implementation methods one to five.

[0101] Specific Implementation Method Seven: The difference between this implementation method and Specific Implementation Methods One through Six is ​​that:

[0102] The detection reward for the i-th UAV Expressed as a formula:

[0103]

[0104] In the formula, This represents the cumulative number of grid cells detected by the i-th drone at time step t. Indicates the total number of grid cells in the detection space; This indicates that the i-th drone performed an action at time step t. The state that was later transitioned to This indicates that the i-th drone has not yet performed any actions at the start of time step t. The previous state, This represents an adjustment parameter, set to 0.2 here, used to control the penalty for repeatedly visiting a probed grid within time t. Represents the probe function, based on the new state. The detection function value is calculated based on whether the UAV has visited a previously detected grid. If it has, the value is 1; otherwise, it is 0.

[0105] Therefore, the exploratory reward is the proportion of new grids visited by the agent under the current action to the total number of grids in the exploration area;

[0106] The cooperative reward for the i-th drone Expressed as a formula:

[0107]

[0108] In the formula, Indicates the weighting coefficient. This represents the time interval for the handover of the i-th drone mission. The shorter the handover time, the higher the reward. Represents the time sensitivity coefficient. This represents the weighting coefficient for controlling the reward of coordinated movement. Indicates the maximum effective overlap time. Represents the distance optimization factor. This indicates the time of overlap in the tracking of the two drones. This indicates the type of task the drone is performing, including tasks such as drone tracking, patrolling, takeoff, charging, returning to base, and handover. To track mission indicators and ensure that UAVs continuously track unmanned surface vessels after detection, a cooperative reward function is implemented to ensure continuous and coordinated patrolling and tracking of enemy targets among UAVs.

[0109] The power reward for the i-th drone Expressed as a formula:

[0110]

[0111] In the formula, This indicates a low battery penalty factor, with a value of 0.9. This represents the low battery indicator function. Indicates the drone's current battery level. This represents the dynamic power adjustment function. This indicates the emergency power penalty coefficient, with a value of 1.6. The emergency power indication function, the dynamic power adjustment function Expressed as a formula:

[0112]

[0113] In the formula, This represents the basic safe power level, defined as 15%. The distance coefficient is expressed as 0.1. d represents the speed of the drone, and d represents the distance from the drone to the nearest available unmanned surface vessel.

[0114] The return reward for the i-th drone Expressed as a formula:

[0115]

[0116] In the formula, This represents the return-to-home weighting coefficient. This indicates the optimal return-to-home indicator, ensuring that the returning drone chooses the nearest unoccupied unmanned surface vessel for charging. This indicates the distance the drone chose to return to. Indicates the maximum permissible return distance. A reward is given for a successful landing;

[0117] The other steps and parameters are the same as those in any of the specific implementation methods one to six.

[0118] Specific Implementation Method Eight: The difference between this implementation method and Specific Implementation Methods One through Seven is that:

[0119] The specific process of training the reinforcement learning model based on the PPO algorithm according to the training set in S3 to obtain the trained reinforcement learning model is as follows:

[0120] S3.1: First, initialize the simulation environment to obtain the initial state of the environment. ;

[0121] The simulation environment is a virtual system whose core function is to simulate the physical laws and task scenarios of the real world, providing intelligent agents (drones) with state feedback, action execution interfaces and reward signals, thereby supporting the training process of reinforcement learning algorithms.

[0122] The preferred simulation environment is based on the Gymnasium framework in Python;

[0123] S3.2: Set the environment state at time step t. The input is fed into the Actor policy network in the PPO algorithm.

[0124] The Actor policy network outputs the set of drone actions at time step t under the current policy. The set of actions of the drone at time step t Input into the simulation environment;

[0125] S3.3: The simulation environment receives and executes... Then, update the position, battery level, and cumulative number of grid cells detected by each drone, so that the environment changes from state t at time step. Transition to the state at time step t+1 ;

[0126] S3.4: Calculate the instantaneous reward at time step t ;

[0127] S3.5: Set the environment state at time step t. The set of actions of the UAV at time step t Instant rewards for time steps t and the state at time step t+1 The data is stored in the experience buffer.

[0128] Once the amount of data in the experience buffer reaches the preset batch size, the Actor and Critic network parameters of the PPO algorithm are updated based on the data in the experience buffer. Then, the buffer is cleared and steps S3.2-3.5 are repeated.

[0129] Training stops when the parameters of the Actor and Critic networks converge, resulting in a trained reinforcement learning model, thereby enabling dynamic failover control of the UAV; other steps and parameters are the same as in one of the specific implementation methods one to seven.

[0130] Specific Implementation Method Nine: This implementation method is a computer storage medium that stores at least one instruction. The at least one instruction is loaded and executed by a processor to implement the aforementioned dynamic takeover control method for UAVs used in submarine-aircraft cooperative interception.

[0131] It should be understood that the instructions include computer program products, software, or computerized methods corresponding to any method described in this invention; the instructions can be used to program computer systems or other electronic devices. Computer storage media may include readable media on which instructions are stored, and may include, but are not limited to, magnetic storage media, optical storage media; magneto-optical storage media include read-only memory (ROM), random access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), and flash memory layers, or other types of media suitable for storing electronic instructions.

[0132] Specific Implementation Method 10: This implementation method is a dynamic takeover control device for UAVs used in boat-aircraft coordinated interception. The device includes a processor and a memory. It should be understood that it includes any device with a processor and a memory described in this invention. The device may also include other units and modules that perform display, interaction, processing, control and other functions through signals or instructions.

[0133] The memory stores at least one instruction, which is loaded and executed by the processor to implement the aforementioned method for dynamic takeover control of unmanned aerial vehicles (UAVs) for coordinated interception by submarines and aircraft.

[0134] Combination Figures 7 to 8 This section explains the key steps in the UAV-UAV cooperative interception method, from target detection to target discovery and the generation of interception commands. The solid green line represents the UAV's trajectory, the solid blue line represents the UAV's trajectory, the dashed blue line represents the straight-line distance from the UAV to the enemy target, and the dashed green line represents the straight-line distance from the UAV to the predicted point. When a UAV on patrol identifies a suspicious target, it captures real-time status information including its current position, heading, and instantaneous speed and uploads it to the central control system. The system predicts the enemy's trajectory over a future period based on the target's motion and calculates one or more predicted interception points. Subsequently, the system assigns the closest UAV with no other missions from all standby UAVs to perform the interception mission. This UAV randomly departs from its standby position, travels at maximum speed towards the predicted target location, and is ready to initiate the interception mission at any time.

[0135] Once the unmanned surface vessel (USV) carrying out the interception mission approaches the enemy target, a drone is assigned to conduct close-range, precise positioning and continuous tracking of the enemy target to guide the USV in the final interception operation.

[0136] Combination Figures 9 to 10 The process is as follows: When the assigned interceptor UAV approaches a certain range near the predicted target location, the central control system determines that it has entered the near-field interception phase and immediately dispatches nearby unattended UAVs with sufficient battery power to provide support. Once the dispatched UAV arrives at the designated search area centered on the predicted location, it performs a circling maneuver to accurately detect the enemy target. Once the UAV accurately identifies the enemy target, its role switches from search mode to tracking mode. To ensure uninterrupted target tracking during this critical phase, the system also employs the aforementioned PPO algorithm for dynamic task scheduling. This algorithm intelligently decides the timing and target of tracking task replacement by comprehensively evaluating factors such as the remaining battery power of the tracking UAV, its relative position to the target, and the distance to the charging UAV, thus enabling multiple UAVs to relay-track the same target, forming a continuous monitoring and guidance link. Throughout the process, the interceptor UAV continuously corrects its course based on real-time data transmitted from the tracking UAV, heading directly towards the latest measured target location until it enters the effective range of its own interception equipment, creating the necessary conditions for final interception.

[0137] Combination Figures 11 to 12The following explanation is provided. The green circle represents an enemy target being locked onto. Guided by real-time, high-precision target data provided by the tracking drone, the interceptor UAV eventually enters the effective range of its own interception system. It then executes the final interception procedure against the enemy target. After confirming successful target destruction, the interceptor UAV sends a mission completion signal to the central control system and autonomously navigates back to its initially designated standby interception position. Simultaneously, the drones previously involved in the tracking mission are rescheduled based on their remaining battery power and the overall system mission status, including returning to the mothership for necessary charging and being directly assigned to patrol missions. The end of this single interception cycle signifies a complete system reset. The central control system automatically switches the mission logic and re-enters the continuous area patrol mode described in step two, thus forming a complete "patrol-detection-tracking-interception-reset" operational loop. This design enables the entire collaborative defense system to operate intelligently and efficiently in a cyclical manner until it receives external instructions to terminate the entire operational mission, ultimately achieving long-term, autonomous, and highly reliable control of the sea area.

[0138] The above description is merely of preferred embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention, and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A dynamic takeover control method for unmanned aerial vehicles (UAVs) used in submarine-aircraft coordinated interception, characterized in that, include: Step 1: Deploy a coordinated interception swarm of drones and unmanned surface vessels; Step 2: Obtain the first real-time status data, the second real-time status data, and the third real-time status data; The first real-time status data includes: real-time status data of each drone in the intercept cluster; The second real-time status data includes: real-time status data of each unmanned surface vessel in the interception cluster; The third real-time status data includes: real-time status data of the intercepted targets in the interception area; Step 3: Input the first real-time state data, the second real-time state data, and the third real-time state data into the trained reinforcement learning model to generate actions and obtain the drone takeover actions. Control the drone according to the drone takeover action.

2. The method for dynamic takeover control of unmanned aerial vehicles (UAVs) for coordinated interception by submarines and aircraft according to claim 1, characterized in that, The training process of the trained reinforcement learning model includes: S1: Obtain the training set; S2: Construct a reinforcement learning model based on the PPO algorithm; S3: Train the reinforcement learning model based on the PPO algorithm using the training set to obtain the trained reinforcement learning model.

3. The method for dynamic takeover control of unmanned aerial vehicles (UAVs) for coordinated interception by submarines and aircraft according to claim 2, characterized in that, The specific process of constructing the reinforcement learning model based on the PPO algorithm in S2 is as follows: S2.1: Environment initialization; the specific process is as follows: Discretize the interception area as A grid of squares of equal size; The total number of grid cells in the region is expressed as: ; and All are positive integers; S2.2: Define the state space and action space of the reinforcement learning model based on the PPO algorithm; S2.3: Define the reward function of the reinforcement learning model based on the PPO algorithm to obtain the reinforcement learning model based on the PPO algorithm.

4. The method for dynamic takeover control of unmanned aerial vehicles (UAVs) for coordinated interception by submarines and aircraft according to claim 3, characterized in that, The specific process of defining the state space and action space of the reinforcement learning model based on the PPO algorithm in S2.2 is as follows: S2.2.1: Define the state space of the reinforcement learning model based on the PPO algorithm; S2.2.2: Define the action space of the reinforcement learning model based on the PPO algorithm; S2.2.3: Define the state transition function of the reinforcement learning model based on the PPO algorithm; S2.2.4: Define the state-value function of the reinforcement learning model based on the PPO algorithm.

5. A dynamic takeover control method for UAVs used in submarine-aircraft coordinated interception according to claim 4, characterized in that, S2.2.1 defines the state space of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows: Define the state at time step t. This can be expressed as a formula: In the formula, This represents the distance between the drone and the unmanned surface vessel at time step t. This represents the distance between the drone and other drones at time step t. This represents the speed of the drone at time step t. Indicates the drone's heading at time step t. This indicates the drone's battery level at time step t. Indicates the type of task performed by the UAV at time step t; S2.2.2 defines the action space of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows: Define the set of actions of the UAV at time step t. This can be expressed as a formula: In the formula, Represents the Cartesian product. This represents the first set of drone actions at time step t; This represents the second set of drone actions at time step t. This represents the set of actions of the Nth drone at time step t; N represents the number of drones. This represents the set of actions performed by the i-th UAV at time step t; i = 1, 2, ..., N; S2.2.3 defines the state transition function of the reinforcement learning model based on the PPO algorithm, and the specific process is as follows: Define the state transition function at time step t, expressed by the formula: In the formula, This represents the state transition function at time t. Represents the Cartesian product. This represents the mapping relationship of the state transition function; This indicates the state at time step t. This indicates the state at time step t+1; This represents the drone's actions at time t; The state-value function of the reinforcement learning model based on the PPO algorithm is defined in S2.2.4, and the specific process is as follows: Define the state-value function of the i-th UAV as follows: In the formula, E represents the expectation. Discount factor; To represent time steps, The instantaneous reward for the i-th drone at time step t. For drone swarm strategy; express Receive status Then, press The probability distribution of random action selection , S represents the state of the environment at time step t. This represents the initial state.

6. A dynamic takeover control method for unmanned aerial vehicles (UAVs) used in submarine-aircraft coordinated interception according to claim 5, characterized in that, The reward function of the PPO algorithm is defined in S2.3, and the specific process is as follows: The reward function for the UAV at time step t is defined as follows: In the formula, This represents the instant reward for the first drone at time step t. Represents the Cartesian product. This represents the instantaneous reward for the Nth drone at time step t. The instantaneous reward for the i-th drone at time step t is expressed by the formula: In the formula, This represents the detection reward for the i-th drone. This represents the cooperative reward for the i-th drone. This represents the battery reward for the i-th drone. This represents the return reward for the i-th drone.

7. A dynamic takeover control method for UAVs used in submarine-aircraft coordinated interception according to claim 6, characterized in that, The detection reward for the i-th UAV Expressed as a formula: In the formula, This represents the cumulative number of grid cells detected by the i-th drone at time step t. Indicates the total number of grid cells in the detection space; This indicates that the i-th drone performed an action at time step t. The state that was later transitioned to This indicates that the i-th drone has not yet performed any actions at the start of time step t. The previous state, Indicates the adjustment parameter. Indicates the probe function, The cooperative reward for the i-th drone Expressed as a formula: In the formula, Indicates the weighting coefficient. This represents the time interval for the handover of the i-th drone mission. Represents the time sensitivity coefficient. This represents the weighting coefficient that controls the reward for coordinated movement. Indicates the maximum effective overlap time. Represents the distance optimization factor. This indicates the time of overlap in the tracking of the two drones. Indicates the task type of the drone's actions. To track task indicators, The power reward for the i-th drone Expressed as a formula: In the formula, This indicates the low battery penalty coefficient. This represents the low battery indicator function. Indicates the drone's current battery level. This represents the dynamic power adjustment function. This indicates the emergency power penalty coefficient. The emergency power indication function, the dynamic power adjustment function Expressed as a formula: In the formula, Indicates the basic safe power level. Represented as distance coefficient, d represents the speed of the drone, and d represents the distance from the drone to the nearest available unmanned surface vessel. The return reward for the i-th drone Expressed as a formula: In the formula, This represents the return-to-home weighting coefficient. Indicates the optimal return-to-home indicator. This indicates the distance the drone chose to return to. Indicates the maximum permissible return distance. This indicates a reward for a successful landing.

8. A dynamic takeover control method for unmanned aerial vehicles (UAVs) used in submarine-aircraft coordinated interception according to claim 7, characterized in that, The specific process of training the reinforcement learning model based on the PPO algorithm according to the training set in S3 to obtain the trained reinforcement learning model is as follows: S3.1: First, initialize the simulation environment to obtain the initial state of the environment. ; S3.2: Set the environment state at time step t. The input is fed into the Actor policy network in the PPO algorithm. The Actor policy network outputs the set of drone actions at time step t. The set of actions of the drone at time step t Input into the simulation environment; S3.3: The simulation environment receives and executes... Then, the environment changes from state at time step t. Transition to the state at time step t+1 ; S3.4: Calculate the instantaneous reward at time step t ; S3.5: Set the environment state at time step t. The set of actions of the UAV at time step t Instant rewards for time steps t and the state at time step t+1 The data is stored in the experience buffer. Once the amount of data in the experience buffer reaches the preset batch size, the Actor and Critic network parameters of the PPO algorithm are updated based on the data in the experience buffer. Then, the buffer is cleared and steps S3.2-3.5 are repeated. Training stops when the parameters of the Actor and Critic networks converge, resulting in a trained reinforcement learning model.

9. A computer storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement a dynamic takeover control method for UAVs used in submarine-aircraft cooperative interception as described in any one of claims 1 to 8.

10. A dynamic takeover control device for unmanned aerial vehicles (UAVs) used in submarine-aircraft coordinated interception, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement a dynamic takeover control method for UAVs used in submarine-aircraft cooperative interception as described in any one of claims 1 to 8.

Images