A method and system for space-air-ground cooperative tracking of moving targets suitable for civil maritime search and rescue

Abstract

The application provides a moving target space-air coordination tracking method and system suitable for civil maritime search and rescue, and the method comprises the following steps: acquiring an accident occurrence site and environmental parameters; constructing a drifting model of a wrecked ship and a fallen person based on the accident occurrence site and the environmental parameters, and acquiring a predicted position and an instantaneous speed of a to-be-tracked target updated at a time step; screening applicable unmanned aerial vehicles and satellites according to the environmental parameters, and acquiring parameter data of the applicable unmanned aerial vehicles and the satellites, and obtaining a satellite subsatellite point position updated over time based on the satellite parameter data; loading the applicable unmanned aerial vehicles and the satellites, the parameter data of the unmanned aerial vehicles and the satellites, and the satellite subsatellite point position updated over time into a trained space-air coordination reinforcement learning model, and inputting the predicted position and the instantaneous speed of the to-be-tracked target updated at the time step, and outputting a coordination tracking strategy.

Description

Technical Field

[0001] This invention belongs to the field of geographic information system technology, specifically relating to a method and system for air-space collaborative tracking of moving targets applicable to civilian maritime search and rescue. Background Technology

[0002] Space-air coordinated tracking refers to the simultaneous use of drones and remote sensing satellites to track targets. Drones and remote sensing satellites are significantly complementary in data acquisition: satellite remote sensing is unaffected by weather, but its spatial resolution is relatively low, providing only large-scale global scene tracking; drones are susceptible to weather conditions, but can acquire high-resolution ground data, enabling precise tracking of people who have fallen into water. When space-air coordinated technology is used to track targets awaiting rescue, search and rescue personnel can simultaneously obtain global and local detailed information about the accident scene, which is of significant value and practical importance for tracking and monitoring targets.

[0003] Current research on space-air coordinated tracking methods for moving targets at sea focuses on monitoring large military targets. Tracking large military targets at sea is characterized by their large size, long distance from shore, and long mission duration. Therefore, research on space-air coordinated tracking of moving targets at sea mainly concentrates on the allocation of large tasks or targets between UAVs and remote sensing satellites, and the mission planning of each UAV and remote sensing satellite. The multi-factor task allocation and planning are primarily accomplished using heuristic algorithms and other algorithms. These space-air coordinated tracking methods are suitable for large-scale missions with long effective periods, but they cannot meet the timeliness and flexibility requirements of short-term near-shore civilian maritime search and rescue missions. Therefore, improving the timeliness and flexibility of space-air coordinated tracking methods has become an urgent problem to be solved.

[0004] Currently, there are two main methods for tracking moving targets using aerospace coordination:

[0005] (1) Use hierarchical verification collaborative planning algorithm to realize the allocation of air-space collaborative tasks. This method mainly realizes the allocation of collaborative tasks between satellites and UAVs by designing hierarchical verification collaborative planning algorithm, group processing and profit increment judgment. After completing the collaborative task allocation, a constraint model is established and a heuristic algorithm is used to solve the observation task sequence of UAVs and satellites respectively. Finally, when UAVs perform tasks, a clustering algorithm is used to further refine the tasks allocated to UAVs by clustering the targets, and the UAV path is planned by the method with the shortest energy consumption (e.g., artificial potential field method). In the collaborative scheme formed by this method, each UAV observes several targets in a loop along a certain optimized route, which is difficult to meet the needs of long-term tracking and monitoring of a target in maritime search and rescue. (2) Establish a multi-stage collaborative task planning model of heterogeneous multi-agent system (MAS). This method constructs a heterogeneous MAS model, models air-space resources and tasks as agents respectively, realizes task planning through negotiation between agents, introduces a market model when establishing the MAS model, regards task agents as consumers and resource agents as producers, and adjusts resource allocation through price levers. When optimizing local task planning for both satellite agents and drone agents, genetic algorithms and particle swarm optimization algorithms are employed to improve the speed and quality of task planning for each agent. However, in maritime search and rescue scenarios, the distribution range of the tracked targets is small. Compared to this method, the experimental scenario can only be considered as a single target. Using a multi-target competition model for task allocation could lead to a waste of computational resources. Furthermore, this method only performs collaborative task planning and does not involve detailed solutions such as satellite payload selection and drone flight path planning, lacking direct feasibility in maritime search and rescue missions. Summary of the Invention

[0006] To overcome the shortcomings of existing air-space cooperative tracking methods, which are applicable to large-scale missions and have long effective periods, failing to meet the timeliness and flexibility requirements of short-term near-shore civilian maritime search and rescue missions, this invention provides an air-space cooperative tracking method and system for moving targets suitable for civilian maritime search and rescue. Utilizing the complementary characteristics of UAV and remote sensing satellite observation capabilities, air-space cooperative technology is used to increase the amount of information available for tracking. A PPO-based reinforcement learning model is constructed, a multi-factor reward function is set, and an objective function incorporating pruning parameters is built to obtain an implementable cooperative tracking strategy. This enhances the timeliness and flexibility of the air-space cooperative tracking method, improving tracking efficiency and cooperative capabilities. The method is simple and efficient, effectively solving the problem of current air-space cooperative technologies being applicable to large-scale missions and having long effective periods, making it suitable for various maritime search and rescue mission scenarios.

[0007] According to one aspect of the present invention, a method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue is provided, characterized in that it includes:

[0008] Obtain the location of the accident and environmental parameters;

[0009] Based on the accident location and environmental parameters, a drift model of the wrecked vessel and the person who fell into the water is constructed to obtain the predicted position and instantaneous velocity of the target to be tracked, updated step by step over time.

[0010] Applicable drones and satellites are selected based on environmental parameters, and parameter data of the applicable drones and satellites are obtained. The satellite sub-satellite position updated over time is obtained based on the satellite parameter data.

[0011] The applicable UAVs and satellites and their parameter data, as well as the satellite nadir position updated over time, are loaded into the trained aerospace cooperative reinforcement learning model. The predicted position and instantaneous velocity of the target to be tracked, updated over time steps, are also input, and the cooperative tracking strategy is output.

[0012] The training process of the aforementioned space-air collaborative reinforcement learning model is as follows:

[0013] A PPO-based aerospace cooperative reinforcement learning model is constructed: the policy network, value network, and objective function of PPO are adopted. In the policy network, the agent and its state and actions are updated with UAV and satellite and their parameter data. The environmental state is updated with the predicted position and instantaneous velocity of the target to be tracked updated by time step. The multi-factor reward function is used as the fitting benchmark for the cumulative expectation of the value network. An objective function with pruning parameters is constructed and the cooperative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output.

[0014] Acquire training data, train the space-air collaborative reinforcement learning model, and output the trained model.

[0015] As a further implementation method, the drift model of the wrecked vessel and the people who fell into the water takes the location of the accident as the origin, sets the wrecked vessel and the people who fell into the water as a number of drifting particles, and sets a drag coefficient for each drifting particle.

[0016] At the start of each time step, the environmental parameters at that time are input, and the instantaneous velocity of the drifting particles is obtained by combining the drag coefficient. During the time step, the position of the drifting particles is updated using the velocity-displacement formula based on the instantaneous velocity of the drifting particles. The location with the maximum density of drifting particles is taken as the predicted position of the target to be rescued, and the instantaneous velocity of the drifting particles located at this predicted position is taken as the drift velocity of the target to be rescued.

[0017] As a further implementation method, in the drift model of the wrecked vessel and the person who fell into the water, the drag coefficient of the drifting particle corresponding to the person who fell into the water is set to a preset high sensitivity range; the drag coefficient of the drifting particle corresponding to the wrecked vessel is set according to the mass, length, width, height and draft of the wrecked vessel.

[0018] As a further implementation method, the multi-factor reward function is a multi-factor weighted fusion function that includes the response time of the aerospace platform, the total duration of target tracking and monitoring, the total energy consumption, the maximum duration of a single target loss, and the diversity of tracking and monitoring perspectives.

[0019] As a further implementation method, the objective function is:

[0020]

[0021] In the formula, This represents the objective function that introduces the clipping parameters; represents the parameters of the policy network; t represents the time step. The expected value of the reward function; The probability ratio of the strategy; The dominant function; This represents the clipping function. These are the trimming parameters.

[0022] As a further implementation method, the training process of the air-space collaborative reinforcement learning model is as follows:

[0023] A training cycle is started with a sample from the training data. The sample is input into the aerospace collaborative reinforcement learning model, and the model is run to start training. The model outputs a collaborative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function. The tracking coverage and reward function value under the collaborative tracking strategy are calculated. The weights of each factor in the model are adjusted according to the performance requirements of the tracking task. When the tracking coverage of the policy network is greater than a preset threshold and the fluctuation of the final reward function value is less than the preset threshold in several consecutive training cycles, it is considered to have converged, and the training of the model is stopped.

[0024] Input the validation set data into the model at this time. When the model reaches the convergence criterion within a preset number of training cycles, it is considered that the model training is successful. Output the model at this time as the trained aerospace-space collaborative reinforcement learning model. Otherwise, return to the training step, continue to input training examples into the aerospace-space collaborative reinforcement learning model, and start a new training cycle.

[0025] As a further implementation, it also includes: synchronously outputting a readable air-space coordination scheme according to a cooperative tracking strategy, the steps of which include:

[0026] Based on the actions of the intelligent agent and the satellite, corresponding text mapping templates are designed. The actions and action timestamps in the collaborative tracking strategy are extracted. The text mapping templates corresponding to the actions and the action timestamps appended in text format are written into the same document, and finally a readable space-air collaborative scheme ordered by timestamps is generated.

[0027] According to another aspect of this specification, a space-air cooperative tracking system for moving targets suitable for civilian maritime search and rescue is provided, comprising:

[0028] The task parameter acquisition module is used to acquire the accident location and environmental parameters;

[0029] The target position prediction module is used to construct a drift model of the wrecked vessel and the person who fell into the water. It inputs the parameters obtained by the task parameter acquisition module into the drift model and outputs the predicted position and instantaneous velocity of the target to be tracked, which are updated step by step over time.

[0030] The drone and satellite screening module is used to screen suitable drones and satellites based on environmental parameters, obtain parameter data of suitable drones and satellites, and obtain the satellite nadir position updated over time based on the satellite parameter data;

[0031] The cooperative tracking strategy acquisition module is used to load applicable UAVs and satellites and their parameter data, as well as the satellite nadir position updated over time, into the trained aerospace cooperative reinforcement learning model, and input the predicted position and instantaneous velocity of the target to be tracked, updated step by step, and output the cooperative tracking strategy; the training process of the aerospace cooperative reinforcement learning model is as follows:

[0032] A PPO-based aerospace cooperative reinforcement learning model is constructed: the policy network, value network, and objective function of PPO are adopted. In the policy network, the agent and its state and actions are updated with UAV and satellite and their parameter data. The environmental state is updated with the predicted position and instantaneous velocity of the target to be tracked updated by time step. The multi-factor reward function is used as the fitting benchmark for the cumulative expectation of the value network. An objective function with pruning parameters is constructed and the cooperative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output.

[0033] Acquire training data, train the space-air collaborative reinforcement learning model, and output the trained model.

[0034] According to another aspect of this specification, an electronic device is provided, including a memory and a processor, the memory storing program instructions executable by the processor, the processor invoking the program instructions to perform a method.

[0035] According to another aspect of this specification, a non-transitory computer-readable storage medium is provided that stores computer instructions that cause the computer to perform a method.

[0036] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention addresses the problem of insufficient air-space coordinated tracking capabilities for targets in maritime search and rescue. It leverages the complementary characteristics of UAV and remote sensing satellite observation capabilities, using air-space coordinated technology to increase the available tracking information. It constructs a PPO-based reinforcement learning model, setting a multi-factor reward function and constructing an objective function incorporating pruning parameters to obtain implementable coordinated tracking strategies. This enhances the timeliness and flexibility of the air-space coordinated tracking method, improving tracking efficiency and coordinated capabilities. It has the advantages of simplicity and high efficiency, effectively solving the problem that current air-space coordinated technologies are applicable to large-scale missions with long effective cycles, and are suitable for various maritime search and rescue mission scenarios. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic diagram of the main process of a space-air cooperative tracking method for moving targets applicable to civilian maritime search and rescue provided by an embodiment of the present invention:

[0039] Figure 2 A flowchart illustrating a method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue, provided as an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of a model of the drifting of a ship and a person who has fallen into the water in an embodiment of the present invention;

[0041] Figure 4 This is a structural diagram of the space-air coordinated tracking model in an embodiment of the present invention;

[0042] Figure 5 This invention provides a schematic diagram of a space-air coordinated tracking system for moving targets suitable for civilian maritime search and rescue, as provided in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] It should be noted that:

[0045] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0046] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be decomposed, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0048] The principle of the air-space cooperative tracking method for moving targets applicable to civilian maritime search and rescue of the present invention is to construct an air-space cooperative tracking PPO reinforcement learning model, so that the reward function of the model reflects the tracking and cooperative performance. The training examples are based on the maritime search and rescue mission scenario. Therefore, when the reward function reaches a high expectation, the model can output an air-space cooperative tracking scheme.

[0049] See Figure 1 , Figure 2 The present invention provides a space-air cooperative tracking method for moving targets applicable to civilian maritime search and rescue, comprising:

[0050] S1. Obtain the location of the accident and environmental parameters;

[0051] S2. Construct a drift model of the wrecked vessel and the person who fell into the water based on the accident location and environmental parameters, and obtain the predicted position and instantaneous velocity of the target to be tracked updated step by step.

[0052] S3. Select suitable drones and satellites based on environmental parameters, obtain parameter data of suitable drones and satellites, and obtain the satellite nadir position updated over time based on satellite parameter data;

[0053] S4. Load the applicable UAV and satellite and their parameter data and the satellite nadir position updated over time into the trained air-space cooperative reinforcement learning model, and input the predicted position and instantaneous velocity of the target to be tracked updated over time steps, and output the cooperative tracking strategy.

[0054] In step S2, the drift model of the wrecked vessel and the people who fell into the water is set as several drifting particles with the accident location as the origin, and a drag coefficient is set for each drifting particle.

[0055] At the start of each time step, the environmental parameters at that time are input, and the instantaneous velocity of the drifting particles is obtained by combining the drag coefficient. Within the time step, the position of the drifting particles is updated using the velocity-displacement formula based on the instantaneous velocity of the drifting particles, and the location with the maximum density of drifting particles is taken as the predicted location of the target to be rescued.

[0056] Furthermore, in the drift model of the wrecked vessel and the person who fell into the water, the drag coefficient of the drifting example corresponding to the person who fell into the water is set to a preset high sensitivity range; the drag coefficient of the drifting particle corresponding to the wrecked vessel is set according to the mass, length, width, height and draft of the wrecked vessel.

[0057] Specifically, the model for the drift of wrecked vessels and people in the water can be input as environmental parameters and the initial position of the moving target. A time step is set, and the current position of the drifting target is updated every short interval (less than 0.5 hours). Please refer to [link / reference] for details. Figure 3 The drift prediction model for wrecked vessels and people falling overboard uses the original location of the accident as the origin. 100 drift particles are assigned to each person and vessel (multiple drift particles are used to reduce the randomness of the results and improve prediction accuracy; the specific number can be adjusted according to actual needs). A drag coefficient is assigned to each drift particle, including ocean current drag coefficient and wind drag coefficient. Because the mass and volume of people falling overboard are relatively small, the drag coefficient of the drift particles representing them is uniformly set to a high-sensitivity value, treating them as freely drifting point masses. Because the mass and volume of vessels are relatively large, the drag coefficients of the drift particles representing the vessels are set according to the vessel's mass, length, width, height, and draft to ensure reasonable sensitivity.

[0058] Using ocean currents, wind speed, and wind direction at the accident location and time as environmental parameters, a drifting particle drift model based on the Lagrange method is constructed as a drift prediction model for wrecked vessels and people who have fallen overboard. This model calculates the wind-driven vector velocity of drifting particles using wind speed, wind direction, and wind drag coefficient, and the ocean current-driven vector velocity using current speed, current direction, and ocean current drag coefficient. The two velocities are combined to obtain the instantaneous velocity of the drifting particles. Assuming that the environmental parameters remain constant within a time step, the position of the drifting particles is updated using a velocity-displacement formula at each time step, and the predicted position of the target to be rescued is the location with the highest density of drifting particles representing the moving target. During implementation, the predicted positions of the targets to be rescued are obtained every 20 minutes, with a time step of 20 minutes.

[0059] Specifically, in the drift model of the wrecked vessel and the person who fell into the water, the drift example corresponding to the person who fell into the water has an ocean current drag coefficient k1 set with a preset high sensitivity value range, specifically from 0.90 to 1.00; the wind drag coefficient k2 is set to 0.020 to 0.035.

[0060] The formula for calculating the ocean current drag coefficient of drifting particles corresponding to the wrecked vessel is as follows:

[0061]

[0062] in, Ship mass (unit: t); The underwater projected area (unit: m², approximately calculated using ship length × draft); The underwater drag coefficient is 0.8-1.0 for streamlined ships and 0.5-0.7 for non-streamlined ships. The proportionality coefficient (taken as 0.0015) (used to balance the units of measurement).

[0063] The formula for calculating the wind drag coefficient is:

[0064]

[0065] in, The windward surface area on the water (unit: m², approximately using ship length × water height); For air resistance coefficients (1.2-1.5 for ships with flat decks, and 1.5-2.0 for ships with complex superstructures); Ship displacement (unit: t); This is the proportionality coefficient (taken as 0.008, used to constrain the results to a reasonable range).

[0066] Specifically, in step S3, the applicable drones and satellites, along with their parameter data, are obtained from the drone and satellite parameter database. The process of constructing the drone and satellite parameter database is as follows: First, the usable drones are identified, and their performance data (drone model), endurance (battery life), flight speed, flight altitude, flight heading, wind resistance, rain resistance, geographic coordinates, compatible payloads (e.g., infrared sensors or high-definition cameras), and drone payload data (payload sensor type, payload resolution, and night vision capability) are compiled and loaded into the drone dataset. Then, the usable satellites are identified, and their names, current two-line element (TLE) orbit data, and satellite payload data (payload sensor type, sensor on / off status, sensor resolution, maximum cloud cover for sensor operation, observation angle, and swath width) are compiled and loaded into the satellite dataset. Finally, the drone dataset and satellite dataset are combined to obtain the drone and satellite parameter database.

[0067] The screening process is as follows: Specifically, factors such as mission time (day / night), wind speed, humidity, rainfall, cloud cover, and mission resolution requirements are used to construct a decision tree to filter available satellites and drones from the database. Based on centralized drone data, the drone's wind resistance, rain resistance, payload resolution, and night vision capabilities are set. Based on centralized satellite data, the satellite's resolution and maximum operational cloud cover are set. The process involves checking whether the drone's wind resistance exceeds the ambient wind speed, its moisture and rain resistance exceeds the ambient humidity and rainfall, its payload resolution exceeds the minimum required drone resolution for the mission, and its night vision capability meets the mission time requirements; and whether the satellite's maximum operational cloud cover exceeds the ambient cloud cover, and its resolution exceeds the minimum required satellite resolution for the mission. Equipment meeting the mission requirements is recorded as the selected suitable satellites and drones.

[0068] Specifically, in step S3, the satellite nadir position updated over time is obtained by writing a corresponding satellite orbit prediction program using TLE data from the satellite dataset, thus obtaining the specific nadir position of the satellite at any given moment (accurate to the second) during the mission period. Preferably, the satellite's two-line orbit data are updated periodically (every 7 days).

[0069] In step S4, the training process of the air-space collaborative reinforcement learning model is as follows:

[0070] S41. Build a PPO-based aerospace cooperative reinforcement learning model, including: initializing the policy network to obtain the agent and its state and actions using UAV and satellite and their parameter data, and obtaining the target state using the predicted position of the target to be tracked; constructing a reward function based on maritime target tracking and an objective function that introduces pruning parameters.

[0071] S42. Obtain training data, train the air-space collaborative reinforcement learning model, and output the trained model.

[0072] Specifically, in step S41, a reinforcement learning model based on PPO is constructed as an air-space collaborative reinforcement learning model. This model uses the applicable satellites and UAVs selected in step 3 as agents, and the predicted target position and instantaneous velocity of the target to be tracked obtained in step 2 as the environmental state of the policy space. The model uses the air-space platform response time (the time interval between mission release and the first observation of the target), the total duration of target tracking and monitoring, the total energy consumption, the maximum duration of single target loss, and the diversity of tracking and monitoring perspectives (UAV perspective and satellite perspective) as factors, with the optimization objectives of extending tracking time, reducing energy consumption, and improving collaborative efficiency.

[0073] Specifically, such as Figure 4 As shown, in the aerospace collaborative reinforcement learning model, two types of intelligent agents are set up: drones and satellites. Each intelligent agent corresponds to a suitable drone or satellite, and the upper and lower limits of the intelligent agent's state are also determined by the corresponding drone and satellite parameters.

[0074] The state of a satellite agent is defined as its satellite nadir position (geographic coordinates) calculated by the satellite orbit prediction program under its corresponding timestamp, sensor on / off status, observation angle, and swath width; the state of a drone agent is defined as its geographic coordinate position, flight altitude, speed, heading, and battery life.

[0075] Satellite intelligent agent actions are divided into adjusting sensor angles and turning sensors on or off; UAV intelligent agent actions are divided into adjusting flight altitude, adjusting flight speed, adjusting flight direction, and turning sensors on or off.

[0076] The environment is defined as the geographical coordinates, speed, and course of the wrecked vessel, as well as the geographical coordinates of the people who fell into the water.

[0077] The environmental status under the mission timestamp includes: the status of the wrecked vessel, which is defined as the environmental status under the mission timestamp, and is determined by the geographical coordinates, speed and heading obtained from the predicted position and instantaneous speed of the wrecked vessel to be tracked; and the status of the person who fell overboard being tracked, which is defined as the geographical coordinates obtained from the predicted position of the person who fell overboard under the mission timestamp.

[0078] It is worth noting that the specific status of the people and boats that have fallen into the water, as obtained by the sensors, is not calculated by the model, but is transmitted back to the rescue personnel for processing.

[0079] The actions of drone intelligent agents are divided into adjusting flight altitude, adjusting flight speed, adjusting flight direction, and turning sensors on or off. The actions of satellite intelligent agents are divided into adjusting sensor angle and turning sensors on or off.

[0080] The model's reward function uses the space-air platform response time (the time interval between mission launch and the first observation of the target), the total duration of target tracking and monitoring, the total energy consumption, the maximum duration of a single target loss, and the diversity of tracking and monitoring perspectives (UAV perspective and satellite perspective) as factors. The initial reward function value is the weighted sum of all factors, mathematically expressed as:

[0081]

[0082] in, The reward function value; Indicates the response time of the aerospace platform. This represents the total duration of tracking and monitoring the target. This represents the total energy consumption. This represents the maximum duration of a single target loss. Factors representing the diversity of tracking and monitoring perspectives are presented in numerical form; + and - indicate rewards and punishments.

[0083] Specifically, the response time of the aerospace platform is the time interval from mission release to the first observation of the target. Whenever the state of the agent or the tracked target changes, the distance between the agent's position and the target's position is calculated. If it is less than the agent's observation range (the satellite observation range is determined by its swath width, and the UAV observation range is determined by its instantaneous field of view and the maximum tilt angle of the gimbal), then it is considered that the target has been observed. The timestamp of the observed target is recorded and collected to obtain the interval between the mission release timestamp and the first observation timestamp of the target.

[0084] Furthermore, the total duration of target tracking and monitoring refers to the sum of the time that the agent observes the target within the task period. Record the timestamp of each agent's transition from not observing the target to observing the target, and the timestamp of transition from observing the target to not observing the target. Calculate the difference to obtain the target tracking time for each time, and sum the target tracking times for all targets.

[0085] Furthermore, the maximum duration of a single target loss is the maximum duration of tracking interruptions within the task time. A timestamp is recorded when all agents are in a state of not observing the target, and a timestamp is recorded when any agent resumes observing the target. The difference is calculated and the timestamps are sorted, with the longest time interval being selected.

[0086] Additionally, the total energy consumption is calculated to simplify the energy consumption of the UAV and satellite. The energy consumption per unit time of the UAV is set according to the UAV's flight speed and altitude. The start and end timestamps of the UAV maintaining a certain speed and altitude are recorded to obtain the UAV's energy consumption. At the same time, the single energy consumption (fixed value) of the satellite's payload operation is set for each start-up. The sum of all energy consumption during the mission period is then calculated.

[0087] Additionally, the diversity of tracking and monitoring perspectives is the number of agents observing the target. Whenever a new agent can be considered as an observed target, the diversity of tracking and monitoring perspectives is incremented by 1.

[0088] Whenever the drone's heading or speed changes, or the satellite's startup state changes, the model updates the generated drone and satellite call-up schemes accordingly. This facilitates visualization of the training effect and allows for intuitive monitoring of the agent's decision-making logic. To improve sample efficiency and enhance the stability of the model when generating drone call-up schemes, this model is based on PPO and utilizes PPO's policy network, value network, and objective function to set hyperparameters and objective functions. The model's hyperparameters include pruning parameters. The model's objective function limits the update magnitude of the drone and satellite call-up schemes through these pruning parameters. This prevents drastic changes in scheme generation that could not guide the drone to perform impossible turns or speed changes. The mathematical expression is as follows:

[0089]

[0090] This formula maximizes the advantage function while ensuring that policy updates do not deviate too far from the old policy. The parameters are explained below:

[0091] This represents the objective function that introduces the clipping parameters;

[0092] : The parameters of the current policy network (i.e., the weights of the neural network);

[0093] Time step: Represents the sequence of interactions between the agent and the environment;

[0094] : Empirical expectation, representing the average of the sampled trajectory data;

[0095] The strategy probability ratio measures the change in preference for a particular action under a new strategy relative to the old strategy. If... This indicates that the new strategy is more inclined to choose that action; conversely, it is less inclined to choose it. , For the current policy in state Select action The probability, This represents the probability that the old strategy (the strategy from the previous iteration) will choose the same action under the same conditions.

[0096] Advantage function, which is the action In state The advantage estimate is the "advantage" of the action relative to the average action. Where Q is the action value function and V is the state value function, if This indicates that the movement is better than the average movement; if If the result is negative, it means that the action is worse than the average action.

[0097] clip represents the clipping function. This represents the trimming parameter, which is usually set to 0.1 or 0.2. Used to Limited to the range inside, if Then cut into Then cut into ;like If it remains unchanged, then it will remain the same.

[0098] In step 4, the training process of the air-space collaborative reinforcement learning model is as follows:

[0099] A training cycle is started with a sample from the training data. The sample is input into the aerospace collaborative reinforcement learning model, and the model is run to start training. The model outputs a collaborative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function. The tracking coverage and reward function value under the collaborative tracking strategy are calculated. The weights of each factor in the model are adjusted according to the performance requirements of the tracking task. When the tracking coverage of the policy network is greater than a preset threshold and the fluctuation of the final reward function value is less than the preset threshold in several consecutive training cycles, it is considered to have converged, and the training of the model is stopped.

[0100] Input the validation set data into the model at this time. When the model reaches the convergence criterion within a preset number of training cycles, it is considered that the model training is successful. Output the model at this time as the trained aerospace-space collaborative reinforcement learning model. Otherwise, return to the training step, continue to input training examples into the aerospace-space collaborative reinforcement learning model, and start a new training cycle.

[0101] Optionally, the training process of the air-space collaborative reinforcement learning model includes:

[0102] S401. Set up multiple training examples. Obtain satellite parameters and orbital data from official websites (such as the National Remote Sensing Data and Application Service Platform), drone data from the official websites of various drone brands, and the occurrence time and geographical coordinates of real-world maritime search and rescue cases from official accident reports released by the Maritime Safety Administration. Obtain specific environmental parameters at the time of the incident from weather websites (such as Earth). Construct a class (or structure) to store the datasets required for model training, facilitating the input of all parameters into the model at once. During training, the model generates a collaborative scheme, namely the drone and satellite calling scheme.

[0103] Input the first training example into the model built in step 4, run the model to start training, and observe the expected change of the reward function value and the changes of the values ​​of each factor in the reward function during the training process. Assume that the UAV and satellite operate according to the collaborative scheme generated by the model in the mission. The response time determines the time from the start of the mission to the first acquisition of target information, the total tracking time determines the amount of target information that can be obtained within the specified time period, the duration of a single target loss determines the information gap in the mission, the diversity of tracking perspectives determines the comprehensiveness of target information obtained in the mission, and energy consumption determines the mission cost. (Different types of search and rescue missions have different performance requirements. The model training needs to adjust the weights of each factor according to the performance requirements of the mission.) Continue to input different training examples for training. Set a minimum threshold for tracking coverage (effective tracking time / total mission time) according to the performance that the model-generated scheme can achieve. Stop training when the tracking coverage of the policy network is greater than the preset threshold and the fluctuation of the final reward function value is less than the preset threshold within N consecutive training cycles (N≥10).

[0104] S402. Construct multiple validation examples using the same method as in S401. When the model can track the new input example with a coverage rate greater than the preset threshold within M training cycles (M≤5) and the fluctuation of the final reward function value is less than the preset threshold, the model is considered to have been successfully trained. Otherwise, repeat step S401.

[0105] Furthermore, the method also includes: step S5, synchronously outputting a readable air-space coordination scheme according to the cooperative tracking strategy, the steps of which include:

[0106] Based on the actions of the intelligent agent and the satellite, corresponding text mapping templates are designed. The actions and action timestamps in the collaborative tracking strategy are extracted. The text mapping templates corresponding to the actions and the action timestamps appended in text format are written into the same document, and finally a readable space-air collaborative scheme ordered by timestamps is generated.

[0107] Specifically, after training, the UAV and satellite agents have learned a stable cooperative tracking strategy. At this point, based on the actions of the UAV and satellite agents described in step 4, a text mapping template is created so that each action of the UAV and satellite agents has a readable text counterpart. The UAV and satellite agent action data given during model runtime are mapped to the text mapping template. After training, the environmental parameters of the current search and rescue mission, the data on the missing persons and the wrecked vessel (predicted location) are output to the trained aerospace cooperative reinforcement learning model. The environmental parameters are then used to filter applicable UAVs and satellites and their parameter data. The model is run to generate a UAV and satellite call scheme (cooperative tracking strategy) that actually guides the cooperative tracking mission. Each time the UAV and satellite agents take an action, the corresponding text mapping template and the timestamp of the action are appended to the same document (txt, csv, etc.) in text format, ultimately generating a readable aerospace cooperative scheme ordered by time.

[0108] Preferably, the aerospace collaborative reinforcement learning model has a program for calling up UAVs and satellites. The screening is carried out during initialization, rather than directly inputting the screened UAV and satellite data into the environment. The UAVs and satellites selected for use may be different in different tasks.

[0109] In reinforcement learning, the model cannot directly plan the satellite trajectory; it can only plan the satellite's operation by determining whether the satellite is in the position of an observable target. Therefore, a satellite orbit prediction program using TLE data is used to obtain the specific sub-satellite point position of the satellite at any given moment (accurate to the second) during the mission period, providing data support for the model's satellite planning. Furthermore, using the drift prediction model obtained in step 2, the predicted position and instantaneous velocity of the target to be rescued are obtained within a 20-minute time step (every 20 minutes) during the mission period.

[0110] Furthermore, the drift model of the wrecked vessel and the people who fell into the water in step 2 was also loaded into the aerospace collaborative reinforcement learning model in the form of a program.

[0111] Based on this, a preferred embodiment is provided, in which the trained air-space collaborative reinforcement learning model realizes the air-space collaborative tracking of moving targets in maritime search and rescue as follows:

[0112] Data input (updating search and rescue mission parameters): First, in the aerospace collaborative reinforcement learning model, obtain the database of accident location and environmental parameters, parameters of people who fell into the water and the wrecked vessel, as well as parameters of UAVs and satellites;

[0113] Data preprocessing: The drift model of the wrecked vessel and the people who fell into the water is invoked. Based on the accident location, environmental parameters, and parameters of the people who fell into the water and the wrecked vessel, the predicted location of the target is obtained. At the same time, the UAV and satellite screening program is invoked to screen applicable UAVs and satellites and their parameter data based on environmental parameters (including the satellite orbit prediction program using TLE data to obtain the specific sub-satellite point position of the satellite at any time during the mission period).

[0114] Model processing: Within a specified step size (task period), the model decides each action, executes the action in the environment, and updates and records the new agent state and expected value of the reward function. During model operation, each action will be written to the collaborative scheme file with a corresponding text mapping template carrying a text-formatted timestamp, until the specified step size is reached and the operation ends. The final collaborative scheme file is a readable space-air collaborative scheme.

[0115] The implementation of the various embodiments of the present invention is based on programmed processing through a system with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a space-air cooperative tracking system for moving targets suitable for civilian maritime search and rescue. This system is used to execute a space-air cooperative tracking method for moving targets suitable for civilian maritime search and rescue as described in the above method embodiments.

[0116] See Figure 5 The system includes:

[0117] The system includes a mission parameter acquisition module for acquiring the accident location and environmental parameters; a target position prediction module for constructing a drift model of the wrecked vessel and the person in the water, inputting the parameters acquired by the mission parameter acquisition module into the drift model, and outputting the predicted position and instantaneous velocity of the target to be tracked, updated step-by-step; a UAV and satellite selection module for selecting suitable UAVs and satellites based on environmental parameters, acquiring parameter data of suitable UAVs and satellites, and obtaining the satellite nadir position updated step-by-step based on the satellite parameter data; and a cooperative tracking strategy acquisition module for loading suitable UAVs and satellites, their parameter data, and the satellite nadir position updated step-by-step into the trained aerospace cooperative reinforcement learning model, and inputting the predicted position and instantaneous velocity of the target to be tracked, updated step-by-step based on the target nadir position. The model tracks the predicted position and instantaneous velocity of the target and outputs a cooperative tracking strategy. The training process of the space-air cooperative reinforcement learning model is as follows: A space-air cooperative reinforcement learning model based on the Produced Position and Velocity of the Object (PPO) is constructed. The PPO's policy network, value network, and objective function are used. In the policy network, the agent's state and actions are updated using UAV and satellite data and their parameters. The environmental state is updated using the predicted position and instantaneous velocity of the target to be tracked, updated step-by-step. A multi-factor reward function is used as the fitting benchmark for the accumulated expectation of the value network. An objective function incorporating pruning parameters is constructed, and the cooperative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output. Training data is acquired, the space-air cooperative reinforcement learning model is trained, and the trained model is output.

[0118] It should be noted that the system embodiments provided by the present invention are used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above system embodiments provided by the present invention. As long as those skilled in the art can improve the modules in the above system embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above system embodiments, and on the premise of ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments.

[0119] The method in this embodiment of the invention is implemented using an electronic device; therefore, it is necessary to introduce the relevant electronic device. For this purpose, embodiments of the present invention provide an electronic device, such as... Figure 6 As shown, the electronic device includes: at least one processor, a communication interface, at least one memory, and a communication bus, wherein the at least one processor, the communication interface, and the at least one memory communicate with each other via the communication bus. The at least one processor invokes logical instructions stored in the at least one memory to execute all or part of the steps of the methods provided in the foregoing method embodiments.

[0120] Furthermore, when the logical instructions in at least one of the aforementioned memories are implemented as software functional units and sold or used as independent products, they are stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, is embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (a personal computer, server, or network device) to execute all or part of the steps of the methods described in the various method embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks—various media for storing program code.

[0121] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, located in one place, or distributed across multiple network units. The purpose of this embodiment is achieved by selecting some or all of the modules according to actual needs. Those skilled in the art will understand and implement this without any inventive effort.

[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] Based on the same technical concept as the foregoing embodiments, the present invention provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute a method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue.

[0127] In summary, this invention provides a space-air cooperative tracking method for moving targets applicable to civilian maritime search and rescue. It integrates callable satellite data and UAV data to form a satellite and UAV dataset; constructs a drift model for ships and people in the water, which can predict the current location range of the moving target by taking environmental parameters and the initial position of the moving target as input; constructs a reinforcement learning model based on near-end strategy optimization to provide a space-air cooperative tracking solution as the output, with the optimization objective of increasing the efficiency of space-air cooperative tracking of moving targets; selects suitable UAVs and satellites according to the maritime environment, and loads space-air platform data, environmental parameters, and predicted target positions into the reinforcement learning model, wherein the environmental parameters and target positions can be updated in real time; constructs maritime search and rescue scenario training and validation sets of task examples, trains the reinforcement learning model, and adjusts the model parameters until the model can provide a better solution for mobilizing UAVs and satellites to cooperate in tracking targets within a short time after receiving validation set examples.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A space-air cooperative tracking method for moving targets applicable to civilian maritime search and rescue, characterized in that, include: Obtain the location of the accident and environmental parameters; Based on the accident location and environmental parameters, a drift model of the wrecked vessel and the person who fell into the water is constructed to obtain the predicted position and instantaneous velocity of the target to be tracked, updated step by step over time. Applicable drones and satellites are selected based on environmental parameters, and parameter data of the applicable drones and satellites are obtained. The satellite sub-satellite position updated over time is obtained based on the satellite parameter data. The applicable UAVs and satellites and their parameter data, as well as the satellite nadir position updated over time, are loaded into the trained aerospace cooperative reinforcement learning model. The predicted position and instantaneous velocity of the target to be tracked, updated over time steps, are also input, and the cooperative tracking strategy is output. The training process of the aforementioned space-air collaborative reinforcement learning model is as follows: A PPO-based aerospace cooperative reinforcement learning model is constructed: the policy network, value network, and objective function of PPO are adopted. In the policy network, the agent and its state and actions are updated with UAV and satellite and their parameter data. The environmental state is updated with the predicted position and instantaneous velocity of the target to be tracked updated by time step. The multi-factor reward function is used as the fitting benchmark for the cumulative expectation of the value network. An objective function with pruning parameters is constructed and the cooperative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output. Acquire training data, train the space-air collaborative reinforcement learning model, and output the trained model.

2. The method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 1, characterized in that, The drift model of the wrecked vessel and the people who fell into the water is set as several drifting particles with the accident site as the origin, and a drag coefficient is set for each drifting particle. At the start of each time step, the environmental parameters at that time are input, and the instantaneous velocity of the drifting particles is obtained by combining the drag coefficient. During the time step, the position of the drifting particles is updated using the velocity-displacement formula based on the instantaneous velocity of the drifting particles. The location with the maximum density of drifting particles is taken as the predicted position of the target to be rescued, and the instantaneous velocity of the drifting particles located at this predicted position is taken as the drift velocity of the target to be rescued.

3. The method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 2, characterized in that, In the drift model of the wrecked vessel and the person who fell into the water, the drag coefficient of the drifting particle corresponding to the person who fell into the water is set within a preset high sensitivity range; the drag coefficient of the drifting particle corresponding to the wrecked vessel is set according to the mass, length, width, height and draft of the wrecked vessel.

4. The method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 1, characterized in that, The multi-factor reward function is a weighted fusion function that includes the response time of the aerospace platform, the total duration of target tracking and monitoring, the total energy consumption, the maximum duration of a single target loss, and the diversity of tracking and monitoring perspectives.

5. A method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 1, characterized in that, The objective function is: ； In the formula, This represents the objective function that introduces the clipping parameters; represents the parameters of the policy network; t represents the time step. The expected value of the reward function; The probability ratio of the strategy; The dominant function; This represents the clipping function. These are the trimming parameters.

6. The method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 1, characterized in that, The training process of the aforementioned space-air collaborative reinforcement learning model is as follows: A training cycle is started with a sample from the training data. The sample is input into the aerospace collaborative reinforcement learning model. The model is run to start training. The collaborative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output. The tracking coverage and reward function value under the collaborative tracking strategy are calculated. The weights of each factor in the model are adjusted according to the performance requirements of the tracking task. When the tracking coverage of the policy network is greater than a preset threshold and the fluctuation of the final reward function value is less than a preset threshold in several consecutive training cycles, it is considered to have converged and the training of the model is stopped. Input the validation set data into the model at this time. When the model reaches the convergence criterion within a preset number of training cycles, it is considered that the model training is successful. Output the model at this time as the trained aerospace-space collaborative reinforcement learning model. Otherwise, return to the training step, continue to input training examples into the aerospace-space collaborative reinforcement learning model, and start a new training cycle.

7. The method for air-space coordinated tracking of moving targets applicable to civilian maritime search and rescue as described in claim 1, characterized in that, Also includes: The steps include synchronously outputting a readable space-air coordination scheme based on the collaborative tracking strategy: Based on the actions of the intelligent agent and the satellite, corresponding text mapping templates are designed. The actions and action timestamps in the collaborative tracking strategy are extracted. The text mapping templates corresponding to the actions and the action timestamps appended in text format are written into the same document, and finally a readable space-air collaborative scheme ordered by timestamps is generated.

8. A space-air cooperative tracking system for moving targets suitable for civilian maritime search and rescue, characterized in that, include: The task parameter acquisition module is used to acquire the accident location and environmental parameters; The target position prediction module is used to construct a drift model of the wrecked vessel and the person who fell into the water. It inputs the parameters obtained by the task parameter acquisition module into the drift model and outputs the predicted position and instantaneous velocity of the target to be tracked, which are updated step by step over time. The drone and satellite screening module is used to screen suitable drones and satellites based on environmental parameters, obtain parameter data of suitable drones and satellites, and obtain the satellite nadir position updated over time based on the satellite parameter data; The cooperative tracking strategy acquisition module is used to load applicable UAVs and satellites and their parameter data, as well as the satellite nadir position updated over time, into the trained aerospace cooperative reinforcement learning model, and input the predicted position and instantaneous velocity of the target to be tracked, updated step by step, and output the cooperative tracking strategy; the training process of the aerospace cooperative reinforcement learning model is as follows: A PPO-based aerospace cooperative reinforcement learning model is constructed: the policy network, value network, and objective function of PPO are adopted. In the policy network, the agent and its state and actions are updated with UAV and satellite and their parameter data. The environmental state is updated with the predicted position and instantaneous velocity of the target to be tracked updated by time step. The multi-factor reward function is used as the fitting benchmark for the cumulative expectation of the value network. An objective function with pruning parameters is constructed and the cooperative tracking strategy of the agent that maximizes the expected value of the multi-factor reward function is output. Acquire training data, train the space-air collaborative reinforcement learning model, and output the trained model.

9. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing program instructions that are executed by the processor, the processor invoking the program instructions to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method described in any one of claims 1 to 7.

Images