Satellite unmanned aerial vehicle cooperative task planning method and system for time-sensitive target

Abstract

The invention provides a satellite unmanned aerial vehicle cooperative task planning method and system for a time-sensitive target, and the method comprises the steps: grouping satellites and unmanned aerial vehicles, and distributing a search discovery task and / or a positioning recognition task to observation resources of different groups according to the task types and observation resource characteristics; performing cooperative scheduling on the grouped satellites and unmanned aerial vehicles; after collaborative scheduling is completed, aiming at a satellite group, distributing observation tasks to corresponding satellites for execution; planning the path of the unmanned aerial vehicle for the unmanned aerial vehicle cluster; observation images of the same area from different satellites and unmanned aerial vehicles are obtained, image information of different time and angles in the observation images is fused and discriminated, and recognition and positioning are completed. And dynamic monitoring is carried out for the time-sensitive target, task distribution is continued according to an information source fusion result and a task planning mechanism, and a satellite unmanned aerial vehicle cooperative task planning closed loop for the time-sensitive target in real time is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of fire monitoring and satellite mission planning, and in particular to a satellite-UAV collaborative mission planning method and system for time-sensitive targets, as well as a corresponding computer terminal and a computer-readable storage medium. Background Art

[0002] Forest fires occur in a wide range of locations and in different types of forests. Satellite remote sensing has the characteristics of wide coverage and multiple spectra, which is suitable for fire monitoring. However, the existing forest fire remote sensing satellite monitoring systems (FIRMS and GWIS) have insufficient revisit cycles under high resolution and cannot meet the needs of real-time dynamic monitoring of fire sites. It is necessary to combine low-orbit monitoring constellations and drones to monitor forest fires. Due to the large heterogeneous characteristics of constellation and drone observation resources, their coordination mechanism is complex, the scale is huge, and task planning is difficult. On the other hand, forest fires are sporadic and develop rapidly, which brings difficulties to forest fire monitoring tasks with dynamic development and strong randomness.

[0003] At present, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected yet. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a satellite-UAV collaborative mission planning method and system for time-sensitive targets, and also provides a corresponding computer terminal and computer-readable storage medium.

[0005] According to one aspect of the present invention, a satellite-UAV collaborative mission planning method for time-sensitive targets is provided, comprising:

[0006] Provide a task planning mechanism, use the task planning mechanism to group satellites and drones, and then assign search and discovery tasks and / or positioning and identification tasks to observation resources in different groups;

[0007] Coordinated scheduling of grouped satellites and drones; after coordinated scheduling, for satellite constellations, observation tasks are assigned to corresponding satellites for execution; for drone clusters, drone paths are planned;

[0008] Obtain observation images of the same area from different satellites and drones respectively, fuse and identify image information at different times and angles in the observation images, complete identification and positioning, and obtain information source fusion results;

[0009] Dynamic monitoring is carried out for time-sensitive targets, and tasks are continuously dispatched based on the information source fusion results and the task planning mechanism, so as to realize a closed loop of satellite-UAV collaborative task planning for time-sensitive targets in real time.

[0010] Preferably, providing a task planning mechanism, using the task planning mechanism to group satellites and drones, and then assigning search and discovery tasks and / or positioning and identification tasks to observation resources in different groups, including:

[0011] The task planning mechanism includes:

[0012] Process the target task data required for planning, and use the induction algorithm to generate readable rules and rule-based decision trees;

[0013] Using the rules, dynamically adjust the grouping of satellites and drones;

[0014] By using the decision tree, task allocation decisions for different tasks and / or different scenarios are made according to task types and observation resource characteristics, and search and discovery tasks and / or positioning and identification tasks are respectively assigned to different groups of observation resources.

[0015] Preferably, the grouped satellites and drones are collaboratively scheduled; after the collaborative scheduling is completed, for the satellite constellation, the observation tasks are assigned to the corresponding satellites for execution; for the drone cluster, the drone paths are planned, including:

[0016] A multi-objective genetic algorithm based on meta-heuristics is used to formulate a satellite constellation and UAV cluster coordination strategy for coordinated scheduling of grouped satellites and UAVs, a constellation task allocation strategy for assigning observation tasks to corresponding satellites, and a single-person or multi-person UAV path coordination planning strategy for planning UAV paths; wherein the multi-objective genetic algorithm based on meta-heuristics includes:

[0017] Set planning goals, including: maximizing synergy, maximizing coverage, minimizing task completion time, and minimizing energy consumption;

[0018] Using multi-layer genetic design, the satellite constellation and UAV cluster coordination strategy, the constellation task allocation strategy and the UAV path coordination planning strategy are planned synchronously to generate the initial population;

[0019] Based on the planning goal, performing fitness evaluation on the initial population;

[0020] The evaluation process is iterated using a multi-objective optimization function to find the frontier solution. When the iteration requirements are met, the multi-objective optimization planning results are output.

[0021] Preferably, the step of respectively acquiring observation images of the same area from different satellites and drones, fusing and distinguishing image information at different times and angles in the observation images, and completing identification and positioning includes:

[0022] A pre-trained image recognition model is used to extract features of observation images of the same area from different satellites and drones and form a comprehensive feature representation to obtain multi-scale transformed image information from different times and angles;

[0023] The image information from different times and angles is fused to obtain an information source fusion result.

[0024] Preferably, the dynamic monitoring for time-sensitive targets and the continued task dispatching according to the information source fusion results and the task planning mechanism include:

[0025] Provide a planning strategy generation model based on deep reinforcement learning;

[0026] A model is generated based on the planning strategy, and the state information and target information of the UAV and satellite are used as the input of the model. The model selects actions, and the environment gives new states and rewards according to the actions. The state information of the UAV and satellite includes: the orbital height, observation angle and resolution information of the satellite, and the flight time, range and resolution information of the UAV. The target information is the fusion result of information sources.

[0027] Based on the task planning mechanism, the PPO method is used to perform dynamic rolling task planning on the planning strategy generation model to learn to maximize the cumulative reward; during the task cycle, a heterogeneous system is used to observe the target in real time to achieve dynamic decision-making and optimize the task allocation in the monitoring process.

[0028] Preferably, the above method further comprises:

[0029] The state information and target information of the drone and satellite are obtained from the environment. The discrete and continuous Actor networks of the planning strategy generation model take actions, outputting actions on continuous airborne and actions in discrete space respectively. The environment feeds back the next state and a positive and negative reward based on this action. The Critic network of the planning strategy generation model trains a state value function to calculate the value used to update the actor network. The satellite is used as an intelligent agent to make decisions in the current state, transition to the next state and obtain benefits. The Markov decision process model M is:

[0030] M={S,A,P,R,γ}

[0031] Among them, S is a finite state set, A is an action set, P is a state transition probability matrix, R is a profit equation, γ is a discount factor, γ∈[0,1], which is used to adjust the importance of future rewards in long-term decision optimization;

[0032] In the satellite integrated mission planning problem, the finite state set S includes the task set and the satellite state, then:

[0033] S=Task,SAT,Drone

[0034] Among them, Task is the observation task information, which includes target information, Drone is the drone information, and SAT is the satellite status information;

[0035] The action set is a finite decision variable. Here, the action set A is designed as:

[0036] A=No.ID

[0037] Among them, No is the mission number selected for this action, and ID is the ID of the satellite / UAV selected in this mission.

[0038] According to another aspect of the present invention, a satellite-UAV collaborative mission planning system for time-sensitive targets is provided, comprising:

[0039] A task planning module, which is used to provide a task planning mechanism, using which satellites and drones are grouped, and then search and discovery tasks and / or positioning and identification tasks are assigned to observation resources in different groups;

[0040] Task planning and scheduling module, which is used to coordinate the scheduling of grouped satellites and drones; after completing the coordinated scheduling, for the satellite constellation, the observation tasks are assigned to the corresponding satellites for execution; for the drone cluster, the drone path is planned;

[0041] A multi-source heterogeneous information source fusion identification module is used to obtain observation images of the same area from different satellites and drones, fuse and identify image information at different times and angles in the observation images, complete identification and positioning, and obtain information source fusion results;

[0042] A closed-loop module for the continuous scheduling, tracking and planning of heterogeneous observation resources is used to dynamically monitor time-sensitive targets, and to continue to dispatch tasks based on the information source fusion results and the task planning mechanism, thereby realizing a real-time closed-loop satellite-UAV collaborative task planning for time-sensitive targets.

[0043] According to a third aspect of the present invention, there is provided a computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, can be used to execute any one of the methods described above in the present invention, or to run the system described above in the present invention.

[0044] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the methods described above in the present invention, or to run the system described above in the present invention.

[0045] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0046] The satellite-UAV collaborative task planning method and system for time-sensitive targets provided by the present invention can efficiently complete the task allocation between satellites and UAVs and allocate appropriate tasks to satellites and UAVs.

[0047] The satellite-UAV collaborative mission planning method and system for time-sensitive targets provided by the present invention can quickly respond to emergency observation missions such as natural disasters and realize rapid mission dispatch.

[0048] The satellite UAV collaborative mission planning method and system for time-sensitive targets provided by the present invention can specify a reasonable satellite UAV mission collaboration strategy and formulate a reasonable collaboration mode for different types of tasks.

[0049] The satellite-UAV collaborative mission planning method and system for time-sensitive targets provided by the present invention can complete multi-target mission planning in the satellite and UAV mission collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0051] Figure 1 This is the task flow chart for the collaborative work of heterogeneous giant observation resources.

[0052] Figure 2 The present invention is a flowchart of a satellite-UAV collaborative mission planning method for time-sensitive targets in a preferred embodiment of the present invention.

[0053] Figure 3 Schematic diagram of the overall architecture of a satellite-UAV collaborative mission planning method for time-sensitive targets in a preferred embodiment of the present invention.

[0054] Figure 4 The present invention is a schematic diagram of a satellite-UAV collaborative mission planning method for time-sensitive targets in a preferred embodiment of the present invention.

[0055] Figure 5 The figure is a schematic diagram of the components of a satellite-UAV collaborative mission planning system for time-sensitive targets in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0057] In the coordination of heterogeneous giant observation resources, the resources that need to be planned include space-based observation satellite resources and air-based UAV resources. Observation resources include satellites with different payloads and UAV observation resources. Satellites with different payloads, such as SAR satellites and optical satellites, have different imaging capabilities, fields of view, resolutions, etc.; UAV observation resources include UAVs with different ranges and payloads. The mission process is as follows: Figure 1 As shown. Figure 1 It can be seen that in the collaborative observation of time-sensitive targets, it is necessary to search and discover large areas first. In this process, satellites with a large field of view need to be used for collaborative detection of large areas. After a suspicious target is found in a certain area, high-resolution observation resources need to be used for collaborative detection of the target area. The main collaborative observation resources are high-resolution optical satellites, SAR satellites, and drones. Among them, it is necessary to perform fusion identification for heterogeneous information sources from multiple sources. After completing the identification and positioning, it is turned to comprehensive revisit scheduling for time-sensitive targets, providing positioning information to users in real time, and using heterogeneous observation resources to track and observe the target in near real time. In this planning process, there are many types of collaborative resources and complex working collaborative modes. It is necessary to mathematically model and comprehensively plan the observation resources with different characteristics. At present, the research on multi-satellite collaborative technology is in development, and the main research focuses on the research of collaborative mechanisms. The requirements for collaboration for different tasks are quite different. However, in existing studies, the types of collaborative resources are single, the targets and task modes are single, and the research on the collaborative mechanism between drones and satellites is relatively small, which cannot meet the planning needs of this task. Moreover, there is dynamic uncertainty in the task planning of time-sensitive targets, which leads to the temporal and spatial complexity of heterogeneous resource allocation problems. Current research lacks the study on the combination of high-mobility dynamic mechanisms and planning for time-sensitive targets.

[0058] In response to the above problems, an embodiment of the present invention provides a satellite-UAV collaborative task planning method for time-sensitive targets. The method studies the collaborative task planning of forest fire monitoring of large heterogeneous satellite constellations. In the heterogeneous collaboration between UAVs and satellites, dynamic monitoring of time-sensitive targets is carried out through task planning, planning scheduling, and continuous scheduling and tracking of multi-source heterogeneous information sources. Finally, the above steps are coordinated to achieve a planning closed loop.

[0059] Specifically, Figure 2As shown, the satellite-UAV collaborative mission planning method for time-sensitive targets provided in this embodiment, based on a hierarchical scheduling planning framework, may include the following operations:

[0060] S1, mission planning mechanism: provide a mission planning mechanism, use the mission planning mechanism to group satellites and UAVs, and then assign search and discovery missions and / or positioning and identification missions to different groups of observation resources; further, formulate a satellite constellation and UAV cluster grouping strategy based on observation resource characteristics such as payload resolution, width, field of view, current position, and UAV range, group satellites and UAVs, and assign search and discovery missions and / or positioning and identification missions to different groups of observation resources based on mission type and observation resource characteristics.

[0061] S2, task planning and scheduling: coordinate the scheduling of grouped satellites and drones; after completing the coordinated scheduling, for the satellite constellation, the observation tasks are assigned to the corresponding satellites for execution; for the drone cluster, the drone paths are planned; further, a coordination mechanism between satellite constellations and drones is formulated, and task coordination scheduling is performed according to the coordination mechanism, and the tasks are coordinated with the appropriate satellite constellations / drones to perform tasks; after completing the coordinated scheduling of tasks, for the satellite constellation, a task allocation strategy is required to allocate tasks, and the observation tasks are assigned to the appropriate satellites for execution; for drones, a single / multi-drone path coordination planning strategy is formulated to plan the drone paths;

[0062] S3, multi-source heterogeneous information source fusion identification: obtain observation images of the same area from different satellites and drones respectively, fuse and identify the image information of different times and angles in the observation images, complete identification and positioning, and obtain the information source fusion result; further, adopt a multi-source heterogeneous information source fusion identification method to fuse and identify the data from different satellites and drones to complete identification and positioning; formulate a multi-source information fusion method based on the image processing algorithm, fuse the observation images of the same area from different satellites and drones, and fuse the image information from different times and angles to improve the accuracy of target recognition and positioning.

[0063] S4, the follow-up scheduling, tracking and planning closed loop of heterogeneous observation resources: dynamic monitoring is carried out for time-sensitive targets, and tasks are continuously dispatched according to the information source fusion results and the task planning mechanism, so as to realize the real-time satellite-UAV collaborative task planning closed loop for time-sensitive targets; further, after identification and positioning, dynamic monitoring is required for time-sensitive targets, and tasks are continuously dispatched according to the information source fusion results and the task planning mechanism, so as to realize the real-time feedback mechanism and the satellite-UAV collaborative task planning closed loop for time-sensitive targets with high dynamic uncertainty.

[0064] In this embodiment, the time-sensitive target may be a dynamic target at the fire scene.

[0065] In some preferred embodiments, the above S1 may further include:

[0066] A task planning mechanism using an adaptive dynamic grouping strategy to distribute tasks; wherein the task planning mechanism includes:

[0067] Process the target task data required for planning, and use the induction algorithm to generate readable rules and rule-based decision trees;

[0068] Use rules to dynamically adjust the grouping of satellites and drones;

[0069] By using the decision tree, task allocation decisions for different tasks and / or different scenarios are made according to the task type and observation resource characteristics, and search and discovery tasks and / or positioning and identification tasks are respectively assigned to different groups of observation resources.

[0070] In some preferred embodiments, the above S2 may further include:

[0071] A multi-objective genetic algorithm based on meta-heuristics is used to formulate a satellite constellation and UAV cluster coordination strategy for coordinated scheduling of grouped satellites and UAVs, a constellation task allocation strategy for assigning observation tasks to corresponding satellites, and a single-person or multi-person UAV path coordination planning strategy for planning the UAV path; wherein, the multi-objective genetic algorithm based on meta-heuristics includes:

[0072] Set planning goals, including: maximizing synergy, maximizing coverage, minimizing task completion time, and minimizing energy consumption;

[0073] Using multi-layer genetic design, the satellite constellation and UAV cluster coordination strategy, the constellation task allocation strategy and the UAV path coordination planning strategy are planned synchronously to generate the initial population;

[0074] Based on the planning objectives, the fitness of the initial population is evaluated;

[0075] The evaluation process is iterated using a multi-objective optimization function to find the frontier solution. When the iteration requirements are met, the multi-objective optimization planning results are output.

[0076] In some preferred implementations, the above S3 may further include:

[0077] A pre-trained image recognition model is used to extract features of observation images of the same area from different satellites and drones and form a comprehensive feature representation to obtain multi-scale transformed image information from different times and angles;

[0078] The image information from different times and angles is fused to obtain the information source fusion result.

[0079] In some preferred embodiments, the above S4 may further include:

[0080] Provide a planning strategy generation model based on deep reinforcement learning;

[0081] Based on the planning strategy generation model, the state information and target information of the UAV and satellite are used as the input of the model. The model selects actions, and the environment gives new states and rewards based on the actions. The state information of the UAV and satellite includes: the orbital height, observation angle and resolution information of the satellite, as well as the flight time, range and resolution information of the UAV; the target information is the result of information source fusion;

[0082] Based on the task planning mechanism, the PPO method is used to perform dynamic rolling task planning on the planning strategy generation model to learn to maximize the cumulative reward. During the task cycle, a heterogeneous system is used to observe the target in real time to achieve dynamic decision-making and optimize the task allocation in the monitoring process.

[0083] Furthermore, the above S4 may also include:

[0084] The state information and target information of the drone and satellite are obtained from the environment. The discrete and continuous Actor networks of the planning strategy generation model take actions, outputting actions on continuous airborne and actions in discrete space respectively. The environment feeds back the next state and a positive or negative reward based on this action. The Critic network of the planning strategy generation model trains a state value function to calculate the value used to update the actor network. The satellite is used as an intelligent agent to make decisions in the current state, change to the next state and obtain benefits. The Markov decision process model M is:

[0085] M={S,A,P,R,γ}

[0086] Among them, S is a finite state set, A is an action set, P is a state transition probability matrix, R is a profit equation, γ is a discount factor, γ∈[0,1], which is used to adjust the importance of future rewards in long-term decision optimization;

[0087] In the satellite integrated mission planning problem, the finite state set S includes the task set and the satellite state, then:

[0088] S=Task,SAT,Drone

[0089] Among them, Task is the observation task information, which includes target information, Drone is the drone information, and SAT is the satellite status information;

[0090] The action set is a finite decision variable. Here, the action set A is designed as:

[0091] A=No.ID

[0092] Among them, No is the mission number selected for this action, and ID is the ID of the satellite / UAV selected in this mission.

[0093] The technical solution provided by the above embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0094] like Figure 3 As shown, the satellite-UAV collaborative mission planning method for time-sensitive targets provided by the above embodiment of the present invention mainly includes the following technical contents and technical difficulties:

[0095] 1. Research on the multi-scale planning mechanism of heterogeneous resources under complex constraint coupling: Due to the multi-type of satellites and drones, the constraints in this problem are complex and coupled. It is necessary to study how to effectively perform task allocation for multi-type satellites under complex constraints. In order to solve this difficulty, in some preferred implementations, an adaptive dynamic grouping strategy is used to dispatch tasks in the task planning stage of S1 above. In this process, satellites and drones are divided into different groups according to the task type and payload characteristics to optimize the configuration and utilization of observation resources. Satellites / drones are grouped using a dynamic classification method based on a decision tree algorithm. The decision tree algorithm is a typical classification method. First, the data is processed, and readable rules and decision trees are generated using an inductive algorithm. Then, the new data is analyzed using decisions. In the process of generating decision numbers, dynamic grouping of different tasks and scenarios is performed and allocation decisions are made based on various factors such as satellite payload type, resolution, orbit, sub-satellite point position, drone range, and payload type. Specifically:

[0096] Define input nodes: The input data is task type and resource characteristics. These input information serve as the starting conditions for decision tree classification.

[0097] Decision node: At the node of the decision tree, a decision is made by comparing the mission characteristics and resource characteristics to allocate satellite or UAV resources.

[0098] Classification node: At each final node, the system determines the grouping and task allocation results of satellites or drones based on the input mission requirements and rules. Each branch path of the decision tree represents a resource selection decision under specific mission conditions. The role of the decision tree is to dynamically adjust the grouping of satellites and drones according to the rules through the analysis of mission data, and provide automated decision support in the mission planning process.

[0099] According to the above information, the information entropy H(X) is calculated. The specific calculation formula is:

[0100]

[0101] P(x) represents the proportion of samples of category x. The larger the information entropy of a feature, the more information it contains; the smaller the information entropy, the less information it contains and the higher the purity. After calculating the information entropy of each factor in the decision tree, the information gain calculation is performed. In the information gain calculation, the sample set is divided by the rule factor, and the difference in information entropy before and after the division is calculated, which is the information gain. The specific formula is as follows:

[0102]

[0103] Information gain represents the purity difference before and after the division according to a certain feature, where H(X) represents the information entropy of the sample before the division. Indicates the number of samples of feature D belonging to a certain attribute, n X Indicates the number of samples before division. Logically, the samples with large information gain value are selected for division to form a decision tree and complete the grouping.

[0104] The grouping rules include the following two points:

[0105] Task type rules: Group tasks according to their characteristics. Different tasks have different resolution, coverage, and response time requirements. Surveillance tasks require high-resolution images, while tracking tasks require long-term continuous observation. The location of the observation resource determines the visible information of the task target, and the location of the observation resource also needs to be considered.

[0106] Payload characteristics: According to the observation resources, satellites and drones have different observation resource characteristics. Satellites have different orbital altitudes, revisit periods, observation angles, and resolutions; drones have different flight times and observation flexibility. These characteristics determine the applicable scenarios and mission types of resources.

[0107] The decision tree is a rule-based hierarchical classification model that can dynamically allocate task resources. In the satellite UAV collaborative mission planning, the decision tree in this embodiment includes:

[0108] Input node: The input data is the task type and resource characteristics. These input information serve as the starting conditions for decision tree classification.

[0109] Decision node: At the node of the decision tree, a decision is made by comparing the mission characteristics and resource characteristics to allocate satellite or UAV resources.

[0110] Classification node: At each final node, the system determines the grouping and task allocation results of satellites or drones based on the input mission requirements and rules. Each branch path of the decision tree represents a resource selection decision under specific mission conditions. The role of the decision tree is to dynamically adjust the grouping of satellites and drones according to the rules through the analysis of mission data, and provide automated decision support in the mission planning process.

[0111] 2. Research on heterogeneous collaborative modes between drones and satellites: There are obvious differences between drones and satellite systems in terms of functions, performance and task execution methods. It is necessary to study the establishment of a satellite-drone heterogeneous system and effectively integrate the operating modes of satellites and drones; in addition, for the target search and identification problems of time-sensitive targets, building an effective collaborative search mechanism and developing an optimization algorithm based on a planning model are one of the core research contents. In order to solve this difficulty, in some preferred implementations, the above S2 generates collaborative scheduling and task allocation strategies based on a meta-heuristic collaborative scheduling method during the task collaboration process, and uses a multi-objective genetic algorithm to set collaborative strategies and scheduling methods. The multi-objective genetic algorithm is an optimization technology based on the principles of natural selection and genetics, which solves optimization problems with multiple conflicting objectives by simulating the natural evolution process. In this embodiment, the planning objectives include maximizing collaborative effectiveness, maximizing coverage, minimizing task completion time, minimizing energy consumption and other objectives. The design of planning objectives is mainly used for population fitness calculation in the iterative process. The collaborative scheduling strategy based on multi-objective genetic algorithm can not only improve the overall efficiency and response speed of the UAV and satellite monitoring system, but also optimize resource utilization and enhance the adaptability and robustness of the system while meeting complex and changing task requirements. In the genetic algorithm, a multi-layer gene design is adopted to synchronously plan the task table and the UAV path. Finally, through the multi-objective optimization function, the frontier solution is iteratively found, and the multi-objective optimization results are output according to different planning requirements. Figure 4 As shown in the figure, specifically: design a random population initialization algorithm to randomly generate an initial population, where the population initialization is set to adopt a multi-layer gene design, synchronously plan the collaborative strategy, the constellation task allocation and the UAV path, and finally iterate through the multi-objective optimization function to find the frontier solution, and output the multi-objective optimization results according to different planning requirements. In collaborative scheduling, the chromosome is designed to be multi-layered, namely the scheduling layer and the path planning layer. It represents the scheduling order of tasks and which tasks are performed by satellites and which tasks are performed by UAVs during the scheduling process. Path planning layer: represents the flight path of the UAV and the specific flight route planning of each UAV.

[0112] After generating the initial population, fitness evaluation is performed, mainly considering the four goals of maximizing synergy, maximizing coverage, minimizing task completion time, and minimizing energy consumption. Next, chromosome operations are performed, including selection, crossover, and mutation. When the iteration requirements are met, the planning results are output.

[0113] 3. Research on identification methods based on multi-source information fusion: Currently, there is limited research on the fusion of observation data of different spectral bands and different resolutions, and it is necessary to study the identification algorithm for multi-source data such as satellites and drones. In order to solve this difficulty, in some preferred implementations, in the heterogeneous multi-source data fusion and identification process of S3 above, multi-scale transformation methods such as wavelet transform are used to fuse images from different sources, retaining the detailed features of the image while enhancing the target performance. In the feature fusion process, the features of the image are extracted and a comprehensive feature representation is formed to enhance the complementary advantages of different image sources, and image information from different times and angles is fused to improve the accuracy of target positioning. Use open source image recognition models such as convolutional neural networks (CNN), regional convolutional neural networks (R-CNN), YOLO, etc. And use transfer learning to use pre-trained models to accelerate the training process and improve the generalization ability of the model during the recognition process.

[0114] 4) Research on dynamic monitoring methods for time-sensitive targets: Time-sensitive targets move at a fast speed and their moving directions are highly uncertain. In the dynamic monitoring process, the difficulty of the research is to realize the closed-loop planning process through the continuous scheduling tracking and information feedback of multi-source heterogeneous information sources; the research is to build a timeliness and real-time feedback mechanism and the real-time scheduling problems brought about by high dynamic uncertainty models and complex collaborative modes. In order to solve this difficulty, the deep reinforcement learning method is used to generate planning strategies in the continuous scheduling tracking of heterogeneous information sources in the above S4. Deep reinforcement learning allows the agent to learn how to achieve goals through interaction with the environment. The agent receives drone and satellite information, target information, and selects actions based on the current strategy. The environment gives new states and rewards based on the actions. The goal of the agent is to maximize the cumulative reward by learning strategies. Deep reinforcement learning uses deep neural networks to approximate the value function or policy function in reinforcement learning. It can learn the optimal strategy in a complex environment, has strong dynamics and high robustness, and is suitable for the monitoring process of time-sensitive targets. The PPO method is used for dynamic rolling task planning. During the task cycle, a heterogeneous system is used to observe the target in near real time, train the intelligent agent to make dynamic decisions and optimize the task allocation during the monitoring process. Figure 4As shown, specifically: the satellite state, drone state and target information are states obtained from the environment, the discrete and continuous Actor networks take actions, and output continuous airborne actions and actions in discrete spaces respectively. The environment feeds back the next state and a positive and negative reward based on this action. The Critic network trains a state value function to calculate the value used to update the actor network. The intelligent agent makes decisions and gains benefits by interacting with the dynamic environment, thereby learning to make the best decisions. The Markov decision process model is the basis of reinforcement learning. In this embodiment, the satellite acts as an intelligent agent, makes decisions in the current state, changes to the next state and gains benefits. The Markov decision process model M is:

[0115] M={S,A,P,R,γ}

[0116] Among them, S is a finite state set, A is an action set, P is a state transition probability matrix, R is a profit equation, γ is a discount factor, γ∈[0,1], which is used to adjust the importance of future rewards in long-term decision optimization.

[0117] In the satellite integrated mission planning problem, the finite state set S includes the task set and satellite state.

[0118] S=Task,SAT,Drone

[0119] Among them, Task is the observation task information, Drone is the drone information, and SAT is the satellite status information.

[0120] The action set is a finite decision variable. Here, the action set A is designed as:

[0121] A=No.ID

[0122] Among them, No is the mission number selected for this action, and ID is the ID of the satellite / UAV selected in this mission.

[0123] Through the above steps, the satellite-UAV collaborative task planning method for time-sensitive targets provided by the above embodiment of the present invention realizes a closed-loop observation process for time-sensitive targets by a multi-source heterogeneous observation system based on the feedback of continuous scheduling results to the upper-level task planning mechanism.

[0124] Based on the same inventive concept, an embodiment of the present invention also provides a satellite-UAV collaborative mission planning system for time-sensitive targets.

[0125] Specifically, Figure 5 As shown, the satellite-UAV collaborative mission planning system for time-sensitive targets provided by this embodiment, based on a hierarchical scheduling planning framework, may include the following modules:

[0126] A mission planning module, which is used to provide a mission planning mechanism, using which satellites and drones are grouped, and then search and discovery missions and / or positioning and identification missions are assigned to observation resources in different groups;

[0127] Task planning and scheduling module, which is used to coordinate the scheduling of grouped satellites and drones; after completing the coordinated scheduling, for the satellite constellation, the observation tasks are assigned to the corresponding satellites for execution; for the drone cluster, the drone path is planned;

[0128] Multi-source heterogeneous information source fusion identification module, which is used to obtain observation images of the same area from different satellites and drones, fuse and identify image information at different times and angles in the observation images, complete identification and positioning, and obtain information source fusion results;

[0129] A closed-loop module for the continuous scheduling, tracking and planning of heterogeneous observation resources. This module dynamically monitors time-sensitive targets, continues to dispatch tasks based on the information source fusion results and the task planning mechanism, and realizes a real-time closed-loop satellite-UAV collaborative task planning for time-sensitive targets.

[0130] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0131] An embodiment of the present invention further provides a computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor can be used to execute any method of the above-mentioned embodiments of the present invention, or to run any system of the above-mentioned embodiments of the present invention.

[0132] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0133] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0134] The processor is used to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0135] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0136] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can be used to execute any method of the above embodiments of the present invention, or to run any system of the above embodiments of the present invention.

[0137] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0138] All matters not covered in the above embodiments of the present invention are well known in the art.

[0139] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A satellite-UAV collaborative mission planning method for time-sensitive targets, characterized in that: include: Provide a task planning mechanism, use the task planning mechanism to group satellites and drones, and then assign search and discovery tasks and / or positioning and identification tasks to observation resources in different groups; Coordinated scheduling of grouped satellites and drones; after coordinated scheduling, for satellite constellations, observation tasks are assigned to corresponding satellites for execution; for drone clusters, drone paths are planned; Obtain observation images of the same area from different satellites and drones respectively, fuse and identify image information at different times and angles in the observation images, complete identification and positioning, and obtain information source fusion results; Dynamic monitoring is carried out for time-sensitive targets, and tasks are continuously dispatched based on the information source fusion results and the task planning mechanism, so as to realize a closed loop of satellite-UAV collaborative task planning for time-sensitive targets in real time.

2. The satellite-UAV collaborative mission planning method for time-sensitive targets according to claim 1 is characterized in that: The present invention provides a task planning mechanism, which is used to group satellites and drones, and then assign search and discovery tasks and / or positioning and identification tasks to observation resources in different groups, including: The task planning mechanism includes: Process the target task data required for planning, and use the induction algorithm to generate readable rules and rule-based decision trees; Using the rules, dynamically adjust the grouping of satellites and drones; By using the decision tree, task allocation decisions for different tasks and / or different scenarios are made according to task types and observation resource characteristics, and search and discovery tasks and / or positioning and identification tasks are respectively assigned to different groups of observation resources.

3. The satellite-UAV collaborative mission planning method for time-sensitive targets according to claim 1, characterized in that: The grouped satellites and drones are coordinated and scheduled; after the coordinated scheduling is completed, the observation tasks are assigned to the corresponding satellites for execution for the satellite constellation; For drone swarms, drone paths are planned, including: A multi-objective genetic algorithm based on meta-heuristics is used to formulate a satellite constellation and UAV cluster coordination strategy for coordinated scheduling of grouped satellites and UAVs, a constellation task allocation strategy for assigning observation tasks to corresponding satellites, and a single-person or multi-person UAV path coordination planning strategy for planning UAV paths; wherein the multi-objective genetic algorithm based on meta-heuristics includes: Set planning goals, including: maximizing synergy, maximizing coverage, minimizing task completion time, and minimizing energy consumption; Using multi-layer genetic design, the satellite constellation and UAV cluster coordination strategy, the constellation task allocation strategy and the UAV path coordination planning strategy are planned synchronously to generate the initial population; Based on the planning goal, performing fitness evaluation on the initial population; The evaluation process is iterated using a multi-objective optimization function to find the frontier solution. When the iteration requirements are met, the multi-objective optimization planning results are output.

4. The satellite-UAV collaborative mission planning method for time-sensitive targets according to claim 1, characterized in that: The method of respectively acquiring observation images of the same area from different satellites and drones, fusing and distinguishing image information at different times and angles in the observation images, and completing identification and positioning includes: A pre-trained image recognition model is used to extract features of observation images of the same area from different satellites and drones and form a comprehensive feature representation to obtain multi-scale transformed image information from different times and angles; The image information from different times and angles is fused to obtain an information source fusion result.

5. The satellite-UAV collaborative mission planning method for time-sensitive targets according to claim 1, characterized in that: The dynamic monitoring of time-sensitive targets and the continued task dispatching based on the information source fusion results and the task planning mechanism include: Provide a planning strategy generation model based on deep reinforcement learning; A model is generated based on the planning strategy, and the state information and target information of the UAV and satellite are used as the input of the model. The model selects actions, and the environment gives new states and rewards according to the actions. The state information of the UAV and satellite includes: the orbital height, observation angle and resolution information of the satellite, and the flight time, range and resolution information of the UAV. The target information is the fusion result of information sources. Based on the task planning mechanism, the PPO method is used to perform dynamic rolling task planning on the planning strategy generation model to learn to maximize the cumulative reward; during the task cycle, a heterogeneous system is used to observe the target in real time to achieve dynamic decision-making and optimize the task allocation in the monitoring process.

6. The satellite-UAV collaborative mission planning method for time-sensitive targets according to claim 5, characterized in that: Also includes: The state information and target information of the drone and satellite are obtained from the environment. The discrete and continuous Actor networks of the planning strategy generation model take actions, outputting actions on continuous airborne and actions in discrete space respectively. The environment feeds back the next state and a positive and negative reward based on this action. The Critic network of the planning strategy generation model trains a state value function to calculate the value used to update the actor network. The satellite is used as an intelligent agent to make decisions in the current state, transition to the next state and obtain benefits. The Markov decision process model M is: M={S,A,P,R,γ} Among them, S is a finite state set, A is an action set, P is a state transition probability matrix, R is a profit equation, γ is a discount factor, γ∈[0,1], which is used to adjust the importance of future rewards in long-term decision optimization; In the satellite integrated mission planning problem, the finite state set S includes the task set and the satellite state, then: S=Task,SAT,Drone Among them, Task is the observation task information, which includes target information, Drone is the drone information, and SAT is the satellite status information; The action set is a finite decision variable. Here, the action set A is designed as: A=No,ID Among them, No is the mission number selected for this action, and ID is the ID of the satellite / UAV selected in this mission.

7. A satellite-UAV collaborative mission planning system for time-sensitive targets, characterized in that: include: A task planning module, which is used to provide a task planning mechanism, using which satellites and drones are grouped, and then search and discovery tasks and / or positioning and identification tasks are assigned to observation resources in different groups; Task planning and scheduling module, which is used to coordinate the scheduling of grouped satellites and drones; after completing the coordinated scheduling, for the satellite constellation, the observation tasks are assigned to the corresponding satellites for execution; for the drone cluster, the drone path is planned; A multi-source heterogeneous information source fusion identification module is used to obtain observation images of the same area from different satellites and drones, fuse and identify image information at different times and angles in the observation images, complete identification and positioning, and obtain information source fusion results; A closed-loop module for the continuous scheduling, tracking and planning of heterogeneous observation resources is used to dynamically monitor time-sensitive targets, and to continue to dispatch tasks based on the information source fusion results and the task planning mechanism, thereby realizing a real-time closed-loop satellite-UAV collaborative task planning for time-sensitive targets.

8. A computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it can be used to perform the method described in any one of claims 1 to 6, or to run the system described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can be used to perform the method described in any one of claims 1 to 6, or to run the system described in claim 7.

Images