Aerial remote sensing multi-platform collaborative observation task planning and optimizing method and system

Abstract

The invention provides an aerial remote sensing multi-platform collaborative observation task planning and optimization method and system, and relates to the technical field of collaborative planning, and the method comprises the steps: receiving the observation demand information of a target region and the resource information of a plurality of aerial remote sensing platforms, building a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm, generating an initial task planning scheme by utilizing the constructed multi-platform collaborative observation task model, optimizing the initial task planning scheme by adopting an improved multi-target particle swarm optimization algorithm, and distributing corresponding specific execution instructions to each aerial remote sensing platform through a security encrypted communication link; state information, including positions, postures, energy levels and equipment working states, fed back by the aerial remote sensing platforms is received and processed in real time, and a current task planning scheme is dynamically adjusted according to the real-time state information based on a federated learning framework by utilizing edge nodes and cloud cooperative computing capability, so that adaptive optimization of tasks is realized.

Description

Technical Field

[0001] The present invention relates to collaborative planning technology, and in particular to a method and system for planning and optimizing collaborative observation tasks of multiple aerial remote sensing platforms. Background Art

[0002] Aerial remote sensing multi-platform collaborative observation is an advanced earth observation method that can quickly obtain large-scale, multi-scale, and multi-source information. It has wide applications in environmental monitoring, resource surveys, disaster assessment, and other fields. With the development of aerial remote sensing technology, multi-platform collaborative observation tasks are becoming increasingly complex, and traditional manual planning methods can no longer meet the requirements of efficient, accurate, and flexible tasks. Existing automated task planning methods lack the ability to optimize the overall complex multi-platform system, and it is difficult to fully utilize the advantages of each platform to achieve synergy. In addition, task planning is often static and cannot be dynamically adjusted according to real-time conditions, resulting in a lack of flexibility in the face of emergencies. Communication constraints and data security issues between platforms are not fully considered, and there may be information leakage risks in actual applications. The computing architecture is usually centralized, prone to single point failures, and difficult to cope with large-scale distributed scenarios. Complex optimization problems with multiple platforms, multiple objectives, and multiple constraints.

[0003] Therefore, there is an urgent need for an intelligent, dynamic and safe aerial remote sensing multi-platform collaborative observation task planning and optimization method to improve the overall performance and adaptability of the system. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for planning and optimizing an aerial remote sensing multi-platform collaborative observation task, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention,

[0006] A method for planning and optimizing aerial remote sensing multi-platform collaborative observation tasks is provided, including:

[0007] Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and build a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion degree of the observation task and the resource utilization efficiency as a reward function. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and load configuration of each aerial remote sensing platform;

[0008] The initial mission planning scheme is optimized by using an improved multi-objective particle swarm optimization algorithm. The optimization objectives include observation coverage, timeliness, energy consumption, and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform. The specific execution instructions include flight control instructions, sensor parameter settings, and data transmission strategies. The corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link.

[0009] It receives and processes the status information fed back by various aerial remote sensing platforms in real time, including location, attitude, energy level and equipment working status. Based on the federated learning framework, it utilizes the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to real-time status information to achieve adaptive optimization of the mission.

[0010] In an optional embodiment,

[0011] The initial mission planning scheme generated by using the constructed multi-platform collaborative observation mission model includes:

[0012] Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, wherein the observation demand information includes the geographic coordinates of the target area, the expected observation time window, the spatial resolution requirement and the observation frequency requirement, and the resource information includes the current position, maximum speed, maximum range, the type of sensors carried and the sensor parameters and the remaining energy of each aerial remote sensing platform;

[0013] Construct a multi-platform collaborative observation task model, which uses a deep Q network as the core algorithm, regards each aerial remote sensing platform as an independent intelligent agent, and the state space of the intelligent agent includes the position of the aerial remote sensing platform, the remaining energy and the current task completion, the action space includes the flight direction, speed adjustment and sensor switch, and the reward function includes the observation task completion and resource utilization efficiency. The observation task completion is calculated by the ratio of the covered area to the target area, and the resource utilization efficiency is calculated by energy consumption and sensor usage time.

[0014] The multi-platform collaborative observation task model is trained by using an experience replay mechanism and a target network technology. The experience replay mechanism randomly samples from historical experience for learning, and the target network improves the convergence performance of the algorithm by updating parameters according to a preset period.

[0015] Creating a simulated environment that incorporates weather changes, obstacles, and communication delays, and generating training data in the simulated environment;

[0016] The training data is used to train the multi-platform collaborative observation task model using an experience replay mechanism and a target network technology. The experience replay mechanism learns by random sampling from past experience. The target network improves the convergence performance of the algorithm by regularly updating parameters to obtain a trained multi-platform collaborative observation task model. The trained multi-platform collaborative observation task model is used to generate an initial task planning plan.

[0017] In an optional embodiment,

[0018] Creating a simulated environment that includes weather changes, obstacles, and communication delays, and generating training data in the simulated environment includes:

[0019] Use open source map data to build a basic geographic environment, convert vector map data into rasterized elevation models and surface classification data through a geographic data processing library to obtain basic geographic environment data, and integrate dynamic weather models based on the basic geographic environment data to generate high-resolution meteorological field data including wind speed, wind direction, temperature and humidity;

[0020] Using the basic geographic environment data and the high-resolution meteorological field data, constructing an obstacle system, including extracting static obstacle information from the basic geographic environment data, generating dynamic obstacle information through a random generation algorithm, and constructing a dynamic obstacle motion model to obtain complete obstacle system data;

[0021] Inputting the basic geographic environment data, the high-resolution meteorological field data and the complete obstacle system data into a communication network simulator preset for an aviation remote sensing scenario, simulating the influence of terrain and atmospheric conditions on signal propagation, and generating multiple communication protocol simulation data;

[0022] According to the preset mission scenario, the basic geographic environment data, the high-resolution meteorological field data, the complete obstacle system data and the multiple communication protocol simulation data are used to initialize the simulation environment and randomly generate the initial positions and state parameters of multiple aerial remote sensing platforms;

[0023] A basic controller is used to simulate the initial behaviors of multiple aerial remote sensing platforms, record environmental states, platform actions and reward values, generate original training data, adjust parameters and transform scenes for the original training data, randomly change meteorological parameters, obstacle distribution and communication conditions, generate diverse training samples, build a parallel computing framework to process the diverse training samples, and generate a final training data set, in which each piece of training data contains environmental states, platform actions, immediate rewards and next state information.

[0024] In an optional embodiment,

[0025] The improved multi-objective particle swarm optimization algorithm is used to optimize the initial mission planning scheme. The optimization objectives include observation coverage, timeliness, energy consumption and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, including:

[0026] Constructing a multi-objective optimization model, wherein the multi-objective optimization model includes an observation coverage objective function, a timeliness objective function, an energy consumption objective function, and a platform load balancing objective function;

[0027] By calculating the adaptive inertia weight based on the distance between the current position of the particle and the global optimal position, dynamically adjusting the learning factor according to the number of iterations, and using the non-dominated solution sorting strategy based on the crowding distance to solve the multi-objective optimization model, an optimized task planning scheme is obtained;

[0028] Input the optimized task planning scheme into the edge computing architecture for parsing, obtain subtasks of each aerial remote sensing platform, perform local path planning on the subtasks of each aerial remote sensing platform, and obtain local path planning results;

[0029] According to the characteristics of the observed target and environmental conditions, the sensor working mode, resolution and sampling frequency are dynamically adjusted using a fuzzy logic controller to obtain sensor parameter settings, and an adaptive data transmission strategy is generated based on the predicted communication quality and data priority.

[0030] The local path planning result, the sensor parameter setting and the adaptive data transmission strategy are packaged into an execution instruction packet in a standard format.

[0031] In an optional embodiment,

[0032] According to the characteristics of the observed target and environmental conditions, the fuzzy logic controller is used to dynamically adjust the sensor working mode, resolution and sampling frequency to obtain the sensor parameter settings. Based on the predicted communication quality and data priority, the adaptive data transmission strategy is generated, including:

[0033] Acquiring target characteristics and environmental conditions, wherein the target characteristics include target size and target moving speed, and the environmental conditions include atmospheric visibility and cloud cover;

[0034] The target characteristics and the environmental conditions are fuzzified, the target size is divided into three fuzzy sets of small, medium and large, and the target moving speed, atmospheric visibility and cloud cover are described using a trapezoidal membership function to perform fuzzy set division to obtain the fuzzified target characteristics and environmental conditions;

[0035] Based on a preset fuzzy rule base, a fuzzy reasoning method is used to reason the fuzzified target features and environmental conditions to obtain a fuzzy reasoning result;

[0036] Defuzzifying the fuzzy inference result by using the centroid method to obtain sensor parameter settings, wherein the sensor parameter settings include a sensor working mode, a sensor resolution, and a sensor sampling frequency, adjusting the working state of the sensor according to the sensor parameter settings, performing an observation task, and acquiring observation data;

[0037] Collect historical throughput, historical packet loss rate, current signal-to-noise ratio, and current network congestion, and input these data into a pre-trained recurrent neural network model to predict future communication quality;

[0038] The observed data is prioritized, data timeliness, data integrity, target importance, and data scarcity are considered, a data priority score is calculated using a weighted sum method, and an adaptive data transmission strategy is generated based on the predicted future communication quality and the data priority score, including determining available transmission bandwidth, selecting a transmission protocol, and allocating bandwidth resources.

[0039] In an optional embodiment,

[0040] Receive and process status information fed back by each aerial remote sensing platform in real time, including location, attitude, energy level and equipment working status. Based on the federated learning framework, use the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning scheme according to real-time status information to achieve adaptive optimization of the task, including:

[0041] Collect status information of multiple aerial remote sensing platforms in real time, the status information including location information, attitude information, energy level and equipment working status, pre-process the status information, deploy edge computing nodes on each of the aerial remote sensing platforms, the edge computing nodes including a lightweight neural network model; use the lightweight neural network model to extract features from the pre-processed status information to obtain a low-dimensional feature vector;

[0042] Constructing a federated learning framework, the federated learning framework comprising a local model deployed on the aerial remote sensing platform and a global model deployed on the cloud, training the local model based on the low-dimensional feature vector, encrypting the parameters of the local model using homomorphic encryption technology, and uploading the encrypted local model parameters to the cloud;

[0043] Using a federated averaging algorithm in the cloud to perform weighted averaging on the encrypted parameters of each local model, updating the global model, generating a new mission planning scheme based on the updated global model, and sending the new mission planning scheme to each aerial remote sensing platform;

[0044] Each aerial remote sensing platform adjusts its own task execution strategy according to the received new task planning scheme, and optimizes the allocation of computing resources and sensor resources according to the energy level and equipment status of each aerial remote sensing platform to obtain an optimized resource allocation scheme;

[0045] The updated task priority and optimized resource allocation plan are used as new input to achieve real-time processing of the aerial remote sensing platform status and continuous adaptive optimization of tasks.

[0046] According to a second aspect of the embodiments of the present invention,

[0047] Provide an aerial remote sensing multi-platform collaborative observation mission planning and optimization system, including:

[0048] The first unit is used to receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and construct a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion degree of the observation task and the resource utilization efficiency as a reward function. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and load configuration of each aerial remote sensing platform;

[0049] The second unit is used to optimize the initial mission planning scheme by using an improved multi-objective particle swarm optimization algorithm, wherein the optimization objectives include observation coverage, timeliness, energy consumption, and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, wherein the specific execution instructions include flight control instructions, sensor parameter settings, and data transmission strategies, and the corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link;

[0050] The third unit is used to receive and process the status information fed back by each aerial remote sensing platform in real time, including position, attitude, energy level and equipment working status. Based on the federated learning framework, it uses the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to real-time status information to achieve adaptive optimization of the mission.

[0051] According to a third aspect of the embodiments of the present invention,

[0052] An electronic device is provided, comprising:

[0053] processor;

[0054] a memory for storing processor-executable instructions;

[0055] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0056] A fourth aspect of the embodiments of the present invention is:

[0057] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0058] In this embodiment, intelligent collaboration and adaptive optimization of multiple aerial remote sensing platforms are achieved, significantly improving the efficiency and quality of observation tasks. The deep reinforcement learning algorithm enables the system to autonomously learn and optimize complex multi-platform collaboration strategies to adapt to changing observation needs and environmental conditions. The improved multi-objective particle swarm optimization algorithm achieves a balance among multiple key indicators and improves the overall task performance. The edge computing architecture reduces the communication burden and improves the system response speed. Secure and encrypted communications ensure the confidentiality and integrity of task instructions. The federated learning framework achieves collaborative learning and model optimization on multiple platforms while protecting data privacy. The real-time adaptive adjustment mechanism enables the system to respond quickly to emergencies, ensures the continuity and reliability of the task, and greatly improves the intelligence level, resource utilization efficiency and task completion quality of the aerial remote sensing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a process flow of a method for planning and optimizing an aerial remote sensing multi-platform collaborative observation task according to an embodiment of the present invention;

[0060] Figure 2 It is a schematic diagram of the structure of the aerial remote sensing multi-platform collaborative observation task planning and optimization system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0063] Figure 1 FIG. 1 is a flow chart of a method for planning and optimizing a multi-platform collaborative observation task for aerial remote sensing according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0064] S101. Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and construct a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion of the observation task and the resource utilization efficiency as reward functions. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and payload configuration of each aerial remote sensing platform.

[0065] Among them, the observation demand information of the target area includes geographic coordinates, expected observation time window, spatial resolution requirements and observation frequency requirements. This information provides basic data for mission planning to ensure that the required observations can be carried out effectively. At the same time, resource information of multiple aerial remote sensing platforms is collected, including their current location, maximum speed, maximum range, sensor types and sensor parameters, remaining energy, etc. These data help evaluate the capabilities and limitations of each platform.

[0066] In an optional embodiment,

[0067] The initial mission planning scheme generated by using the constructed multi-platform collaborative observation mission model includes:

[0068] Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, wherein the observation demand information includes the geographic coordinates of the target area, the expected observation time window, the spatial resolution requirement and the observation frequency requirement, and the resource information includes the current position, maximum speed, maximum range, the type of sensors carried and the sensor parameters and the remaining energy of each aerial remote sensing platform;

[0069] Construct a multi-platform collaborative observation task model, which uses a deep Q network as the core algorithm, regards each aerial remote sensing platform as an independent intelligent agent, and the state space of the intelligent agent includes the position of the aerial remote sensing platform, the remaining energy and the current task completion, the action space includes the flight direction, speed adjustment and sensor switch, and the reward function includes the observation task completion and resource utilization efficiency. The observation task completion is calculated by the ratio of the covered area to the target area, and the resource utilization efficiency is calculated by energy consumption and sensor usage time.

[0070] The multi-platform collaborative observation task model is trained by using an experience replay mechanism and a target network technology. The experience replay mechanism randomly samples from historical experience for learning, and the target network improves the convergence performance of the algorithm by updating parameters according to a preset period.

[0071] Creating a simulated environment that incorporates weather changes, obstacles, and communication delays, and generating training data in the simulated environment;

[0072] The training data is used to train the multi-platform collaborative observation task model using an experience replay mechanism and a target network technology. The experience replay mechanism learns by random sampling from past experience. The target network improves the convergence performance of the algorithm by regularly updating parameters to obtain a trained multi-platform collaborative observation task model. The trained multi-platform collaborative observation task model is used to generate an initial task planning plan.

[0073] Exemplarily, first, the observation demand information of the target area and the resource information of multiple aerial remote sensing platforms are received. The observation demand information includes the geographic coordinates of the target area, the expected observation time window, the spatial resolution requirements, and the observation frequency requirements. For example, the target area is a rectangular area between 30° and 31° north latitude and 114° and 115° east longitude, the expected observation time window is from 10:00 to 16:00 on July 1, 2023, the spatial resolution requirement is 1 meter, and the observation frequency requirement is once per hour. The resource information includes the current position, maximum speed, maximum range, sensor type and sensor parameters, and remaining energy of each aerial remote sensing platform. For example, the current position of an aerial remote sensing platform is 29.5° north latitude and 113.5° east longitude, with a maximum speed of 200 km / h and a maximum range of 1,000 km. It is equipped with optical sensors and infrared sensors, and the remaining energy can support 6 hours of flight.

[0074] Next, a multi-platform collaborative observation task model is constructed. This model uses a deep Q network as the core algorithm and regards each aerial remote sensing platform as an independent intelligent agent. The state space of the intelligent agent includes the location of the aerial remote sensing platform, the remaining energy, and the current task completion. The action space includes the flight direction, speed adjustment, and sensor switching. The reward function includes the observation task completion and resource utilization efficiency. The observation task completion is calculated by the ratio of the covered area to the target area. For example, if 80 square kilometers have been covered and the total area of ​​the target area is 100 square kilometers, the task completion is 80%. Resource utilization efficiency is calculated by energy consumption and sensor usage time. For example, if the energy consumption is 40% of the total energy and the sensor usage time is 60% of the total available time, the resource utilization efficiency can be expressed as 50%.

[0075] Then, the multi-platform collaborative observation task model is trained using the experience replay mechanism and target network technology. The experience replay mechanism randomly samples from historical experience for learning. For example, the most recent 10,000 experience data are stored and 256 are randomly selected for learning each time training. The target network improves the convergence performance of the algorithm by updating parameters according to a preset period. For example, the target network parameters are updated every 100 training iterations.

[0076] In order to improve the robustness and generalization ability of the model, a simulation environment is created that includes weather changes, obstacles, and communication delay factors. Weather changes can include parameters such as wind speed, wind direction, and visibility. For example, the wind speed range is 0-20 m / s, the wind direction is 0-360 degrees, and the visibility range is 1-20 km. Obstacles can include mountains, tall buildings, etc., which are achieved by setting no-fly zones. Communication delays can be set to random values ​​of 0-5 seconds. Training data is generated in this simulation environment, including the task execution process under different weather conditions, obstacle distributions, and communication delays.

[0077] Using the generated training data, the multi-platform collaborative observation task model is trained using the experience replay mechanism and the target network technology. The experience replay mechanism randomly samples from past experience for learning. For example, 1024 data are randomly selected from the experience pool for batch learning each time training. The target network improves the convergence performance of the algorithm by regularly updating parameters. For example, the target network parameters are updated every 1000 training iterations. During the training process, the exploration and exploitation are balanced by gradually reducing the exploration rate (e.g., from the initial 0.9 to 0.1).

[0078] After the training is completed, the trained multi-platform collaborative observation task model is used to generate the initial mission planning scheme. The specific steps are as follows: First, the target area is divided into several sub-areas. For example, a target area of ​​100 square kilometers is divided into 100 sub-areas of 1 square kilometer. Then, according to the current status and mission requirements of each aerial remote sensing platform, a series of actions are generated for each platform using the trained model, including flight paths, speed adjustments, and sensor usage plans. Finally, the action sequences of each platform are combined into a complete mission planning scheme, including the flight route, observation schedule, and sensor usage plan of each platform.

[0079] In this embodiment, through the deep reinforcement learning method, the system can automatically learn the optimal decision-making strategy in a complex environment, and can generate high-quality mission planning solutions more quickly than traditional manual planning methods. By introducing a simulated environment with factors such as weather changes, obstacles, and communication delays, the trained model can better cope with various uncertainties and interference factors in the actual mission execution process. By incorporating resource utilization efficiency into the reward function, the model can optimize energy consumption and sensor use as much as possible while completing the observation task, thereby extending the working time of the aerial remote sensing platform. By continuously learning and updating the model, the system can continuously optimize the decision-making strategy based on the actual mission execution, so that mission planning can adapt to dynamically changing environments and needs.

[0080] In an optional embodiment,

[0081] Creating a simulated environment that includes weather changes, obstacles, and communication delays, and generating training data in the simulated environment includes:

[0082] Use open source map data to build a basic geographic environment, convert vector map data into rasterized elevation models and surface classification data through a geographic data processing library to obtain basic geographic environment data, and integrate dynamic weather models based on the basic geographic environment data to generate high-resolution meteorological field data including wind speed, wind direction, temperature and humidity;

[0083] Using the basic geographic environment data and the high-resolution meteorological field data, constructing an obstacle system, including extracting static obstacle information from the basic geographic environment data, generating dynamic obstacle information through a random generation algorithm, and constructing a dynamic obstacle motion model to obtain complete obstacle system data;

[0084] Inputting the basic geographic environment data, the high-resolution meteorological field data and the complete obstacle system data into a communication network simulator preset for an aviation remote sensing scenario, simulating the influence of terrain and atmospheric conditions on signal propagation, and generating multiple communication protocol simulation data;

[0085] According to the preset mission scenario, the basic geographic environment data, the high-resolution meteorological field data, the complete obstacle system data and the multiple communication protocol simulation data are used to initialize the simulation environment and randomly generate the initial positions and state parameters of multiple aerial remote sensing platforms;

[0086] A basic controller is used to simulate the initial behaviors of multiple aerial remote sensing platforms, record environmental states, platform actions and reward values, generate original training data, adjust parameters and transform scenes for the original training data, randomly change meteorological parameters, obstacle distribution and communication conditions, generate diverse training samples, build a parallel computing framework to process the diverse training samples, and generate a final training data set, in which each piece of training data contains environmental states, platform actions, immediate rewards and next state information.

[0087] For example, first, the basic geographic environment is constructed using open source map data. OpenStreetMap is selected as the data source, and the GDAL (Geospatial Data Abstraction Library) geographic data processing library is used to convert the vector map data into a rasterized elevation model and surface classification data.

[0088] Based on the basic geographic environment data obtained, the dynamic weather model is integrated to generate high-resolution meteorological field data. The WRF (Weather Research and Forecasting) model is used as a dynamic weather simulation tool, the basic geographic environment data is input, and the initial conditions and boundary conditions are set. The simulation calculation obtains high-resolution meteorological field data including wind speed, wind direction, temperature and humidity.

[0089] Next, the obstacle system is constructed using basic geographic environment data and high-resolution meteorological field data. Static obstacle information, such as buildings and mountains, is extracted from the basic geographic environment data. Dynamic obstacle information, such as moving vehicles and aircraft, is generated using a random generation algorithm. A dynamic obstacle motion model is constructed to describe its motion trajectory and speed changes. For example, 100 dynamic obstacles can be randomly generated in the simulation area, including 50 ground vehicles (speed 0-120 km / h) and 50 drones (speed 0-100 km / h), and the motion trajectory of each obstacle is described by a Bezier curve. This information is combined to form a complete obstacle system data.

[0090] Input basic geographic environment data, high-resolution meteorological field data, and complete obstacle system data into a pre-set communication network simulator for aerial remote sensing scenarios. You can use NS-3 (Network Simulator 3) as a communication network simulator to configure wireless communication protocols suitable for aerial remote sensing applications, such as LTE, 5G, or satellite communications. The simulator considers the impact of terrain and atmospheric conditions on signal propagation and generates simulation data for multiple communication protocols. For example, you can simulate parameters such as signal strength, transmission delay, and packet loss rate of LTE and 5G communication protocols under different weather conditions.

[0091] According to the preset mission scenario, the simulation environment is initialized using the various data generated above. A typical aerial remote sensing mission scenario can be set, such as high-resolution imaging of an area of ​​100 square kilometers. In the initialized environment, the initial positions and state parameters of multiple aerial remote sensing platforms are randomly generated.

[0092] Use the basic controller to simulate the initial behavior of multiple aerial remote sensing platforms. The basic controller can use simple rule-based algorithms, such as straight-line flight or serpentine scanning. Record the environmental state, platform action and reward value to generate raw training data. The environmental state includes the platform's position, speed, energy level, surrounding obstacles and weather conditions. The platform's actions include speed changes, heading adjustments, and sensor switches. The reward value is calculated based on the task completion and resource utilization efficiency.

[0093] The original training data is parameterized and scene-changed to randomly change meteorological parameters, obstacle distribution, and communication conditions to generate diverse training samples. For example, the original wind speed can be randomly adjusted within the range of ±5 m / s, the obstacle position can be randomly offset within the range of ±100 m, and the communication delay can be randomly changed within the range of ±10 ms. In this way, the original data can be expanded 10 times to obtain richer training samples.

[0094] Build a parallel computing framework to process diverse training samples and generate the final training data set. You can use Apache Spark as a parallel computing framework to distribute data processing tasks to multiple computing nodes and perform them simultaneously. Each final training data contains environmental state, platform action, immediate reward, and next state information. For example, a training data may include: current state (platform position [x, y, z], speed v, energy level e, distance to the nearest obstacle d, current wind speed w), performed action (speed change Δv, heading change Δθ, sensor switch s), reward r obtained, and next state information.

[0095] In this embodiment, by integrating actual geographic data, dynamic weather models and complex obstacle systems, a simulation environment closer to the real world is created, which helps to train a more robust and adaptable aerial remote sensing mission planning model. Through parameter adjustment and scene transformation, a large number of diverse training samples are generated, covering a wider range of mission scenarios and environmental conditions, which is conducive to improving the generalization ability of the model. The parallel computing framework is used to process a large number of training samples, which significantly improves the data processing speed and enables a sufficient amount of high-quality training data to be generated within a limited time. Through modular design, the parameters of the geographical environment, meteorological conditions, obstacle systems and communication models can be easily adjusted to adapt to different research needs and application scenarios. By comprehensively considering multiple influencing factors, such as terrain, weather, obstacles and communication conditions, the generated training data more comprehensively reflects the various situations that may be encountered during the actual mission execution, which helps to train a more reliable mission planning model.

[0096] S102. An improved multi-objective particle swarm optimization algorithm is used to optimize the initial mission planning scheme. The optimization objectives include observation coverage, timeliness, energy consumption and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform. The specific execution instructions include flight control instructions, sensor parameter settings and data transmission strategies. The corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link.

[0097] Among them, coverage rate refers to the proportion of the target area that the platform can cover, timeliness focuses on the speed of task completion, energy consumption involves the energy consumption required to perform the task, and platform load balancing ensures that the tasks between platforms are reasonably distributed to avoid excessive concentration or waste of resources.

[0098] During the optimization process, by initializing the particle swarm, each particle represents a possible task planning scheme. Each particle is iteratively updated according to the set goals and constraints to find the optimal solution. The improved algorithm evaluates the performance of each particle by introducing a fitness evaluation mechanism, and adjusts the particle's speed and position based on the evaluation results to gradually approach the global optimal solution. This method effectively balances multiple optimization objectives and ensures that various indicators are comprehensively considered.

[0099] After the optimization is completed, the generated mission planning scheme is parsed into specific execution instructions. The specific execution instructions of each aerial remote sensing platform include flight control instructions, sensor parameter settings and data transmission strategies. Flight control instructions involve the flight path and altitude adjustment of the platform, sensor parameter settings include the sensor's on state, acquisition frequency and sensitivity, etc., and the data transmission strategy focuses on how to effectively transmit the acquired data back to the processing center.

[0100] In order to ensure the security of executing instructions, the instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link. This link uses advanced encryption technology to ensure the security and integrity of data during transmission and prevent information leakage and tampering. After receiving the instructions, each platform can efficiently execute the corresponding tasks according to the instructions to ensure that the tasks proceed smoothly as planned.

[0101] In an optional embodiment,

[0102] The improved multi-objective particle swarm optimization algorithm is used to optimize the initial mission planning scheme. The optimization objectives include observation coverage, timeliness, energy consumption and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, including:

[0103] Constructing a multi-objective optimization model, wherein the multi-objective optimization model includes an observation coverage objective function, a timeliness objective function, an energy consumption objective function, and a platform load balancing objective function;

[0104] By calculating the adaptive inertia weight based on the distance between the current position of the particle and the global optimal position, dynamically adjusting the learning factor according to the number of iterations, and using the non-dominated solution sorting strategy based on the crowding distance to solve the multi-objective optimization model, an optimized task planning scheme is obtained;

[0105] Input the optimized task planning scheme into the edge computing architecture for parsing, obtain subtasks of each aerial remote sensing platform, perform local path planning on the subtasks of each aerial remote sensing platform, and obtain local path planning results;

[0106] According to the characteristics of the observed target and environmental conditions, the sensor working mode, resolution and sampling frequency are dynamically adjusted using a fuzzy logic controller to obtain sensor parameter settings, and an adaptive data transmission strategy is generated based on the predicted communication quality and data priority.

[0107] The local path planning result, the sensor parameter setting and the adaptive data transmission strategy are packaged into an execution instruction packet in a standard format.

[0108] Exemplarily, a multi-objective optimization model is first constructed. The model includes four objective functions: observation coverage objective function, timeliness objective function, energy consumption objective function and platform load balancing objective function. The observation coverage objective function is used to quantify the coverage of the target area by the mission planning scheme, which can be expressed by calculating the ratio of the observed area to the total target area. The timeliness objective function measures the time required to complete the task, which can be expressed by calculating the total time from the start of the task to the completion of all target observations. The energy consumption objective function evaluates the energy consumption during the execution of the task, which can be calculated by accumulating the energy consumption of each platform during flight and sensor operation. The platform load balancing objective function is used to evaluate whether the distribution of tasks among the platforms is balanced, which can be expressed by calculating the variance of the task volume of each platform.

[0109] Next, an improved multi-objective particle swarm optimization algorithm is used to solve the above multi-objective optimization model. The improvements of the algorithm are mainly reflected in three aspects: First, the inertia weight is dynamically adjusted by calculating the distance between the current position of the particle and the global optimal position, so that the movement of particles in the search space is more flexible. Secondly, the learning factor is dynamically adjusted according to the number of iterations to increase the global search capability in the early stage of optimization and enhance the local search capability in the later stage. Finally, a non-dominated solution sorting strategy based on crowding distance is used to maintain and update the Pareto optimal solution set to improve the diversity and convergence of the algorithm.

[0110] Specifically, the particle swarm size can be set to 100 and the maximum number of iterations can be set to 1000. Each particle represents a feasible mission planning scheme, which contains information such as the flight path and observation time of each platform. In each iteration, the position and velocity of the particle are updated, the values ​​of each objective function are calculated, and the Pareto optimal solution set is updated according to the non-dominated relationship. Finally, a set of optimized mission planning schemes are obtained.

[0111] The optimized mission planning scheme is input into the edge computing architecture for analysis. The edge computing architecture consists of edge nodes deployed on each aerial remote sensing platform and a central control node on the ground. The central control node is responsible for decomposing the global mission planning scheme into subtasks for each platform, and the edge node is responsible for performing local planning and optimization of subtasks. For example, for a system consisting of 5 aerial remote sensing platforms, the target area of ​​100 square kilometers can be divided into 5 sub-areas, and each platform is responsible for the observation task of one sub-area.

[0112] Local path planning is performed for each subtask of the aerial remote sensing platform. The improved Rapid Exploration Random Tree (RRT*) algorithm is used for path planning, taking into account factors such as platform motion constraints, obstacle avoidance, and energy consumption. For example, for a subtask that requires observing 10 target points, an optimal path with a total length of 50 kilometers and a flight time of 30 minutes can be generated.

[0113] According to the characteristics of the observed target and environmental conditions, the fuzzy logic controller is used to dynamically adjust the sensor working mode, resolution and sampling frequency. The input of the fuzzy logic controller includes target importance, weather conditions, flight altitude, etc., and the output is the sensor parameter setting. For example, when observing important targets and weather conditions are good, the resolution of the optical sensor can be set to the highest (such as 0.1 meters) and the sampling frequency can be set to 2 times per second; when observing secondary targets or weather conditions are poor, the resolution (such as 0.5 meters) and sampling frequency (such as 1 time per second) can be reduced to save energy and storage space.

[0114] Generate an adaptive data transmission strategy based on the predicted communication quality and data priority. Use machine learning methods (such as long short-term memory network LSTM) to predict future communication quality, and dynamically adjust the data transmission priority and compression ratio based on the importance and timeliness of the data. For example, when the communication quality is predicted to be good, high-resolution important target data can be transmitted first; when the communication quality is poor, only low-resolution summary data can be transmitted, and the complete data can be stored locally and transmitted after the communication conditions improve.

[0115] Finally, the local path planning results, sensor parameter settings, and adaptive data transmission strategies are encapsulated into a standard execution instruction package. The execution instruction package uses XML format and contains route information (such as waypoint coordinates, flight altitude, speed), sensor control instructions (such as working mode, resolution, sampling frequency), and data transmission strategies (such as priority, compression ratio). For example, an execution instruction package may contain the coordinates and time information of 20 waypoints, 5 different sensor parameter settings, and 3 levels of data transmission strategies.

[0116] In this embodiment, a multi-objective optimization algorithm is used to simultaneously consider multiple objectives such as observation coverage, timeliness, energy consumption, and load balancing, and a more comprehensive and efficient task planning solution is obtained. The adaptive ability of the system is improved, and the system can better cope with complex and changing task environments by dynamically adjusting sensor parameters and data transmission strategies. The flexibility and robustness of task execution are enhanced, and the combination of global planning and local optimization is realized through the edge computing architecture, which improves the system's responsiveness to emergencies. The utilization of system resources is optimized, and the communication bandwidth and energy consumption are reduced while ensuring data quality through adaptive data transmission strategies. The degree of refinement of task execution is improved, and precise control of the aerial remote sensing platform is achieved through detailed execution instruction packages.

[0117] In an optional embodiment,

[0118] According to the characteristics of the observed target and environmental conditions, the fuzzy logic controller is used to dynamically adjust the sensor working mode, resolution and sampling frequency to obtain the sensor parameter settings. Based on the predicted communication quality and data priority, the adaptive data transmission strategy is generated, including:

[0119] Acquiring target characteristics and environmental conditions, wherein the target characteristics include target size and target moving speed, and the environmental conditions include atmospheric visibility and cloud cover;

[0120] The target characteristics and the environmental conditions are fuzzified, the target size is divided into three fuzzy sets of small, medium and large, and the target moving speed, atmospheric visibility and cloud cover are described using a trapezoidal membership function to perform fuzzy set division to obtain the fuzzified target characteristics and environmental conditions;

[0121] Based on a preset fuzzy rule base, a fuzzy reasoning method is used to reason the fuzzified target features and environmental conditions to obtain a fuzzy reasoning result;

[0122] Defuzzifying the fuzzy inference result by using the centroid method to obtain sensor parameter settings, wherein the sensor parameter settings include a sensor working mode, a sensor resolution, and a sensor sampling frequency, adjusting the working state of the sensor according to the sensor parameter settings, performing an observation task, and acquiring observation data;

[0123] Collect historical throughput, historical packet loss rate, current signal-to-noise ratio, and current network congestion, and input these data into a pre-trained recurrent neural network model to predict future communication quality;

[0124] The observed data is prioritized, data timeliness, data integrity, target importance, and data scarcity are considered, a data priority score is calculated using a weighted sum method, and an adaptive data transmission strategy is generated based on the predicted future communication quality and the data priority score, including determining available transmission bandwidth, selecting a transmission protocol, and allocating bandwidth resources.

[0125] Exemplarily, first, the target characteristics and environmental conditions are obtained. The target characteristics include the target size and the target moving speed, and the environmental conditions include the atmospheric visibility and cloud cover. This information can be obtained in a variety of ways, such as advance intelligence information, real-time observation data or feedback from other sensors. For example, for a moving target, it can be known through advance intelligence information that its size is approximately 10 meters × 5 meters × 3 meters, and its moving speed is 30 kilometers per hour through real-time radar tracking. At the same time, the current atmospheric visibility is 10 kilometers and the cloud cover is 3 / 10 through the weather station data.

[0126] Next, the target characteristics and environmental conditions are fuzzified. This step is to convert precise numerical values ​​into fuzzy language descriptions for subsequent fuzzy reasoning. For the target size, it can be divided into three fuzzy sets: small, medium, and large, and described using a trapezoidal membership function. For example, the membership function of a small target can be defined as (0, 0, 5, 10), a medium target as (5, 10, 15, 20), and a large target as (15, 20, ∞, ∞). Similarly, fuzzy set divisions are performed on the target movement speed, atmospheric visibility, and cloud cover. For example, the target movement speed can be divided into three fuzzy sets: slow, medium, and fast, the atmospheric visibility can be divided into three fuzzy sets: low, medium, and high, and the cloud cover can be divided into three fuzzy sets: few, medium, and many.

[0127] Then, based on the preset fuzzy rule base, the fuzzified target features and environmental conditions are inferred using fuzzy reasoning methods. The fuzzy rule base is a series of if-then statements that describe the relationship between input and output. For example, a rule may be "If the target size is small and the moving speed is fast, and the atmospheric visibility is high and the cloud cover is small, then the sensor resolution should be high and the sampling frequency should be high." The fuzzy reasoning process includes three steps: fuzzification, rule evaluation, and rule aggregation. In the rule evaluation stage, the minimum operator is used to calculate the trigger strength of each rule; in the rule aggregation stage, the maximum operator is used to combine the outputs of all rules.

[0128] Next, the centroid method is used to defuzzify the fuzzy inference results to obtain specific sensor parameter settings. The centroid method is a commonly used defuzzification method that obtains accurate output values ​​by calculating the centroid of a fuzzy set. Sensor parameter settings include sensor operating mode, sensor resolution, and sensor sampling frequency. For example, for the above example, the result obtained after defuzzification is: the sensor operates in active mode, the resolution is set to 0.1 meters, and the sampling frequency is set to 10 times per second. According to these parameter settings, the working state of the sensor is adjusted, the observation task is performed, and the observation data is obtained.

[0129] In order to generate an adaptive data transmission strategy, it is first necessary to predict the future communication quality. Collect data such as historical throughput, historical packet loss rate, current signal-to-noise ratio, and current network congestion, and input these data into a pre-trained recurrent neural network model. The recurrent neural network model can capture the long-term dependencies of time series data and is suitable for predicting communication quality. For example, a long short-term memory network (LSTM) can be used as a specific recurrent neural network structure, input the communication quality data of the past 24 hours, and predict the change in communication quality in the next hour.

[0130] At the same time, the observation data is prioritized. Consider factors such as data timeliness, data integrity, target importance, and data scarcity. Data timeliness reflects the real-time requirements of the data and can be evaluated based on the difference between the data generation time and the current time. Data integrity indicates the quality and availability of the data, which can be measured by the completeness and accuracy of the data. Target importance reflects the importance of the observation target in the task and can be determined based on the predefined target priority. Data scarcity indicates the uniqueness and irreplaceability of the data and can be evaluated by comparing the number of similar data. The data priority score is calculated using a weighted summation method, that is, the score of each factor is multiplied by the corresponding weight coefficient and then summed. For example, the timeliness weight can be set to 0.3, the integrity weight to 0.2, the importance weight to 0.3, and the scarcity weight to 0.2. The final priority score is calculated based on the specific scores of each factor.

[0131] Finally, based on the predicted future communication quality and data priority score, an adaptive data transmission strategy is generated. First, the available transmission bandwidth is determined. Based on the predicted communication quality, the available transmission bandwidth in the future is estimated. Then, a suitable transmission protocol is selected. For example, when the communication quality is good, the TCP protocol can be selected to ensure reliable data transmission. When the communication quality is poor, the UDP protocol can be selected to improve transmission efficiency. Finally, bandwidth resources are reasonably allocated based on the data priority score. Data with higher priority is allocated more bandwidth resources to ensure that it can be transmitted in time; data with lower priority can appropriately reduce the transmission rate or delay transmission.

[0132] In this embodiment, the fuzzy logic controller is used to effectively handle complex and uncertain environments and enhance the robustness of the system. The communication quality prediction method based on machine learning improves the system's adaptability to changes in the network environment and optimizes data transmission efficiency. The data priority mechanism based on multi-factor comprehensive evaluation ensures the timely transmission of important data and improves the task execution effect of the system. The implementation of the adaptive data transmission strategy effectively balances transmission efficiency and reliability and improves the overall performance of the system in a complex network environment.

[0133] S103. Receive and process the status information fed back by each aerial remote sensing platform in real time, including position, attitude, energy level and equipment working status. Based on the federated learning framework, utilize the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to the real-time status information to achieve adaptive optimization of the mission.

[0134] In an optional embodiment,

[0135] Receive and process status information fed back by each aerial remote sensing platform in real time, including location, attitude, energy level and equipment working status. Based on the federated learning framework, use the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning scheme according to real-time status information to achieve adaptive optimization of the task, including:

[0136] Collect status information of multiple aerial remote sensing platforms in real time, the status information including location information, attitude information, energy level and equipment working status, pre-process the status information, deploy edge computing nodes on each of the aerial remote sensing platforms, the edge computing nodes including a lightweight neural network model; use the lightweight neural network model to extract features from the pre-processed status information to obtain a low-dimensional feature vector;

[0137] Constructing a federated learning framework, the federated learning framework comprising a local model deployed on the aerial remote sensing platform and a global model deployed on the cloud, training the local model based on the low-dimensional feature vector, encrypting the parameters of the local model using homomorphic encryption technology, and uploading the encrypted local model parameters to the cloud;

[0138] Using a federated averaging algorithm in the cloud to perform weighted averaging on the encrypted parameters of each local model, updating the global model, generating a new mission planning scheme based on the updated global model, and sending the new mission planning scheme to each aerial remote sensing platform;

[0139] Each aerial remote sensing platform adjusts its own task execution strategy according to the received new task planning scheme, and optimizes the allocation of computing resources and sensor resources according to the energy level and equipment status of each aerial remote sensing platform to obtain an optimized resource allocation scheme;

[0140] The updated task priority and optimized resource allocation plan are used as new input to achieve real-time processing of the aerial remote sensing platform status and continuous adaptive optimization of tasks.

[0141] Exemplarily, first, the status information of multiple aerial remote sensing platforms is collected in real time. The status information includes location information, attitude information, energy level and equipment working status. The location information is usually provided by satellite navigation systems such as GPS or Beidou, including longitude, latitude and altitude. The attitude information describes the spatial attitude of the platform, including pitch angle, roll angle and yaw angle, which is usually provided by an inertial measurement unit (IMU). The energy level reflects the remaining energy of the platform, such as the battery power percentage. The equipment working status includes the working mode, fault status, etc. of each sensor and communication device.

[0142] The collected status information is preprocessed, including data cleaning, normalization and standardization. Data cleaning is used to remove outliers and noise and improve data quality. Normalization unifies data of different dimensions to the same scale, such as converting location information into relative coordinates relative to the take-off point. Standardization adjusts the data to a distribution with a mean of 0 and a variance of 1, which is conducive to subsequent machine learning model processing.

[0143] Deploy edge computing nodes on each aerial remote sensing platform, including lightweight neural network models. Lightweight neural network models are designed for resource-constrained edge devices, with smaller model sizes and lower computational complexity, such as MobileNet or SqueezeNet. These models can achieve efficient feature extraction with limited computing resources.

[0144] A lightweight neural network model is used to extract features from the preprocessed state information to obtain a low-dimensional feature vector. The purpose of feature extraction is to convert high-dimensional raw data into a low-dimensional, more representative feature representation. This process can greatly reduce the amount of data while retaining key information, which is beneficial for subsequent model training and reasoning.

[0145] Next, we build a federated learning framework. Federated learning is a distributed machine learning method that allows multiple parties to jointly train models without sharing original data. In this system, the federated learning framework includes a local model deployed on the aerial remote sensing platform and a global model deployed on the cloud. The local model is responsible for processing local data and updating model parameters, while the global model integrates updates from various local models to generate a unified model.

[0146] Based on the low-dimensional feature vector obtained above, the local model is trained. The local model can be a neural network, such as a multi-layer perceptron or a convolutional neural network, which is used to learn the mapping relationship between state information and task planning. During the training process, the model continuously adjusts its parameters through the back-propagation algorithm to minimize the error between the predicted results and the actual task planning.

[0147] Use homomorphic encryption technology to encrypt the parameters of the local model. Homomorphic encryption is a special encryption method that allows specific algebraic operations to be performed on ciphertext. The decrypted result is consistent with the result of the same operation on the plaintext. This technology can protect the privacy of model parameters while allowing the cloud to perform necessary calculations. Common homomorphic encryption algorithms include Paillier encryption and BGV scheme.

[0148] Upload the encrypted local model parameters to the cloud. This step needs to consider network bandwidth and transmission security. Compression technology can be used to reduce the amount of transmitted data, and secure transmission protocols such as HTTPS can be used to ensure the security of data transmission.

[0149] In the cloud, the federated average algorithm is used to perform weighted average of the encrypted parameters of each local model and update the global model. The federated average algorithm is a simple and effective parameter aggregation method that performs weighted average of model parameters according to the amount of data or other weight factors of each local model. This process is performed in the encrypted domain without the need to decrypt the parameters, thus protecting the privacy of each platform.

[0150] Generate a new mission plan based on the updated global model. This step involves converting the output of the global model into a specific mission plan. Heuristic algorithms or reinforcement learning methods can be used to generate the optimal task allocation and path planning based on the platform status and task priority predicted by the model.

[0151] The new mission plan is sent to each aerial remote sensing platform. This process needs to take into account communication delay and reliability, and a distributed consistency protocol can be used to ensure that all platforms receive the same mission plan.

[0152] Each aerial remote sensing platform adjusts its own mission execution strategy based on the new mission planning scheme received. This includes updating the flight path, adjusting sensor parameters, reordering mission priorities, etc. At the same time, the allocation of computing resources and sensor resources is optimized according to the energy level and equipment status of each platform. For example, for platforms with lower energy levels, the operating frequency of non-critical sensors can be reduced to allocate more computing resources to navigation and communication tasks.

[0153] Finally, the updated task priority and optimized resource allocation plan are used as new inputs to restart the state information collection and processing process, realizing real-time processing of the aerial remote sensing platform state and continuous adaptive optimization of tasks. This forms a closed-loop adaptive optimization system that can continuously adjust and optimize task execution according to real-time status.

[0154] In this embodiment, the adaptability and robustness of the aerial remote sensing system are improved, and the task planning can be dynamically adjusted according to the real-time status information, effectively responding to emergencies such as equipment failure and energy shortage. Through the federated learning framework, multi-platform collaborative learning is achieved, which improves model performance while protecting data privacy. The architecture that combines edge computing with cloud computing not only reduces the communication burden, but also makes full use of the powerful computing power of the cloud. The secure computing solution based on homomorphic encryption ensures the security of sensitive information during transmission and calculation. The adaptive optimization of tasks and the dynamic allocation of resources significantly improve the overall efficiency of the system and the quality of task completion, and extend the continuous working time of the aerial remote sensing platform.

[0155] Figure 2 FIG. 1 is a schematic diagram of a system for planning and optimizing aerial remote sensing multi-platform collaborative observation tasks according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0156] The first unit is used to receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and construct a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion degree of the observation task and the resource utilization efficiency as a reward function. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and load configuration of each aerial remote sensing platform;

[0157] The second unit is used to optimize the initial mission planning scheme by using an improved multi-objective particle swarm optimization algorithm, wherein the optimization objectives include observation coverage, timeliness, energy consumption, and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, wherein the specific execution instructions include flight control instructions, sensor parameter settings, and data transmission strategies, and the corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link;

[0158] The third unit is used to receive and process the status information fed back by each aerial remote sensing platform in real time, including position, attitude, energy level and equipment working status. Based on the federated learning framework, it uses the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to real-time status information to achieve adaptive optimization of the mission.

[0159] According to a third aspect of the embodiments of the present invention,

[0160] An electronic device is provided, comprising:

[0161] processor;

[0162] a memory for storing processor-executable instructions;

[0163] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0164] A fourth aspect of the embodiments of the present invention is:

[0165] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0166] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for planning and optimizing aerial remote sensing multi-platform collaborative observation tasks, characterized in that: include: Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and build a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion degree of the observation task and the resource utilization efficiency as a reward function. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and load configuration of each aerial remote sensing platform; The initial mission planning scheme is optimized by using an improved multi-objective particle swarm optimization algorithm. The optimization objectives include observation coverage, timeliness, energy consumption, and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform. The specific execution instructions include flight control instructions, sensor parameter settings, and data transmission strategies. The corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link. It receives and processes the status information fed back by various aerial remote sensing platforms in real time, including location, attitude, energy level and equipment working status. Based on the federated learning framework, it utilizes the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to real-time status information to achieve adaptive optimization of the mission.

2. The method according to claim 1, characterized in that The initial mission planning scheme generated by using the constructed multi-platform collaborative observation mission model includes: Receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, wherein the observation demand information includes the geographic coordinates of the target area, the expected observation time window, the spatial resolution requirement and the observation frequency requirement, and the resource information includes the current position, maximum speed, maximum range, the type of sensors carried and the sensor parameters and the remaining energy of each aerial remote sensing platform; Construct a multi-platform collaborative observation task model, which uses a deep Q network as the core algorithm, regards each aerial remote sensing platform as an independent intelligent agent, and the state space of the intelligent agent includes the position of the aerial remote sensing platform, the remaining energy and the current task completion, the action space includes the flight direction, speed adjustment and sensor switch, and the reward function includes the observation task completion and resource utilization efficiency. The observation task completion is calculated by the ratio of the covered area to the target area, and the resource utilization efficiency is calculated by energy consumption and sensor usage time. The multi-platform collaborative observation task model is trained by using an experience replay mechanism and a target network technology. The experience replay mechanism randomly samples from historical experience for learning, and the target network improves the convergence performance of the algorithm by updating parameters according to a preset period. Creating a simulated environment that incorporates weather changes, obstacles, and communication delays, and generating training data in the simulated environment; The training data is used to train the multi-platform collaborative observation task model using an experience replay mechanism and a target network technology. The experience replay mechanism learns by random sampling from past experience. The target network improves the convergence performance of the algorithm by regularly updating parameters to obtain a trained multi-platform collaborative observation task model. The trained multi-platform collaborative observation task model is used to generate an initial task planning plan.

3. The method according to claim 2, characterized in that Creating a simulated environment that includes weather changes, obstacles, and communication delays, and generating training data in the simulated environment includes: Use open source map data to build a basic geographic environment, convert vector map data into rasterized elevation models and surface classification data through a geographic data processing library to obtain basic geographic environment data, and integrate dynamic weather models based on the basic geographic environment data to generate high-resolution meteorological field data including wind speed, wind direction, temperature and humidity; Using the basic geographic environment data and the high-resolution meteorological field data, constructing an obstacle system, including extracting static obstacle information from the basic geographic environment data, generating dynamic obstacle information through a random generation algorithm, and constructing a dynamic obstacle motion model to obtain complete obstacle system data; Inputting the basic geographic environment data, the high-resolution meteorological field data and the complete obstacle system data into a communication network simulator preset for an aviation remote sensing scenario, simulating the influence of terrain and atmospheric conditions on signal propagation, and generating multiple communication protocol simulation data; According to the preset mission scenario, the basic geographic environment data, the high-resolution meteorological field data, the complete obstacle system data and the multiple communication protocol simulation data are used to initialize the simulation environment and randomly generate the initial positions and state parameters of multiple aerial remote sensing platforms; A basic controller is used to simulate the initial behaviors of multiple aerial remote sensing platforms, record environmental states, platform actions and reward values, generate original training data, adjust parameters and transform scenes for the original training data, randomly change meteorological parameters, obstacle distribution and communication conditions, generate diverse training samples, build a parallel computing framework to process the diverse training samples, and generate a final training data set, in which each piece of training data contains environmental states, platform actions, immediate rewards and next state information.

4. The method according to claim 1, characterized in that: The improved multi-objective particle swarm optimization algorithm is used to optimize the initial mission planning scheme. The optimization objectives include observation coverage, timeliness, energy consumption and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, including: Constructing a multi-objective optimization model, wherein the multi-objective optimization model includes an observation coverage objective function, a timeliness objective function, an energy consumption objective function, and a platform load balancing objective function; By calculating the adaptive inertia weight based on the distance between the current position of the particle and the global optimal position, dynamically adjusting the learning factor according to the number of iterations, and using the non-dominated solution sorting strategy based on the crowding distance to solve the multi-objective optimization model, an optimized task planning scheme is obtained; Input the optimized task planning scheme into the edge computing architecture for parsing, obtain subtasks of each aerial remote sensing platform, perform local path planning on the subtasks of each aerial remote sensing platform, and obtain local path planning results; According to the characteristics of the observed target and environmental conditions, the sensor working mode, resolution and sampling frequency are dynamically adjusted using a fuzzy logic controller to obtain sensor parameter settings, and an adaptive data transmission strategy is generated based on the predicted communication quality and data priority. The local path planning result, the sensor parameter setting and the adaptive data transmission strategy are packaged into an execution instruction packet in a standard format.

5. The method according to claim 4, characterized in that According to the characteristics of the observed target and environmental conditions, the fuzzy logic controller is used to dynamically adjust the sensor working mode, resolution and sampling frequency to obtain the sensor parameter settings. Based on the predicted communication quality and data priority, the adaptive data transmission strategy is generated, including: Acquiring target characteristics and environmental conditions, wherein the target characteristics include target size and target moving speed, and the environmental conditions include atmospheric visibility and cloud cover; The target characteristics and the environmental conditions are fuzzified, the target size is divided into three fuzzy sets of small, medium and large, and the target moving speed, atmospheric visibility and cloud cover are described using a trapezoidal membership function to perform fuzzy set division to obtain the fuzzified target characteristics and environmental conditions; Based on a preset fuzzy rule base, a fuzzy reasoning method is used to reason the fuzzified target features and environmental conditions to obtain a fuzzy reasoning result; Defuzzifying the fuzzy inference result by using the centroid method to obtain sensor parameter settings, wherein the sensor parameter settings include a sensor working mode, a sensor resolution, and a sensor sampling frequency, adjusting the working state of the sensor according to the sensor parameter settings, performing an observation task, and acquiring observation data; Collect historical throughput, historical packet loss rate, current signal-to-noise ratio, and current network congestion, and input these data into a pre-trained recurrent neural network model to predict future communication quality; The observed data is prioritized, data timeliness, data integrity, target importance, and data scarcity are considered, a data priority score is calculated using a weighted sum method, and an adaptive data transmission strategy is generated based on the predicted future communication quality and the data priority score, including determining available transmission bandwidth, selecting a transmission protocol, and allocating bandwidth resources.

6. The method according to claim 1, characterized in that Receive and process status information fed back by each aerial remote sensing platform in real time, including location, attitude, energy level and equipment working status. Based on the federated learning framework, use the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning scheme according to real-time status information to achieve adaptive optimization of the task, including: Collect status information of multiple aerial remote sensing platforms in real time, the status information including location information, attitude information, energy level and equipment working status, pre-process the status information, deploy edge computing nodes on each of the aerial remote sensing platforms, the edge computing nodes including a lightweight neural network model; use the lightweight neural network model to extract features from the pre-processed status information to obtain a low-dimensional feature vector; Constructing a federated learning framework, the federated learning framework comprising a local model deployed on the aerial remote sensing platform and a global model deployed on the cloud, training the local model based on the low-dimensional feature vector, encrypting the parameters of the local model using homomorphic encryption technology, and uploading the encrypted local model parameters to the cloud; Using a federated averaging algorithm in the cloud to perform weighted averaging on the encrypted parameters of each local model, updating the global model, generating a new mission planning scheme based on the updated global model, and sending the new mission planning scheme to each aerial remote sensing platform; Each aerial remote sensing platform adjusts its own task execution strategy according to the received new task planning scheme, and optimizes the allocation of computing resources and sensor resources according to the energy level and equipment status of each aerial remote sensing platform to obtain an optimized resource allocation scheme; The updated task priority and optimized resource allocation plan are used as new input to achieve real-time processing of the aerial remote sensing platform status and continuous adaptive optimization of tasks.

7. An aerial remote sensing multi-platform collaborative observation task planning and optimization system, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit is used to receive observation demand information of the target area and resource information of multiple aerial remote sensing platforms, and construct a multi-platform collaborative observation task model based on a deep reinforcement learning algorithm. The multi-platform collaborative observation task model takes each aerial remote sensing platform as an intelligent agent, and takes the completion degree of the observation task and the resource utilization efficiency as a reward function. The constructed multi-platform collaborative observation task model is used to generate an initial task planning scheme, and the initial task planning scheme includes the flight path, observation timing and load configuration of each aerial remote sensing platform; The second unit is used to optimize the initial mission planning scheme by using an improved multi-objective particle swarm optimization algorithm, wherein the optimization objectives include observation coverage, timeliness, energy consumption, and platform load balancing. Based on the edge computing architecture, the optimized mission planning scheme is parsed into specific execution instructions for each aerial remote sensing platform, wherein the specific execution instructions include flight control instructions, sensor parameter settings, and data transmission strategies, and the corresponding specific execution instructions are distributed to each aerial remote sensing platform through a secure and encrypted communication link; The third unit is used to receive and process the status information fed back by each aerial remote sensing platform in real time, including position, attitude, energy level and equipment working status. Based on the federated learning framework, it uses the collaborative computing capabilities of edge nodes and the cloud to dynamically adjust the current mission planning plan according to real-time status information to achieve adaptive optimization of the task.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images