A method for agricultural unmanned aerial vehicle heterogeneous cluster cooperative task allocation and path planning

Abstract

The present application relates to a kind of heterogeneous cluster collaborative task allocation and path planning method for agricultural unmanned aerial vehicle, the method includes the following steps: S1: system modeling and initialization, the cooperative data acquisition task of the unmanned aerial vehicle mechanism cluster in agricultural environment is modeled as a decentralized partially observable Markov decision process;S2: multidimensional space definition definition, the global state space, joint action space and local observation space of the decentralized partially observable Markov decision process;S3: mixed reward function design;S4: cluster strategy training;S5: online distributed execution.The present application effectively solves the problems that the existing method is poor in adaptability to dynamic changes in environment, does not fully consider the heterogeneous characteristics of unmanned aerial vehicle and the separation of decision and execution, significantly improves the operation efficiency, safety and collaborative intelligence level of agricultural unmanned aerial vehicle cluster in complex unstructured environment.

Description

Technical Field

[0001] This invention relates to the field of smart agriculture and autonomous unmanned aerial vehicle (UAV) systems, and in particular to a method for collaborative task allocation and path planning for heterogeneous clusters of agricultural UAVs. Background Technology

[0002] With the development of precision agriculture, unmanned aerial vehicle (UAV) swarm technology has become a research frontier in agricultural remote sensing and phenotyping due to its high efficiency and high coverage in large-scale farmland operations. Against this backdrop, Cooperative Task Allocation and Path Planning (CTAPP) of multiple UAV systems has become a core key technology.

[0003] Existing CTAPP methods can be categorized as follows: 1. Centralized Offline Planning Methods: These methods are typically based on predefined flight patterns (such as grid-based or plow-based methods) or traditional optimization algorithms (such as Genetic Algorithm (GA) or Particle Swarm Optimization (PSO)). A central controller calculates a set of static, pre-defined optimal or suboptimal paths and task allocation schemes before the task begins. However, this method has fundamental limitations: Poor environmental adaptability: It cannot cope with common dynamic disturbances in farmland environments, such as sudden obstacles (birds, temporary facilities), sudden weather changes (wind field disturbances), and dynamic adjustments to task priorities caused by data analysis.

[0004] Low system fault tolerance: If a drone leaves due to malfunction or insufficient power, the entire preset plan may fail and cannot be reconfigured online.

[0005] High computational complexity: For large-scale clusters or fine meshes, the optimization algorithm takes a long time to solve, making it difficult to apply to scenarios with high real-time requirements.

[0006] 2. Distributed Online Coordination Methods: Some studies have attempted to employ distributed decision-making mechanisms based on market auctions and negotiation protocols to improve system response speed. However, these methods are mostly based on the assumption of homogeneous intelligent agents, that is, that all drones possess the same functions, endurance, and flight performance. This greatly limits their effectiveness in practical agricultural applications for the following reasons: Ignoring heterogeneity: In agricultural applications, to complete multi-source data acquisition (such as visible light imaging, hyperspectral analysis, and lidar mapping), swarms must consist of heterogeneous drones carrying different sensor payloads. These drones differ significantly in mission execution capabilities, endurance, and maneuverability. Existing methods fail to adequately consider this heterogeneity, leading to unreasonable task allocation, failure to leverage the strengths of different drone models, and potentially assigning unsuitable drones to tasks they cannot complete, thus slowing down overall efficiency.

[0007] 3. Separation of Decision-Making and Execution: The two methods mentioned above generally adopt an open-loop framework of "planning first, then execution," with the planning and execution phases completely separated. During execution, the UAV passively executes preset waypoints, lacking the ability to replan online and make autonomous decisions based on real-time perception information, resulting in a low overall level of system intelligence.

[0008] In recent years, Multi-Agent Reinforcement Learning (MARL) has provided a new paradigm for solving online distributed decision-making problems in multi-agent systems. Its ability to learn optimal policies through continuous interaction between agents and the environment is theoretically well-suited for solving cooperative control problems in dynamic environments. However, directly applying MARL to heterogeneous agricultural drone swarms still faces many significant challenges: Problem modeling complexity: Agricultural environments are typically partially observable, with each drone only acquiring localized information. Defining a representative local observation space for this is a significant challenge.

[0009] The challenge of designing reward functions: Reward functions need to be carefully designed to balance global task efficiency (such as scan coverage and task completion speed) with individual safety constraints (such as collision avoidance and power management), so as to incentivize heterogeneous individuals to engage in cooperative behavior rather than competing with each other.

[0010] Stability under communication constraints: Field operations may be subject to communication delays or bandwidth limitations. How to ensure the convergence and stability of the MARL strategy under limited communication conditions is a practical problem that urgently needs to be solved.

[0011] Therefore, existing technologies lack a collaborative task allocation and path planning method that can deeply integrate the characteristics of heterogeneous drone platforms, adapt to dynamic changes in the agricultural environment, and possess online real-time intelligent decision-making capabilities. This has become a bottleneck in improving the efficiency and intelligence level of agricultural drone swarm operations. Summary of the Invention

[0012] This invention provides a collaborative task allocation and path planning method that can deeply integrate the characteristics of heterogeneous UAV platforms, adapt to dynamic changes in the agricultural environment, and possess online real-time intelligent decision-making capabilities. It aims to significantly improve the operational efficiency, intelligence level, and practicality of agricultural UAV swarms in tasks such as high-throughput plant phenotypic data collection. Specifically, the technical solution provided in this application is as follows: In a first aspect, the present invention provides a method for heterogeneous cluster collaborative task allocation and path planning for agricultural drones, the method comprising the following steps: S1: System modeling and initialization, modeling the collaborative data acquisition task of the UAV cluster in an agricultural environment as a decentralized partially observable Markov decision process; S2: Multidimensional space definition: Define the global state space, joint action space, and local observation space of the decentralized partially observable Markov decision process; wherein, the joint action space is the Cartesian product of the individual actions of each agent, and is a hybrid action space, including continuous control actions and discrete decision actions; S3: Hybrid reward function design. A hybrid reward function is designed, which combines global efficiency reward and local behavior reward by weighting and integrating them to jointly optimize the overall task efficiency of the cluster and individual safety behavior during training. S4: Cluster strategy training. A multi-agent reinforcement learning algorithm based on a centralized training and distributed execution framework is used to train the strategies of each agent in the heterogeneous cluster to obtain the optimal cooperative task allocation and path planning strategy. S5: Online distributed execution. During the execution of tasks in a physical system, each agent makes decisions and executes individual actions in a distributed manner based on its own local observations and a pre-trained policy network, thereby achieving online collaborative task allocation and real-time path planning.

[0013] Based on the above technical solution, the heterogeneous UAV cluster includes reconnaissance UAVs, verification UAVs, and scanning UAVs. The reconnaissance UAVs are equipped with visible light cameras, and their individual actions include heading angle change Δheading and velocity change Δvelocity for continuous flight control, as well as discrete actions for triggering image acquisition. The verification UAVs are equipped with hyperspectral imagers, and the scanning UAVs are equipped with lidar sensors. The individual actions of the verification UAVs and scanning UAVs include discrete actions for selecting a specific target point from the global task queue, and continuous actions Δvelocity for controlling flight speed.

[0014] Based on the above technical solution, in step S1, the collaborative task model is defined by a six-tuple, which includes N, S, A, O, T, and R; where: N: represents a finite set of drone agents, corresponding to the various types of drones in the cluster; S: Represents the global state space of the system; A: Represents the joint action space of the drone agents, defined as the Cartesian product of the individual action spaces of all drone agents; O: Represents the local observation space of the unmanned aerial vehicle (UAV) agent; T: S × A × S → [0, 1] is the state transition probability function; R: S × A → R is the shared reward function.

[0015] Furthermore, in step S2, the multidimensional space definition is based on the aforementioned Dec-POMDP model, and the following spaces are specifically defined: Define a global state space S: the global state s_t ∈ S is a complete description of the global information of the system at time t, specifically including: The real-time physical state of all agents, including position coordinates P_i, velocity vector V_i, and remaining battery power E_i; The status of the global task queue Q includes the list of areas to be reconnoitered, the coordinates of identified suspected target points (such as Youzhu) and their processing status (to be verified, verified, excluded). Environmental status information, including the real-time wind field vector W estimated through the fusion of sensor or meteorological data.

[0016] Define a joint action space A: the joint action a_t ∈ A is the combination of individual actions taken by all agents at time t. For different types of agents, their individual action spaces are defined as follows: For a visible light imaging UAV, its action a_t^i is a hybrid action space, including: a two-dimensional continuous action [Δheading, Δvelocity], used to control the changes in heading angle and velocity; and a discrete binary action [triggering shooting], used to control image acquisition.

[0017] For hyperspectral sensing UAVs and LiDAR mapping UAVs, their action a_t^i is also a hybrid action space, including: a discrete action [Select_Target_j], which is used to select a specific target point j from the global task queue Q; and a one-dimensional continuous action [Δvelocity], which is used to control the velocity change to optimize the path to the target.

[0018] Define the local observation space O: Each agent i's local observation o_t^i ∈ O at time t is a partial observation of the global state s_t, specifically including: Self-status information (location, speed, battery level); Its currently assigned list of tasks or target points; The state summary information of neighboring agents obtained through inter-machine communication links, including their identity ID, current location, and current task type, is used to achieve collaborative decision-making.

[0019] Furthermore, the expression for the hybrid reward function in step S3 is: R = ω1 * R_global + ω2 * R_local, where, R_global represents the global efficiency reward, used to incentivize the rapid completion of large-area environmental scanning and multiple target verification tasks; specifically, it includes: +R_area: Positive reward based on the newly added effective scan area per unit time; +R_verify: A positive reward for successfully arriving at and verifying a suspected target point; -R_time: Penalizes the total time taken for a task, incentivizing improved efficiency.

[0020] R_local represents the local behavior reward, used to penalize collision risks with other drones or obstacles, penalize high-energy-consuming flight behaviors, and reward the completion of subtasks; specifically including: -R_collision: Negative reward when the distance between the agent and an obstacle or other agent is below a safe threshold; -R_energy: Penalizes high-power flight behaviors (such as high speed and sharp turns) to encourage energy-efficient flight; +R_task_complete: Positive reward for an agent to independently complete a subtask (such as flying to the target point).

[0021] ω1 and ω2 are pre-set weighting coefficients used to balance global task objectives with local behavioral constraints.

[0022] Furthermore, the global efficiency reward includes a positive reward based on the scanning area per unit time, a positive reward for completing the target verification task, and a negative reward for the total time consumed.

[0023] Furthermore, the local behavioral rewards include negative rewards for collision risk, negative rewards for high-energy-consuming behaviors, and positive rewards for completing sub-tasks.

[0024] Based on the above technical solution, step S4, cluster strategy training, includes a training phase and an execution phase; During the training phase, a central commentator network with access to the global state and all drone agent actions is used to evaluate the value of joint actions and guide the actor network of each drone agent to update the policy. During the implementation phase, the central commentator network is dismantled, and each drone agent independently generates individual actions based solely on its own local observations, utilizing its locally deployed, pre-trained actor network, thus achieving fully distributed autonomous online decision-making.

[0025] More preferably, in step S4, the multi-intelligence reinforcement learning algorithm is one or more of the MADDPG, QMIX, and MAPPO algorithms.

[0026] Based on the above technical solution, step S5 is executed online in a distributed manner, applying the trained strategy to the physical system: The agent converts the actions a_t^i (such as target point selection and speed command) output by the policy network into a smooth, trackable continuous flight trajectory by the airborne low-level flight controller (such as a PID controller or a model predictive controller MPC).

[0027] A dedicated communication middleware periodically synchronizes the status (such as location and battery level) of each agent within the cluster and updates the global task queue Q, ensuring that each agent's decisions are based on the latest and consistent system view, thereby guaranteeing the effectiveness and robustness of collaboration.

[0028] Based on the above technical solution, the method is adapted to unstructured agricultural environments, which include tea gardens, orchards, and open fields.

[0029] Secondly, based on the same invention, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor is capable of executing the program and implementing the method described in the first aspect.

[0030] The present invention has the following beneficial effects: Compared with the prior art, the technical solution provided by the present invention has the following significant advantages: 1. It achieves advanced dynamic environment adaptability and online intelligent decision-making capabilities. This invention, based on the Multi-Agent Reinforcement Learning (MARL) framework, constructs a closed-loop system of "perception-decision-execution," fundamentally overcoming the rigidity and sluggishness of traditional pre-programming and offline optimization methods. Specifically, it manifests as follows: Real-time response: The system can make new decisions and generate paths within milliseconds to seconds based on real-time local observations, such as sudden obstacles or changes in wind field. Dynamic task reorganization: When task priorities change, such as when a new suspected superior strain is discovered, or when a cluster member fails, the system can dynamically adjust the task allocation scheme online to achieve adaptive task reorganization, ensuring the continuity and integrity of tasks. A leap in intelligence: This capability transforms drone swarms from traditional "automated execution tools" into "intelligent agent swarms" with online learning and autonomous collaboration capabilities, significantly improving their level of intelligence.

[0031] 2. Through optimization of the collaborative mechanism, overall operational efficiency has been significantly improved. This invention, through a carefully designed hybrid reward function, guides heterogeneous UAVs to learn a globally optimal cooperative strategy at the algorithmic level, rather than an individual optimal strategy, thereby bringing a qualitative improvement in efficiency. Reduce duplication and wasted time: Through the synchronization of the global task queue and reward-based coordination, the phenomenon of multiple drones competing for the same task or repeatedly scanning the same area is effectively avoided, which significantly reduces the total flight mileage and ineffective operation time. Leveraging heterogeneous advantages: By designing differentiated motion spaces for drones with different roles, specialized division of labor and assembly line operations are achieved; the examples show that this method reduces the average time for hyperspectral drones to respond to suspected points from 180 seconds in the traditional method to 95 seconds, improving efficiency by about 89%.

[0032] Maximizing global efficiency: The R_global reward directly incentivizes rapid coverage and verification, enabling the system to complete more effective work per unit of time.

[0033] 3. The system's security and operational robustness have been comprehensively enhanced. This invention embeds security constraints directly into the decision-making mechanism of the intelligent agent, rather than relying on post-event emergency handling, thereby proactively ensuring the safe and stable operation of the system. Active collision avoidance: The R_collision penalty term enables the agent to spontaneously learn collision avoidance strategies during training, which can automatically maintain a safe distance in dense swarm flight and avoid dynamic obstacles, greatly reducing the risk of collision. Energy Management and Endurance Optimization: The R_energy penalty term encourages energy-efficient flight behavior and prompts drones with low battery levels to proactively return or reduce high-energy-consuming actions, thereby balancing the energy consumption of each node and extending the overall effective operating time of the cluster; experiments show that overall mission energy consumption can be reduced by 30%; Addressing uncertainty: The partially observable Markov decision process (POMDP) ​​model and distributed execution (CTDE) architecture give the system a natural fault tolerance to observation errors of individual nodes or brief communication interruptions, thus improving its robustness in complex farmland environments.

[0034] 4. It achieves true collaborative optimization for heterogeneous clusters, making applications more targeted. Compared to traditional methods that assume homogeneity of agents, this invention fully considers the practical realities of agricultural applications from the very beginning of modeling: Differentiated modeling: The state space, action space and reward objectives are tailored for different types of drones, so that their decision-making mechanisms are highly matched with their physical capabilities and mission objectives; Maximizing synergistic benefits: This transforms drones with different roles, such as "reconnaissance," "analysis," and "surveying," from simple superpositions into an organically coordinated whole. For example, visible light drones can efficiently detect suspected locations and promptly generate accurate verification tasks for hyperspectral drones. The latter, without needing to conduct wide-area searches, can directly execute high-value tasks, achieving a synergistic effect of "1+1>2." Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For those skilled in the art, other embodiments can be derived from the provided drawings without creative effort.

[0036] Figure 1 : Overall flowchart of the method of the present invention; Figure 2 : A schematic diagram of the POMDP model and MARL training framework in this invention; Figure 3 This invention illustrates an example of a heterogeneous drone swarm performing collaborative task allocation and path planning in a tea garden environment.

[0037] Figure 4 : Field test operation diagram of this invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. The following embodiments are only for illustrative purposes and are not intended to limit the scope of the present invention in any way. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Unless otherwise specified, the experimental methods used in the present invention are conventional methods. Unless otherwise specified, the materials and reagents used in the present invention are commercially available. Furthermore, other terms used in the present invention, unless otherwise specified, generally have the meanings commonly understood by those skilled in the art.

[0039] Example 1: A Tea Tree Phenotypic Collaborative Acquisition System Based on Multi-Agent Reinforcement Learning 1.1 This invention provides a tea tree phenotypic collaborative acquisition system based on multi-agent reinforcement learning, comprising the following four layers: S1, Perception Layer: Composed of a cluster of drones with heterogeneous roles and sensors, including: Reconnaissance UAV intelligent agent: Equipped with a visible light camera and a lightweight edge computing unit, it is used for large-area cruise reconnaissance. It processes video streams in real time through target detection models (such as YOLO series algorithms) deployed on the airborne computing unit, identifies suspected superior individual plants with prominent phenotypic characteristics such as bud density, and generates a global task queue containing geographic coordinates. Verification-type unmanned aerial vehicle (UAV) agent: Equipped with a hyperspectral imager, it is responsible for receiving instructions and performing fixed-point hovering scans on suspected points in the task queue to obtain spectral information of biochemical components (such as tea polyphenols and amino acids) at the leaf level. Scanning-type unmanned aerial vehicle (UAV) agent: Equipped with LiDAR, it is responsible for performing three-dimensional laser scanning on key areas or verified superior individual plant clusters delineated by the reconnaissance agent to obtain three-dimensional canopy structure parameters (such as plant height, canopy width, and leaf area index) with centimeter-level accuracy.

[0040] The hardware configuration of the perception layer is as follows: Drone platform: The DJI Matrice 300 RTK drone platform is used to provide high-precision positioning capabilities (real-time differential positioning accuracy can reach the centimeter level) to ensure the spatial accuracy of the collected data.

[0041] Sensors and workloads: Reconnaissance-type unmanned aerial vehicle (UAV): Equipped with a DJI H20T hybrid sensor payload, it primarily utilizes a 20-megapixel visible light camera for reconnaissance. The onboard computing unit is an Intel NUC compute stick, deploying a pruned and optimized YOLOv5s target detection model for real-time identification of densely sprouting plants.

[0042] The verification drone agent is equipped with a Cucumber UHD 185 Firefly airborne hyperspectral imager with a spectral range of 450-950nm and a spectral resolution better than 8nm. It is used to hover at suspected points to acquire hyperspectral cubic data.

[0043] Scanning-type drone agent: Equipped with a DJI Zenmuse L1 LiDAR module, integrating a Livox LiDAR module, a high-precision inertial navigation (IMU) and a mapping camera, it can achieve up to 3 echoes and a point frequency of 240,000 points / second to acquire high-precision three-dimensional point clouds of the canopy.

[0044] S2, Communication Layer: A low-latency, high-bandwidth communication network built on 5G or Low Power Wide Area Network (LPWAN) technology, using lightweight communication protocols such as MQTT / RTPS to achieve real-time and reliable interaction of state information (such as location, power, and observation data), task instructions, and control commands between agents in the perception layer and between the perception layer and the decision layer.

[0045] The communication layer is configured as follows: It adopts a hybrid communication solution that combines 4G / 5G public network with DJI SkyPort 2.0 image transmission module.

[0046] Control commands, status information, and other low-latency, low-data-volume information are transmitted via a 5G network (operator: China Mobile, average latency <30ms) to ensure real-time control.

[0047] Large amounts of data, such as hyperspectral data and lidar point clouds, are transmitted at high speed to the ground station within the local area network via SkyPort image transmission, avoiding the occupation of public network bandwidth and the generation of high traffic costs.

[0048] The communication protocol mainly adopts the MQTT protocol. Topics are divided according to agent ID and information type to achieve orderly and efficient subscription and publication of information.

[0049] S3, Decision and Control Layer: As the intelligent core of the system, it includes a multi-agent reinforcement learning (MARL) algorithm engine running on an edge computing server or in the cloud. This engine is configured as follows: The system receives local observation information from each agent (such as its own pose, sensor data, and remaining battery power); it then fuses this information with the state of the global task queue (such as the number, location, and priority of target points to be scanned) to construct a unified global state representation s_t for collaborative decision-making. s_t is the global state at time step t, serving as the basis for the decision-making brain (MARL engine). This global state s_t aims to provide the decision center with the most complete environmental information possible.

[0050] The entire collaborative data acquisition task is modeled as a Partially Observable Markov Decision Process (POMDP). This model accurately describes the following characteristics of the system: each UAV agent can only obtain local, noisy observations of the environment (such as limited field of view and uncertain target recognition results), and cannot perceive the complete global state. The decision center constructs a global state estimate for collaborative decision-making by fusing the local observation information of all agents, and calculates the optimal action command based on this estimate.

[0051] The MARL algorithm (such as MADDPG) calculates and outputs the optimal action instruction a_t for each agent. This action a_t is a joint action that includes the specific control commands that each agent should execute at time step t, such as the coordinates of the aircraft's next waypoint, flight speed, and sensor operation commands (e.g., on / off, scan mode). The reward function of the MARL algorithm is a hybrid reward mechanism, including global rewards, local rewards, and sparse rewards.

[0052] The hardware for the decision-making and control layer is as follows: An NVIDIA Jetson AGX Orin (64GB) edge computing workstation is deployed at the edge of a tea plantation. This device features 1792 CUDA cores and 56 Tensor cores, providing up to 275 TOPS of AI computing power for MARL algorithm inference, meeting the computing power requirements for real-time decision-making.

[0053] S4, Application Layer: This is a virtual laboratory platform for intelligent tea tree breeding, equipped with multimodal data automatic registration and fusion algorithms. It is used to receive, spatiotemporally register, and fuse and analyze multi-source heterogeneous data uploaded from the perception layer, and generate comprehensive phenotypic analysis reports and visualization results (such as distribution maps of superior plants, biochemical component heat maps, and three-dimensional canopy models).

[0054] 1.2 This invention also provides a collaborative data acquisition method based on the above system, comprising the following steps: Step S1: System initialization. Each UAV agent performs a self-check and reports its status information (GPS location, battery level, sensor status). The decision layer loads a high-precision digital map of the tea garden and task parameters (such as target area and task priority).

[0055] Step S2: The reconnaissance UAV intelligent agent conducts a large-scale patrol of the tea garden along a pre-planned zigzag path. It analyzes the visible light video stream in real time using an airborne lightweight deep learning model, identifies individual plants with suspected superior phenotypic characteristics, and adds their geographical coordinates to the global dynamic task queue.

[0056] Step S3: The MARL algorithm engine of the decision layer receives local observation information from all agents (including their own state, local environment information, and task queue state) and merges them into a global state representation s_t.

[0057] Step S4: The MARL algorithm engine calculates the joint action a_t at the current moment using the trained policy network based on the current global state s_t and the preset hybrid reward function. The joint action includes the specific control instructions of each agent (e.g., the verification agent flies to the specified coordinates, the scanning agent performs a fine scan of a certain area, and the reconnaissance agent continues to cruise or avoids).

[0058] Step S5: Send the action instructions to each agent through the communication layer for execution, forming a closed loop of "perception-decision-action".

[0059] Step S6: Repeat steps S3-S5 until the task is completed (e.g., all target areas have been scanned) or the termination condition is met (e.g., insufficient power or communication interruption). All collected data will be automatically uploaded to the application layer platform for fusion analysis and storage.

[0060] The hybrid reward function R is designed as follows: Global reward R_global: Used to optimize overall system performance, including: reward R1 for the percentage of scanned area per unit time, reward R2 for the number of successfully verified superior individual plants, and penalty R3 for total task time consumption.

[0061] Local reward R_local: Used to regulate the behavior of individual agents and avoid abnormal actions, including: collision avoidance penalty R4, power consumption penalty R5, and subtask completion reward R6.

[0062] Sparse reward R_sparse: To encourage accurate discovery, once a genuine and excellent single plant is found through ground sampling and verification, the system will trace back and give a one-time high reward R7 to all agents who participated in the discovery and verification process of that plant.

[0063] 1.3 Software and Algorithm Implementation 1. Simulation training environment: Before being deployed in the field, the entire system algorithm was trained and validated in a digital twin tea garden model built on the Robot Operating System (ROS2 Humble) and Gazebo Fortress simulation environment. The simulation environment simulated the three-dimensional morphology, spectral characteristics, and real flight physics characteristics (such as wind disturbance) of the tea trees. The agent learned cooperative strategies through millions of steps of training in the simulation environment.

[0064] 2. Core Algorithm and Parameters: Algorithm framework: The decision layer adopts the MADDPG (Multi-Agent Deep Deterministic Policy Gradient) algorithm, which uses a framework of centralized critics and distributed actors, making it very suitable for the heterogeneous collaborative scenario of this invention.

[0065] Network structure: Both the Actor and Critic networks are three-layer fully connected neural networks (256-128-64 neurons) with ReLU activation function.

[0066] Key hyperparameters: learning rate (Actor: 0.0001, Critic: 0.001), discount factor γ = 0.95, replay buffer size = 1e6, batch size = 1024.

[0067] Specific settings for the reward function: R1 (Scan Area Bonus): A bonus of +0.1 is awarded if the scanned area reaches 1% of the total area per minute.

[0068] R2 (Verification Success Reward): +5 for each confirmed excellent single plant.

[0069] R3 (Time-consuming penalty): -0.01 penalty per second.

[0070] R4 (Collision Penalty): Each collision warning results in a penalty of -2, and a collision ends the turn with a penalty of -20.

[0071] R5 (Battery Penalty): -0.1 for every 10% drop in battery level.

[0072] R6 (Subtask Reward): +1 for successfully completing a fixed-point scan / verification task.

[0073] R7 (Sparse Reward): After verification by ground truth, for each confirmed superior single plant, all participating agents are retroactively rewarded with +20.

[0074] 3. Algorithm and system adaptation details To ensure the efficient operation of this system, the multi-agent reinforcement learning algorithm and the target detection algorithm have been specifically designed and adapted for the heterogeneous UAV collaborative data acquisition scenario, as detailed below: (1) Adaptation of the Multi-Agent Reinforcement Learning Algorithm (MADDPG): State space design: Each agent's local observations (oti) include its own GPS coordinates, altitude, remaining battery power, sensor status, and location and distance relative to the nearest task point. The decision layer integrates the local observations of all agents and the global task queue information to construct the global state (st-st).

[0075] Action Space Design: The joint action command `at` is a composite command vector. For the flight platform, the output is three-dimensional waypoint coordinates or velocity commands; for the sensor payload, the output is control commands such as on / off and scanning modes. The action space of the verification and scanning agents is designed as navigation behavior targeting the mission point coordinates, thus distinguishing it from the cruise reconnaissance behavior of the reconnaissance agent.

[0076] Network input / output dimension adaptation: Actor Network: The input is the local observation oti of a single agent (dimension None, observation dimension None, observation dimension), and the output is the continuous action vector ati of the agent (dimension None, action dimension None, action dimension), such as heading angle and velocity value.

[0077] The Centralized Critic network takes as input a concatenation of observations and actions of all agents (ot, at) (with dimensions None, total observation dimension + total action dimension None, total observation dimension + total action dimension None), and outputs a Q-value estimate for the global state-action pair.

[0078] Training Method: The MARL engine is trained offline in a digital twin tea garden simulation environment built on ROS / Gazebo. During training, the Critic network can acquire global information from all agents to guide the Actor network updates. During deployment, each Actor network runs independently on an edge server, making decisions based solely on local observations, thus realizing the "Centralized Training with Decentralized Execution" (CTDE) paradigm and solving the non-stationarity problem in multi-agent environments.

[0079] (2) Adaptation to the object detection algorithm (YOLOv5): Model lightweighting: To adapt to the limited computing power of the onboard computing unit (Intel NUC) of the reconnaissance UAV, the YOLOv5s model is pruned and quantized. While ensuring accuracy, the model size is compressed to less than 5MB to meet the frame rate requirements of real-time inference (>15 FPS).

[0080] Data adaptation and training: The model was fine-tuned using a large number of labeled tea tree canopy images (including images from different growth stages and under different light conditions), with a focus on optimizing the recognition accuracy of the phenotypic feature of "bud density". The model output is the bounding box of suspected superior individual plants and their pixel coordinates in the image. Combined with RTK positioning information and UAV hovering attitude, the coordinates are transformed and mapped to the precise geographic coordinates (latitude and longitude) of the suspected points, and then published to the global task queue.

[0081] 1.4 Implementation Process and Results In practice, the system successfully achieved fully autonomous collaborative operation. The reconnaissance drone cruised at a speed of 5 m / s and an altitude of 5 m above the canopy, continuously publishing the coordinates of identified potential superior individual plants (with the bud density threshold set to 1.5 times the normal value) to the global task queue. Upon receiving the information, the decision engine dynamically dispatched verification and scanning drones to the target location.

[0082] The actual test results are as follows: Taking the teaching and experimental tea garden of Qingdao Agricultural University (about 20 mu) as an example, the tea garden has flat terrain and regular tea ridges, which is suitable for conducting drone collaborative operation tests.

[0083] Response timeliness: After receiving instructions, the hyperspectral UAV can reach the target point and complete the fixed-point scanning within an average of 90 seconds.

[0084] Operational efficiency: The multimodal data collection of all suspected superior plants in the 20-mu tea garden was completed in about 2 hours, which is 75% more efficient than the traditional manual collection method (which takes about 8 hours).

[0085] Data accuracy: The collected hyperspectral data were verified by ground sampling, and the retrieved tea polyphenol content was in high agreement with the laboratory biochemical determination results (coefficient of determination R). 2 >0.85); the accuracy of the three-dimensional canopy model reconstructed from LiDAR data is better than 5cm.

[0086] Explanation of the criteria for judging "superior single plants": The criteria for determining "suspected superior individual plants" (such as the budding density threshold) are not fixed, but rather a dynamically configurable system parameter. Its setting basis and working principle are as follows: 1. Basis for determining the threshold: The threshold is mainly determined based on historical data statistics and the knowledge of breeding experts.

[0087] During system initialization, historical phenotypic data of the target tea tree variety under normal growth conditions (such as the average and standard deviation of the number of buds per unit area over the years) can be imported, and the historical average value can be used as the benchmark for the "normal value".

[0088] Breeding experts can dynamically set and adjust the threshold coefficient k (e.g., k=1.5, 1.3, 1.7, etc.) on the human-computer interface of the edge computing workstation or application layer platform according to the current breeding objectives (such as focusing on yield or stress resistance). The "germination density threshold" is "historical normal value × coefficient k". This design makes the judgment criteria both data-driven and expert-experienced.

[0089] 2. System adaptability and configurability: The judgment criteria are highly configurable, and users can flexibly adjust the threshold coefficient according to different tea varieties, different growth stages (such as spring tea and summer tea), and different breeding objectives (high yield, high resistance, and high quality), rather than being limited to a fixed value.

[0090] Furthermore, the system possesses self-learning capabilities. After receiving the ground sampling verification results, the application layer platform can feed back the verification data (such as the finally confirmed truly superior individual plants and their phenotypic data at that time) to the decision-making layer to update the "historical normal value" database. This allows the threshold benchmark to be continuously optimized as the breeding process progresses, becoming more and more accurate with use.

[0091] 3. Multifactorial nature of the determination: Furthermore, the determination of "superior individual plants" does not solely rely on a single phenotypic feature (such as bud density). The model carried by the reconnaissance UAV agent can be upgraded to a multi-task learning model, simultaneously identifying multiple phenotypic indicators (such as bud density, new shoot color, leaf spread, etc.), and using weighted scoring or decision tree models for comprehensive judgment to generate a more comprehensive score for suspected superior individual plants, further improving the probability of detection and reducing the false positive rate.

[0092] Example 2: A heterogeneous swarm collaborative task allocation and path planning method for agricultural drones 2.1 Based on Example 1, we further invented a heterogeneous swarm collaborative task allocation and path planning method for agricultural drones. This method operates on a heterogeneous swarm system composed of visible light imaging drones, hyperspectral sensing drones, and LiDAR mapping drones, and includes the following steps: Step S1: System Modeling and Initialization The collaborative data acquisition task of the heterogeneous UAV swarm in a complex agricultural environment is modeled as a decentralized partially observable Markov decision process (Dec-POMDP). This model is defined by a six-tuple <N, S, A, O, T, R>, where: N: Represents a finite set of intelligent agents, corresponding to the various types of drones in the cluster; S: Represents the global state space of the system; A: Represents the joint action space of the agents, defined as the Cartesian product of the individual action spaces of all agents; O: Represents the local observation space of the agent; T: S × A × S → [0, 1] is the state transition probability function; R: S × A → R is the shared reward function.

[0093] Step S2: Multidimensional Space Definition Based on the Dec-POMDP model described above, the following space is specifically defined: Define a global state space S: the global state s_t ∈ S is a complete description of the global information of the system at time t, specifically including: The real-time physical state of all agents, including position coordinates P_i, velocity vector V_i, and remaining battery power E_i; The status of the global task queue Q includes the list of areas to be reconnoitered, the coordinates of identified suspected target points (such as Youzhu) and their processing status (to be verified, verified, excluded). Environmental status information, including the real-time wind field vector W estimated through the fusion of sensor or meteorological data.

[0094] Define a joint action space A: the joint action a_t ∈ A is the combination of individual actions taken by all agents at time t. For different types of agents, their individual action spaces are defined as follows: For a visible light imaging UAV, its action a_t^i is a hybrid action space, including: a two-dimensional continuous action [Δheading, Δvelocity], used to control the changes in heading angle and velocity; and a discrete binary action [triggering shooting], used to control image acquisition.

[0095] For hyperspectral sensing UAVs and LiDAR mapping UAVs, their action a_t^i is also a hybrid action space, including: a discrete action [Select_Target_j], which is used to select a specific target point j from the global task queue Q; and a one-dimensional continuous action [Δvelocity], which is used to control the speed change to optimize the path to the target.

[0096] Define the local observation space O: Each agent i's local observation o_t^i ∈ O at time t is a partial observation of the global state s_t, specifically including: Its own status information (location, speed, battery level); its currently assigned task list or target point; and the status summary information of neighboring agents obtained through inter-machine communication links, including their identity ID, current location, and current task type, for collaborative decision-making.

[0097] Step S3: Design of Hybrid Reward Function Design a hybrid reward function R(s_t, a_t) to simultaneously guide the system to optimize global task efficiency and ensure individual safe behavior during training. Its expression is: R = ω1* R_global + ω2* R_local Where: R_global is the global efficiency reward, used to incentivize the cluster to quickly complete the overall task, including: +R_area: Positive reward based on the newly added effective scan area per unit time; +R_verify: A positive reward for successfully arriving at and verifying a suspected target point; -R_time: Penalizes the total time taken for a task, incentivizing improved efficiency.

[0098] R_local represents local behavioral rewards, used to constrain individual behavior to ensure safety and sustainability, including: -R_collision: Negative reward when the distance between the agent and an obstacle or other agent is below a safe threshold; -R_energy: Penalizes high-power flight behaviors (such as high speed and sharp turns) to encourage energy-efficient flight; +R_task_complete: Positive reward for an agent to independently complete a subtask (such as flying to the target point).

[0099] ω1 and ω2 are pre-set weighting coefficients used to balance the relative importance between the global task objective and local behavioral constraints.

[0100] Step S4: Algorithm Training and Distributed Decision-Making Based on the CTDE Framework A multi-agent reinforcement learning algorithm under the centralized training and distributed execution (CTDE) paradigm is used to train the agent's policy.

[0101] During the training phase, a Central Critic network with access to the global state s_t and all agent actions is used to evaluate the value of joint actions and guide the actor network of each agent in policy updates. The algorithm can employ MADDPG, QMIX, or MAPPO.

[0102] During the execution phase, the central commentator network is dismantled. Each agent independently generates individual actions a_t^i based solely on its own local observations o_t^i, utilizing its locally deployed, pre-trained actor network, achieving fully distributed, autonomous online decision-making.

[0103] Step S5: Online Execution and Smooth Path Generation Execute the trained strategy in the physical system: The agent converts the actions a_t^i (such as target point selection and speed command) output by the policy network into a smooth, trackable continuous flight trajectory by the airborne low-level flight controller (such as a PID controller or a model predictive controller MPC).

[0104] A dedicated communication middleware periodically synchronizes the status (such as location and battery level) of each agent within the cluster and updates the global task queue Q, ensuring that each agent's decisions are based on the latest and consistent system view, thereby guaranteeing the effectiveness and robustness of collaboration.

[0105] 2.2 System Hardware and Software Configuration Hardware Platform: The physical system in this embodiment is built on the DJI Matrice 300 RTK drone platform. The heterogeneous cluster includes: one visible light imaging drone equipped with a Zenmuse P1 aerial survey camera, one hyperspectral sensing drone equipped with a Zenmuse H20T (including a hyperspectral module), and one LiDAR mapping drone equipped with a DJI L1 LiDAR module. All platforms are equipped with an onboard computing unit (NVIDIA Jetson Xavier NX) for local decision-making.

[0106] Communication system: The cluster communicates via an ad hoc network based on the IEEE 802.11ac (Wi-Fi 5) protocol, with a period of 200ms, for synchronizing status information and task queues.

[0107] Software environment: The algorithm development framework is PyTorch 1.9.0, the simulation environment is built based on Unity 2020.3 LTS and ML-Agents Toolkit, and the flight control of the physical system is based on ROS Noetic and PX4 Autopilot.

[0108] 2.3 Algorithm Implementation and Training Details Algorithm selection: This embodiment adopts the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm because it is very suitable for handling the characteristics of the agents in this scheme having a mixed (continuous + discrete) action space.

[0109] Network Structure: Each agent's actor network and central critic network are fully connected neural networks with two hidden layers, containing 256 and 128 units respectively, and using ReLU activation function. The actor network's output layer uses Tanh activation function for continuous actions and Softmax activation function for discrete actions.

[0110] Key hyperparameters: discount factor γ = 0.99; actor network learning rate = 0.0001; critic network learning rate = 0.001; experience replay buffer size = 1e6; batch size = 1024.

[0111] Reward function weights: After parameter tuning, the weight coefficients in this embodiment are set to ω1=0.7 (emphasizing global efficiency) and ω2=0.3 (emphasizing local security).

[0112] Simulation training: Training was conducted in a high-fidelity digital tea garden simulation environment (including tea tree models, terrain undulations, and wind field models). The total number of training steps was 5 million, taking approximately 72 hours (using a workstation equipped with an NVIDIA RTX 3090 GPU).

[0113] 2.4 Physical Deployment and Testing Process After training is completed, the trained strategy parameters are deployed to the onboard computing unit of the physical UAV.

[0114] Test environment: The flight test was conducted in a real experimental tea garden with an area of ​​5 acres. The tea garden has a typical row planting structure and uneven terrain.

[0115] Baseline Comparison: The method of this invention is compared with a typical online task allocation method based on greedy rules. Both methods are executed in the same region and the same task set (containing 20 predefined potential optimal plant coordinate points).

[0116] Evaluation metrics: The main evaluation metrics include: ① the average response time of the hyperspectral UAV from receiving the command to arriving at the target point; ② the total energy consumption of the entire cluster to complete all tasks (estimated based on the battery consumption model); ③ the number of safety incidents (such as collision warnings).

[0117] 2.5 Test Results and Performance Analysis The test results fully demonstrate the effectiveness and superiority of the method of this invention: In terms of efficiency, the method of this invention significantly reduces the average time for a hyperspectral UAV to respond and reach its target from 180 seconds using a greedy algorithm to 95 seconds. This is mainly due to the MARL strategy learning forward-looking task allocation and efficient cooperative path planning, thus avoiding ineffective flight.

[0118] In terms of energy consumption: The method of this invention reduces the total energy consumption of the entire mission by about 30% compared to the baseline by optimizing the overall flight path of the cluster and the individual flight behavior (such as uniform and smooth flight).

[0119] Regarding safety: Throughout the entire test, the drone swarm based on the method of this invention did not trigger any collision warnings based on distance thresholds. This indicates that the agent has successfully learned an effective collision avoidance strategy through training.

[0120] In terms of heterogeneous collaboration: it was observed that visible light drones prioritize wide-area scanning and marking of suspected points, while hyperspectral drones efficiently shuttle between suspected points for verification, forming an efficient "scan-verify" workflow, which fully demonstrates the advantages of heterogeneous collaboration.

[0121] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solutions and concepts of this invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. A method for heterogeneous swarm collaborative task allocation and path planning for agricultural drones, characterized in that, The method includes the following steps: S1: System modeling and initialization, modeling the collaborative data acquisition task of the UAV cluster in an agricultural environment as a decentralized partially observable Markov decision process; The heterogeneous drone cluster includes reconnaissance drones, verification drones, and scanning drones; The reconnaissance drone is equipped with a visible light camera and a lightweight edge computing unit, which is an Intel NUC compute stick and deploys a trimmed and optimized YOLOv5s target detection model. S2: Multidimensional space definition: Define the global state space, joint action space, and local observation space of the decentralized partially observable Markov decision process; wherein, the joint action space is the Cartesian product of the individual actions of each agent, and is a hybrid action space, including continuous control actions and discrete decision actions; S3: Hybrid reward function design. A hybrid reward function is designed, which combines global efficiency reward and local behavior reward by weighting and integrating them to jointly optimize the overall task efficiency of the cluster and individual safety behavior during training. The expression for the hybrid reward function in step S3 is: R = ω1 R_global + ω2 R_local, where R_global is a global efficiency reward used to incentivize the rapid completion of large-area environmental scanning and multiple target verification tasks; R_local is a local behavior reward used to penalize the risk of collision with other drones or obstacles, penalize high-energy flight behavior, and reward the completion of subtasks; ω1 and ω2 are pre-defined weighting coefficients used to balance global task objectives with local behavioral constraints; The global efficiency reward includes: a reward of the percentage of scanned area per unit time R1, a reward of the number of successfully verified superior individual plants R2, and a penalty of total task time R3. The local behavior rewards include: collision avoidance penalty R4, power consumption R5, and subtask completion reward R6. The hybrid reward function also includes a sparse reward term: to encourage accurate discovery, once the ground sampling verification confirms the discovery of a truly excellent single plant, the system traces back and gives all agents who participated in the discovery and verification process of the plant a one-time high reward of R7. The reward function is specifically set as follows: R1: A bonus of +0.1 is awarded if the scanned area reaches 1% of the total area per minute; R2: For each confirmed superior single plant, the reward is +5; R3: Penalty per second 0.01; R4: Warning penalty for each collision 2. If a collision occurs, the turn ends and a penalty is applied. 20; R5: Penalty for every 10% drop in battery level 0.1; R6: Successfully completing a fixed-point scanning / verification task earns +1 reward; R7: After verification by ground truth, for each confirmed outstanding single plant, all participating agents will be awarded +20. S4: Cluster strategy training, using a multi-agent reinforcement learning algorithm based on a centralized training and distributed execution framework to train the strategies of each agent in the heterogeneous cluster to obtain the optimal cooperative task allocation and path planning strategy; the multi-agent reinforcement learning algorithm is MADDPG. S5: Online distributed execution. During the execution of tasks in a physical system, each agent makes decisions and executes individual actions in a distributed manner based on its own local observations and a pre-trained policy network, thereby achieving online collaborative task allocation and real-time path planning. The method is adapted to unstructured agricultural environments, namely tea gardens.

2. The heterogeneous swarm collaborative task allocation and path planning method for agricultural drones according to claim 1, characterized in that, In step S1, the collaborative task model is defined by a six-tuple, which includes N, S, A, O, T, and R; where: N: represents a finite set of drone agents, corresponding to the various types of drones in the cluster; S: Represents the global state space of the system; A: Represents the joint action space of the drone agents, defined as the Cartesian product of the individual action spaces of all drone agents; O: Represents the local observation space of the unmanned aerial vehicle (UAV) agent; T: S × A × S → [0, 1] is the state transition probability function; R: S × A → R is the shared reward function.

3. The heterogeneous swarm collaborative task allocation and path planning method for agricultural drones according to claim 1, characterized in that, Step S4, cluster strategy training, includes a training phase and an execution phase. During the training phase, a central commentator network with access to the global state and all drone agent actions is used to evaluate the value of joint actions and guide the actor network of each drone agent to update the policy. During the implementation phase, the central commentator network is dismantled, and each drone agent independently generates individual actions based solely on its own local observations, utilizing its locally deployed, pre-trained actor network, thus achieving fully distributed autonomous online decision-making.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The processor is capable of executing the above-described program and implementing the method as described in any one of claims 1-3.

Images