Multi-modal interaction based ai unmanned aerial vehicle (uav) emergency remote collaborative control method and system

Abstract

The application relates to the technical field of unmanned vehicle emergency, and specifically provides an AI unmanned vehicle emergency remote cooperative control method and system for auxiliary food distribution based on multi-modal interaction, which comprises the following steps: firstly, demand data is acquired and verified through multi-modal interaction, and demand instructions containing implicit demand and priority are output; secondly, distribution priority rules are constructed according to the instructions, and a cooperative scheduling scheme of unmanned vehicles and tasks is generated through an AI scheduling model; thirdly, a distribution path scheme is generated in combination with path influencing factors, and a hierarchical re-planning mechanism is set; then, differentiated control instructions are generated and issued; finally, the unmanned vehicles and the auxiliary food state are monitored in real time, and the scheme is dynamically adjusted according to the comprehensive health degree. Before the instructions are issued, load balancing verification and path pre-flight risk deduction are further carried out, the whole-process cooperative control of emergency distribution is realized, the distribution reliability of high-priority tasks and the cluster scheduling efficiency are improved, and the application is suitable for various emergency auxiliary food distribution scenes.

Description

Technical Field

[0001] This invention relates to the field of drone emergency technology, specifically to an AI-based drone remote collaborative control method and system for emergency food delivery based on multimodal interaction. Background Technology

[0002] In special scenarios such as emergency rescue, disaster relief, and public health emergencies, the timely and safe delivery of non-staple foods is crucial for ensuring the basic living needs of those in distress and maintaining emergency order. Traditional ground delivery methods are easily limited by factors such as road damage, traffic control, and complex terrain, resulting in problems such as low delivery efficiency, limited coverage, and high personnel safety risks, making it difficult to meet the rapid response needs in emergency scenarios.

[0003] Drone delivery, with its advantages of mobility, rapid response, and independence from ground conditions, is gradually becoming an important technological means for emergency food delivery. However, existing drone emergency delivery solutions still have many technical bottlenecks, such as a single method of demand acquisition, a lack of priority-based scheduling mechanisms for tasks, and mostly static path planning. The single method of demand acquisition refers to the current reliance on one-way text or voice interaction, which cannot accurately identify implicit needs in emergency scenarios (such as food category adaptation for special groups or temperature control requirements for refrigerated foods), and it is difficult to verify the authenticity and consistency of demand data. The lack of a priority-based scheduling mechanism means that high-priority tasks (such as fresh food replenishment in the core areas of disaster zones) are not prioritized based on the urgency of different emergency scenarios, which can easily lead to resource misallocation and delays in critical tasks. The mostly static path planning means that it is not adaptable to dynamic factors such as real-time weather, airspace control, and temporary obstacles, and lacks a pre-emptive risk assessment process, which can easily lead to execution risks such as flight path deviation and endurance interruptions.

[0004] To address these issues, this invention provides an AI-based remote collaborative control method and system for emergency delivery of non-staple foods using drones, based on multimodal interaction. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides an AI-based drone-based emergency remote collaborative control method and system for food delivery based on multimodal interaction, in order to solve the problems in existing technologies.

[0006] One embodiment of the present invention provides an emergency remote collaborative control method for AI drone food delivery based on multimodal interaction, comprising the following steps: S10. Use a multimodal interaction method to obtain demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios. Verify the demand data through a pre-trained multimodal verification model. Based on the verification results, output demand instructions containing implicit demands, non-staple food attributes, emergency scenario relationships and emergency scenario priorities. S20. Construct priority rules for the delivery of non-staple foods based on demand instructions, match drone resources that are compatible with delivery needs through an AI scheduling model, and generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. S30. Obtain the path influencing factors in the emergency scenario, input the collaborative scheduling plan and path influencing factors into the path planning model, generate a delivery route plan that can avoid path influencing factors, and set a dynamic path replanning trigger mechanism. S40. Based on the collaborative scheduling scheme and delivery route scheme, generate control commands and transmit the control commands to the drone to execute the non-staple food delivery task; S50. The multimodal monitoring module is used to obtain real-time status information including the status of non-staple food and the flight status of the drone during the drone delivery process. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

[0007] This application also relates to an AI-based drone-based emergency remote collaborative control system for food delivery, comprising: The instruction generation module is used to acquire demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios using a multimodal interaction method, verify the demand data through a pre-trained multimodal verification model, and output demand instructions containing implicit demands, non-staple food attributes, emergency scenario correlations and emergency scenario priorities based on the verification results. The task generation module is used to construct priority rules for the delivery of non-staple foods based on demand instructions, and to match drone resources that are suitable for delivery needs through an AI scheduling model to generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. The route generation module is used to obtain the route influencing factors in emergency scenarios, input the collaborative scheduling plan and the route influencing factors into the route planning model, generate a delivery route plan that can avoid the route influencing factors, and set a dynamic route replanning trigger mechanism. The command transmission module is used to generate control commands based on the collaborative scheduling scheme and delivery route scheme, and transmit the control commands to the drone to execute the food delivery task; The feedback and adjustment module is used to acquire real-time status information, including the status of the non-staple food and the flight status of the drone, during the drone delivery process using the multimodal monitoring module. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

[0008] The above embodiments provide an AI-based drone-based emergency remote collaborative control method and system for non-staple food delivery, which has the following beneficial effects: By acquiring and verifying demand data through multimodal interaction, it is possible to accurately uncover hidden delivery needs in emergency scenarios and ensure data authenticity and consistency, solving the problems of single demand acquisition and inaccurate identification in traditional methods. By constructing delivery priority rules and combining them with AI scheduling to generate collaborative solutions, it is possible to achieve hierarchical management of task priorities, ensuring that high-priority tasks receive priority resource support and avoiding delays in critical tasks due to resource mismatch. By integrating multi-source dynamic path factors to generate path solutions and setting up a hierarchical replanning mechanism, it can adapt to dynamic emergency environments and improve the path's anti-interference and flexibility. By generating differentiated instructions and establishing a receipt and breakpoint resume mechanism, it ensures that instructions are issued accurately and reliably and that high-priority instructions are executed first. Through multimodal monitoring and comprehensive status health level adjustment, it takes into account both drone flight safety and the quality of non-staple food transportation, forming a closed-loop management of the entire process. Overall, it improves the reliability, timeliness, and intelligent collaboration level of AI drone emergency delivery, meeting various emergency non-staple food delivery needs. Attached Figure Description

[0009] Figure 1 A flowchart illustrating an emergency remote collaborative control method for AI drone-based non-staple food delivery based on multimodal interaction, provided in an embodiment of the present invention; Detailed Implementation The technical solutions in the embodiments of the present invention will now be clearly and completely described in conjunction with the accompanying drawings.

[0010] Reference Figure 1 One embodiment of the present invention provides an emergency remote collaborative control method for AI drone food delivery based on multimodal interaction, comprising the following steps: S10. Use a multimodal interaction method to obtain demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios. Verify the demand data through a pre-trained multimodal verification model. Based on the verification results, output demand instructions containing implicit demands, non-staple food attributes, emergency scenario relationships and emergency scenario priorities. S20. Construct priority rules for the delivery of non-staple foods based on demand instructions, match drone resources that are compatible with delivery needs through an AI scheduling model, and generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. S30. Obtain the path influencing factors in the emergency scenario, input the collaborative scheduling plan and path influencing factors into the path planning model, generate a delivery route plan that can avoid path influencing factors, and set a dynamic path replanning trigger mechanism. S40. Based on the collaborative scheduling scheme and delivery route scheme, generate control commands and transmit the control commands to the drone to execute the non-staple food delivery task; S50. The multimodal monitoring module is used to obtain real-time status information including the status of non-staple food and the flight status of the drone during the drone delivery process. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

[0011] In this embodiment, step S10 is the multimodal acquisition and verification step for emergency delivery needs. Its core purpose is to accurately collect and verify delivery demand data, providing a reliable demand basis for subsequent scheduling and route planning. Assuming the emergency scenario is the delivery of non-staple foods to remote towns after an earthquake, the multimodal interaction methods can cover voice interaction, visual image interaction, and gesture interaction. For example, rescuers can report via voice, "50 boxes of refrigerated milk and 100 boxes of ready-to-eat dry food need to be delivered to the disaster relief site; there are many elderly people at the site, so low-sugar items are needed." They can also upload images of the resettlement site and use gestures to indicate the urgency of the delivery. The non-staple food attribute information includes categories (refrigerated fresh food, ordinary dry food, low-sugar food, etc.), storage and transportation requirements (refrigeration requires 0-4℃, fragile items require protection from bumps, etc.), shelf life, etc. The emergency scenario demand information includes the scope of the assisted area (e.g., three core resettlement sites in the disaster-stricken town), the size of the assisted population (e.g., each... Each resettlement site has approximately 200 people, and the urgency of delivery is assessed (e.g., core resettlement sites need to be delivered within 6 hours). A pre-trained multimodal validation model can verify the above multi-source demand data. For example, it can verify whether the voice-reported "50 boxes of refrigerated milk" matches the refrigerated storage conditions of the resettlement sites in the images, and whether the demand for low-sugar foods is consistent with the proportion of elderly people among the assisted population. If the verification is successful, a demand instruction is output. The implicit demand in the instruction may include "refrigerated milk needs to be compatible with the drone's cargo hold temperature control function." The emergency scenario correlation can reflect "priority delivery association between core resettlement sites and refrigerated non-staple foods." The emergency scenario priority can be divided into "core resettlement sites as first-level priority and peripheral villages as second-level priority," thereby ensuring the completeness and adaptability of the demand instructions.

[0012] Step S20 is the matching and scheduling step between drone resources and delivery tasks. Its core is to establish delivery priority rules adapted to emergency needs and to accurately bind drones to tasks. First, based on the demand instructions output in S10, priority rules for non-staple food delivery are constructed. For example, the rules can clearly state that "the weight of refrigerated non-staple food delivery tasks in the first-priority scenario is higher than that of ordinary dry food tasks in the second-priority scenario, and low-sugar foods should be prioritized for matching with drones equipped with cargo compartments for categorized storage." Then, real-time status data of the drone cluster is obtained. For example, the drone cluster includes multi-rotor refrigerated drones (with a 0-4℃ temperature-controlled cargo compartment and a payload of 50kg), fixed-wing heavy-duty drones (without temperature control and a payload of 200kg), and small multi-rotor drones (with a payload of 10kg, suitable for narrow alleyways). Their status data includes remaining battery power (e.g., refrigerated drones with 80% remaining battery power) and available cargo compartment space (e.g., refrigerated drones with refrigerated cargo compartments). The data includes the remaining storage space for 30 boxes of milk in the cargo hold and the current airspace location (e.g., the refrigerated drone is on standby at an emergency supplies distribution point). After inputting priority rules and drone status data into the AI ​​scheduling model, a collaborative scheduling plan can be generated. For example, the plan will specify that "one refrigerated drone is exclusively bound to the core resettlement point for the delivery of 50 boxes of refrigerated milk, two fixed-wing drones will undertake the delivery of 100 boxes of ready-to-eat dry food to the surrounding villages, and one spare small multi-rotor drone will be configured as an emergency replacement for the refrigerated drone." The relationships in the plan can include the order of task execution (refrigerated tasks take off first) and the exclusive binding of resources (refrigerated cargo holds are only used for fresh food products), thereby achieving precise matching of resources and needs.

[0013] The AI ​​scheduling model is a demand-resource bidirectional verification and matching model based on deep reinforcement learning. Its overall architecture includes four core layers: a data preprocessing layer, a priority weight allocation layer, a resource adaptation verification layer, and a solution iteration output layer. The data preprocessing layer first normalizes and extracts features from the input non-standardized data. For example, it converts parameters such as the drone's remaining battery power, payload capacity, and cargo compartment temperature control range into standardized values ​​of 0-1, and converts emergency task priorities into quantitative levels of 1-5, while filtering out invalid resource data such as drone malfunctions and insufficient battery power. The priority weight allocation layer uses an attention mechanism algorithm to assign differentiated matching weights to delivery tasks of different levels. For example, a matching weight of 0.8 is assigned to the first-priority task of refrigerated food for core resettlement points in disaster areas, while a weight of 0.5 is assigned to the second-priority task of ordinary dry rations for peripheral villages, ensuring that high-priority tasks prioritize locking high-quality drone resources. The resource adaptation verification layer uses a deep Q-network (DQN) algorithm to perform multiple rounds of matching simulation, simultaneously verifying the compatibility between the drone's hardware capabilities and task requirements. For example, when simulating the matching of refrigerated drones with the core placement point for refrigerated milk tasks, it will verify whether the remaining power can support the round trip and whether the cargo hold temperature control meets the storage and transportation requirements of 0-4℃. At the same time, it will also verify the fit between the heavy payload attributes of fixed-wing drones and the needs of bulk delivery of ordinary dry food, and eliminate invalid matching combinations such as "small payload drones undertaking large batch tasks" or "drones without temperature control undertaking refrigerated tasks". The solution iteration output layer will select the combination solution with the best delivery time, the highest resource utilization rate and the strongest emergency support capability based on the matching results of multiple rounds of simulation. It will also embed pre-set logic for abnormal backup. For example, while outputting the main matching solution, it will automatically bind 1-2 backup drone resources to high-priority tasks to ensure that the collaborative scheduling solution output by the model meets both basic delivery needs and has the risk backup capability in emergency scenarios.

[0014] Step S30 involves planning and rescheduling emergency delivery routes. Its core is generating delivery routes adapted to the dynamic environment and establishing a trigger mechanism for route adjustments. First, it acquires route-influencing factors for the emergency scenario, such as real-time weather data (e.g., moderate rain or strong winds in the delivery area 3 hours later), airspace control information (e.g., temporary rescue airspace 100-200 meters above the disaster area requiring detours), coordinates of temporary obstacles (e.g., the specific latitude and longitude of collapsed buildings on main roads in towns), and time constraints for transporting non-staple foods (e.g., refrigerated milk must be delivered within 6 hours with uninterrupted cargo temperature control). After inputting these factors along with the collaborative scheduling scheme from S20 into the route planning model, a delivery route plan can be generated. For example, for refrigerated drones, a route can be planned to "take off from the distribution center..." The main route will "detour through temporarily controlled airspace, avoid collapsed buildings, pass through three meteorologically stable airspaces, and finally reach the core resettlement point." An alternative route will also be planned, "detouring through the airspace behind the mountain in case of strong winds." The main route will prioritize avoiding areas affected by moderate rain to ensure the temperature control stability of the refrigerated cargo compartments. Furthermore, a dynamic route replanning trigger mechanism needs to be set up. For example, it can be set to "automatically trigger route adjustment when real-time weather shows winds of level 8 or higher on the original route, or when the area of ​​temporarily controlled airspace expands," thereby ensuring that the route plan can cope with dynamic changes in emergency scenarios and guarantee the smooth progress of the delivery mission.

[0015] The path planning model is a dynamic A-type improved path decision model based on multi-source constraint fusion. Its core architecture includes four key layers: environmental perception preprocessing layer, multi-constraint weight allocation layer, path search iteration layer, and primary and backup path output layer. The environmental perception preprocessing layer first standardizes and labels the input multi-source path influencing factors. For example, it converts wind speed levels in real-time meteorological data into risk quantification values ​​of 0-10, labels airspace control information with different constraint tags such as "absolute no-fly zone" and "limited-time passage," and converts obstacle coordinates into rasterized obstacle areas recognizable by a Geographic Information System (GIS). Simultaneously, it associates the timeliness constraints of non-staple food transportation and labels refrigerated tasks with constraint association rules of "timeliness priority > airspace detour priority." The multi-constraint weight allocation layer configures differentiated weights based on the core needs of emergency delivery. For example, it assigns a weight of 0.7 to the temperature control continuity constraint for refrigerated food, a weight of 0.6 to the meteorological risk constraint of strong winds above level 8, and the highest weight of 0.9 to the no-fly zone constraint, ensuring that high-priority constraints are satisfied first in path planning. The path search iteration layer uses an improved dynamic A / B algorithm. Algorithms, in traditional A The algorithm incorporates multiple constraint weighting factors into its path cost calculation logic. For example, when calculating the cost of a path node, it considers not only the track length but also factors such as meteorological risk value, airspace constraint level, and remaining timeliness. Simultaneously, the algorithm performs multiple rounds of iterative searches. For instance, the first round generates a basic track, the second round verifies whether the track meets the continuous temperature control requirements of the refrigerated cargo hold, and the third round simulates the feasibility of the track after weather changes, eliminating risky paths such as "passing through strong wind airspace" or "entering a no-fly zone." The primary and backup path output layer integrates the iteration results, generating both a primary and backup path, and assigning different trigger conditions to the two paths. For example, the primary path is marked "activated when meteorological wind force ≤ 6 and airspace control is not temporarily adjusted," while the backup path is marked "automatically switches when the original path wind force ≥ 7." Furthermore, it embeds status verification prompts for key nodes in the path plan, such as "real-time transmission of cargo hold temperature is required when passing through XX airspace," ensuring that the path plan is both adaptable to the current environment and has emergency backup capabilities to cope with dynamic changes.

[0016] Step S40 is the generation and transmission of delivery control instructions. Its core is to build a bridge between the scheduling scheme, the route scheme and the drone execution terminal to start the delivery task. First, based on the S20 collaborative scheduling scheme and the S30 delivery route scheme, the task execution command elements of each UAV are extracted. For example, the command elements of a refrigerated UAV include specific flight track coordinates (such as key waypoints such as XX°XX′XX″N, XX°XX′XX″E), task execution sequence (takeoff within 1 hour and arrival within 6 hours), cargo hold environmental control parameters (maintaining temperature control of 0-4℃ throughout the process), airspace access permission certificate (pass code for temporary rescue airspace), and emergency communication channel (dedicated communication frequency band with the ground command center). Then, these elements are integrated to generate control commands that can be recognized by the UAVs. For example, the command will specify "fly according to the preset waypoint after takeoff, transmit cargo hold temperature data every 10 minutes, and switch to the backup route in case of strong winds". Finally, the control commands are transmitted to the UAVs through a stable communication link. For example, an emergency dedicated network or satellite communication link can be used to issue commands to ensure that refrigerated UAVs, fixed-wing UAVs, etc. can accurately receive commands. After all UAVs confirm receipt of commands, the food delivery task can be started, ensuring the reliability of command transmission and the synchronization of execution.

[0017] Step S50 involves real-time monitoring and dynamic adjustment of the entire delivery process. Its core is to achieve real-time perception of the drone and the status of the food products, and to adapt and adjust the solution in case of anomalies. First, a multimodal monitoring module collects dual status information. Drone flight status data may include real-time trajectory deviation (e.g., whether it deviates from the preset waypoint by more than 5 meters) and the rate of change of remaining battery power (e.g., whether the battery power decreases by more than 2% per minute). For example, trajectory data is obtained through the drone's GPS positioning module, and battery power decrease data is obtained through a battery sensor. Food product status data may include cargo compartment temperature fluctuations (e.g., whether the refrigerated milk cargo compartment exceeds the 0-4℃ range) and the integrity of the packaging (e.g., whether the packaging of ready-to-eat dry food is damaged). For example, temperature data is obtained through a cargo compartment temperature and humidity sensor, and image data is obtained through... The identification module identifies the packaging status; the preset status change thresholds can be set as "track deviation exceeding 5 meters, cargo compartment temperature fluctuation exceeding 3°C, and power decay rate exceeding 2% / minute". When the monitored status information exceeds the threshold, for example, the refrigerated drone's track deviation reaches 6 meters or the cargo compartment temperature rises to 6°C, feedback information is immediately generated and dynamic adjustment is initiated. The collaborative scheduling plan (such as using a backup drone to undertake part of the task) or delivery route plan (such as adjusting the track back to the preset route or switching to a shorter path to shorten the duration of temperature control anomalies) can be adjusted in a targeted manner to form a closed-loop control of the delivery process, ensuring the safety of non-staple foods and timely delivery.

[0018] It should be noted that the specific parameters, device types, algorithm selections, and rule settings listed in the above embodiments are merely illustrative examples provided to facilitate a direct understanding of the technical solutions of this invention by those skilled in the art, and do not constitute a limitation on the scope of protection of this invention. Those skilled in the art can reasonably select, adjust, or replace the above content based on the type of emergency scenario in actual application (such as post-earthquake disasters, public health emergencies, mountain rescue, etc.), the scale of non-staple food distribution (such as township-level resettlement sites, urban community emergency stations, etc.), the configuration level of drone swarms, and emergency distribution management needs. For example, the deep Q-network algorithm can be replaced with a similar deep deterministic policy gradient (DDPG) algorithm, and the improved dynamic A network algorithm can be replaced with a similar one. The algorithm can be replaced with an improved Dijkstra algorithm, or the drone equipment can be replaced with a similar refrigerated drone or heavy-duty drone that meets airspace flight standards. These conventional adjustments and replacements based on the core concept of this invention do not depart from the technical principles and protection scope of this invention, and are clear and easily implemented by those skilled in the art.

[0019] In one embodiment, step S10, the training and construction of the pre-trained multimodal verification model, specifically includes the following steps: S101. Obtain multimodal training data, wherein the multimodal training data includes at least speech time-series data, visual image data, and gesture data; S102. Construct an associated tag set, which includes emergency scenario attribute tags, multi-dimensional attribute tags for non-staple foods, implicit demand tags, and association rules between the tags; bind the multimodal training data with the associated tag set to form labeled training samples; S103. Input the training samples into the spatiotemporal attention network architecture, extract speech temporal features through the temporal attention layer, extract visual spatial features through the spatial attention layer, extract gesture spatiotemporal features through the spatiotemporal fusion layer, and perform cross-modal fusion with the speech temporal features and visual spatial features to obtain the basic fusion feature vector. S104. Based on the association rules and basic fusion feature vectors of the association label set, the spatiotemporal attention network architecture is trained to obtain a trained multimodal verification model, which is used to verify the requirement data and output the requirement matching degree and implicit requirement completion results.

[0020] In this embodiment, step S101 is the collection and organization of multimodal training data. Its core is to acquire multiple types of raw training data covering emergency delivery demand scenarios, providing sufficient and comprehensive data support for model training. Each type of multimodal training data must conform to the actual interactive scenarios of emergency food delivery. For example, voice time-series data can include voice segments of rescue personnel reporting needs in different rescue scenarios, specifically, voices from earthquake-stricken township rescue personnel reporting "50 boxes of refrigerated milk needed at the resettlement site, and low-sugar products are needed as there are many elderly people," and voices from community workers reporting "200 boxes of ready-to-eat dry food needed in the controlled area, and gluten-free food needed in some buildings" under public health lockdown scenarios. The temporal variation characteristics of the voices (such as the speed and tone changes of urgent expressions like "must be delivered within 6 hours") must also be collected simultaneously. Visual image data can cover various emergency scenario images (such as refrigerated storage facilities at disaster area resettlement sites). Photos, including photos of material receiving points in locked-down communities, physical images of different types of non-staple foods (such as packaging images of refrigerated milk and category labels for low-sugar dry goods), and adaptation diagrams of drone storage and transportation equipment (such as temperature control labels for refrigerated cargo compartments); gesture data can include auxiliary gestures used by rescue personnel when reporting via voice, such as quickly waving to indicate "urgent delivery task," making a "refrigerated" gesture with both hands (arms wrapped around to simulate a low-temperature environment) to assist in explaining storage and transportation needs, and making a "layering" gesture to indicate "low-sugar and regular foods need to be stored separately," etc. During actual collection, multiple types of data can be simultaneously collected and classified for archiving through the microphone, camera, and motion sensors of the emergency command terminal.

[0021] Step S102 involves constructing the associated label set and labeling training samples. Its core is establishing a mapping between multimodal data and emergency delivery demand elements, providing clear supervisory labels for model training. First, an associated label set is constructed. Emergency scenario attribute labels can be categorized into disaster type (earthquake, flood, public health event), scenario emergency level (Level 1, 2, 3), and characteristics of the recipient population (mostly elderly, concentrated with children, general population), etc. The full-dimensional attribute labels for non-staple foods can cover categories such as type (refrigerated fresh food, room temperature dry food, ready-to-eat food), storage and transportation requirements (0-4℃ refrigeration, shockproof, gluten-free, low sugar), and shelf-life levels (within 7 days, within 1 month, more than 3 months), etc. Implicit demand labels can include equipment. Subcategories are defined, including those for adaptability requirements (refrigerated cargo hold, categorized storage), timeliness requirements (delivery within 6 hours, delivery within 24 hours), and special requirements (narrow aisle drones, backup drones). Association rules between tags can be pre-defined. For example, the scenario tag "post-earthquake + Level 1 emergency + predominantly elderly" needs to be associated with the food attributes "low sugar + 0-4℃ refrigeration + delivery within 6 hours" and implicit requirement tags; "public health lockdown + Level 2 emergency + general population" needs to be associated with the tag combination "ready to eat at room temperature + delivery within 24 hours." Subsequently, the multimodal training data collected by S101 is precisely bound to the associated tag set. For example, the multimodal data of "earthquake relief personnel reporting 50 boxes of refrigerated low-sugar milk + a map of refrigerated facilities at the resettlement site + emergency waving gestures" is bound to the tag combination "post-earthquake + Level 1 emergency + predominantly elderly + refrigerated low-sugar milk + 0-4℃ refrigeration + delivery within 6 hours," forming standardized labeled training samples to provide supervision for subsequent model training.

[0022] Step S103 is the cross-modal feature extraction and fusion step. Its core is to achieve deep feature extraction and fusion of multimodal data through a spatiotemporal attention network architecture to explore the potential correlations between data. First, labeled training samples are input into the spatiotemporal attention network architecture. The temporal attention layer assigns key information weights to the speech temporal data. For example, it assigns higher attention weights to temporal segments of core expressions such as "within 6 hours," "many elderly," and "refrigerated," while filtering out meaningless interjection segments. Finally, it extracts speech temporal feature vectors containing core temporal features of demand. The spatial attention layer performs key region identification and feature extraction on the visual image data. For example, it prioritizes key regions such as refrigerated equipment at the placement point and category labels of non-staple foods in the image, weakening invalid features of the background environment and generating visual images containing key spatial information. The visual-spatial feature vector is used for the gesture action data. The spatiotemporal fusion layer then performs spatiotemporal feature analysis on the gesture action data. For example, it captures the spatiotemporal features such as the frequency of the "emergency wave" and the duration of the "refrigeration gesture". At the same time, it performs cross-modal fusion of the gesture spatiotemporal feature vector with the aforementioned speech temporal feature vector and visual-spatial feature vector. Through feature dimension alignment and weight allocation algorithms, it generates a basic fusion feature vector that contains the core information of multimodal data. For example, the basic fusion feature vector of a sample will simultaneously contain the speech temporal feature of "delivery within 6 hours", the visual-spatial feature of "refrigeration equipment existence", and the gesture spatiotemporal feature of "emergency need".

[0023] Step S104 is the model training and performance achievement step, the core of which is to enable the spatiotemporal attention network architecture to have the ability to verify requirements and complete implicit requirements through supervised training. First, based on the association rules of the associated label set set set in S102, a loss function for model training is constructed. The loss function needs to consider the matching degree between the basic fused feature vector and the label set, as well as the recognition accuracy of implicit demand labels. Then, the labeled training samples are input into the spatiotemporal attention network architecture in batches. Predicted labels are generated through forward propagation. The difference between the predicted labels and the real labels is compared and the network parameters are adjusted through backpropagation. For example, when the model fails to recognize the implicit demand of "low sugar" corresponding to the "mostly elderly" scenario, the loss function will trigger parameter adjustment to strengthen the association weight between "recipient characteristics" and "special attributes of non-staple foods". After multiple rounds of iterative training, when the model's demand matching accuracy is stable above 95% and the implicit demand completion accuracy is stable above 92%, training can be stopped, and a trained multimodal validation model is obtained. This model can be used in the actual application of S10 to perform cross-modal feature fusion on real-time multimodal demand data, output the matching degree between demand data and emergency scenarios, and complete implicit delivery demands not explicitly stated in voice and images.

[0024] In one embodiment, step S10 specifically includes the following steps: S11. Collect multimodal real-time interactive data, including at least voice time-series data, visual image data, and gesture data, and extract real-time demand data containing non-staple food attribute information and emergency scenario demand information from the multimodal real-time interactive data. S12. Input the multimodal real-time interaction data corresponding to the real-time demand data into the trained multimodal verification model, extract speech temporal features through the time attention layer, extract visual spatial features through the spatial attention layer, extract gesture spatiotemporal features through the spatiotemporal fusion layer, and complete cross-modal fusion to generate a real-time fusion feature vector. S13. Based on the real-time fusion feature vector and the association rules of the associated tag set, determine the consistency between the non-staple food attribute information carried by the multimodal real-time interactive data and the emergency scenario demand information, and output the demand matching degree; and based on the association rules between the emergency scenario attribute tags and the non-staple food full-dimensional attribute tags, output the implicit demand completion result. S14. If the demand matching degree is greater than or equal to a preset threshold, based on the real-time demand data and the implicit demand completion result, a demand instruction containing implicit demand, non-staple food attributes, emergency scenario association relationship and emergency scenario priority is generated.

[0025] In this embodiment, step S11 is the multimodal real-time interactive data acquisition and real-time demand data extraction step. Its core is to acquire real-time multimodal interactive data at the emergency site and extract effective delivery demand information from it. First, through the microphone, high-definition camera, motion capture sensor and other devices of the emergency command terminal, multimodal real-time interactive data of rescue personnel is collected simultaneously. For example, rescue personnel at the scene report in real time via voice that "50 boxes of refrigerated milk and 100 boxes of ready-to-eat dry food need to be delivered to 3 core resettlement points. The elderly account for more than 60% of the resettlement points, and the delivery needs to be completed within 6 hours." At the same time, they upload visual image data such as photos of refrigerated storage facilities at the resettlement points and a distribution diagram of the assisted population. They also quickly wave their hands to indicate the urgency of the delivery and double-handedness. The system collects hand gestures indicating refrigerated storage and transportation requirements by having the arms wrapped around the body. Subsequently, it performs preliminary screening and information extraction on the collected multimodal real-time interactive data, filtering out meaningless background noise, blurry images, and invalid hand gestures. From this data, it extracts information on the attributes of non-staple foods (50 boxes of refrigerated low-sugar milk, 100 boxes of room-temperature ready-to-eat dry food, with refrigerated milk requiring storage and transportation at 0-4℃) and emergency scenario requirements (3 core resettlement points, approximately 200 people per resettlement point, delivery within 6 hours, and a predominantly elderly population among the recipients), forming structured real-time demand data.

[0026] Step S12 is the real-time multimodal feature fusion step. Its core is to use a trained multimodal validation model to extract features from real-time interactive data and perform cross-modal fusion, generating a real-time fused feature vector with multi-dimensional demand information. First, the multimodal real-time interactive data corresponding to the real-time demand data in S11 is input into the trained multimodal validation model. The model's temporal attention layer assigns key information weights to the speech temporal data. For example, it assigns high attention weights (above 0.8) to temporal segments of core expressions such as "within 6 hours," "mostly elderly," and "refrigerated," filtering out temporal features of meaningless interjections like "um" and "ah," and extracting speech temporal feature vectors containing delivery timeliness and special category requirements. The spatial attention layer focuses on key areas of the visual image data, prioritizing the identification of core areas such as refrigerated storage equipment at the placement point and low-sugar labels on food packaging, while weakening background vegetation and other elements. Invalid information such as tents is removed to generate a visual spatial feature vector containing scene storage and transportation conditions and special attributes of food. The spatiotemporal fusion layer then analyzes the spatiotemporal features of gesture data. For example, it captures the spatiotemporal information such as the frequency of "rapid waving" (3 times per second) and the duration of the "hugging the refrigerated area" gesture (about 5 seconds), extracts the gesture spatiotemporal feature vector, and then performs cross-modal fusion with the aforementioned speech temporal feature vector and visual spatial feature vector through a feature dimension alignment algorithm. Finally, a real-time fused feature vector that combines the core requirements of speech, image, and gesture is generated. For example, this vector will simultaneously contain multi-dimensional features such as "6-hour emergency delivery", "refrigerated storage and transportation conditions are met", and "low-sugar food requirements".

[0027] Step S13 involves demand consistency judgment and implicit demand completion. Its core is to verify the rationality of real-time demand data and automatically complete implicit delivery demands that are not explicitly stated. First, based on the real-time fusion feature vector generated in S12, and combined with pre-defined association rules for the associated tag set, the consistency of the demand data is judged. For example, the matching between "refrigerated milk demand" and "resettlement sites have refrigerated storage facilities," and the compatibility between "mostly elderly" and "low-sugar food demand" are verified. A preset threshold for demand matching is set at 90%. The model algorithm calculates that the demand matching degree of the real-time demand data is 96%, meeting the consistency requirement. Subsequently, the model automatically completes implicit demands based on association rules. For example, based on the tag combination of "post-earthquake disaster + Level 1 emergency + mostly elderly + refrigerated milk," implicit demands such as "refrigerated milk needs to be compatible with drone temperature-controlled cargo holds" and "low-sugar food needs to be prioritized for categorized storage" are completed, forming a complete implicit demand completion result, providing a comprehensive basis for demand instruction generation.

[0028] Step S14 is the compliance requirement instruction generation step. Its core is to integrate various requirement information to generate standardized requirement instructions under the premise that the requirement matching degree meets the standard. Since the requirement matching degree obtained in S13 is 96% ≥ the preset threshold of 90%, the real-time requirement data is determined to be compliant and valid. At this time, based on the real-time requirement data in S11 and the implicit requirement completion results in S13, standardized requirement instructions are generated. The instructions will explicitly include implicit requirements (requiring temperature-controlled cargo compartment for drones and classified storage of low-sugar food), food attributes (50 boxes of 0-4℃ refrigerated low-sugar milk and 100 boxes of room temperature ready-to-eat dry food), emergency scenario association (priority delivery association between 3 core resettlement points and refrigerated low-sugar milk), and emergency scenario priority (core resettlement points are first-level priority and peripheral villages are second-level priority). This instruction can be directly transmitted to the subsequent scheduling module to provide accurate requirement guidance for drone resource matching and path planning.

[0029] In one embodiment, step S20 specifically includes the following steps: S21. Parse the demand instruction and extract the delivery constraints, emergency scenario priorities, and resource adaptation requirements of implicit needs for the non-staple food. S22. Construct a priority rule for the delivery of non-staple foods based on the priority of emergency scenarios and delivery constraints. In the priority rule for the delivery of non-staple foods, the weight of the emergency scenario level is higher than the weight of the non-staple food category. The resource adaptation requirements of implicit needs are the trigger threshold for adjusting the priority of non-staple food delivery. The delivery tasks are divided into high-priority tasks and ordinary tasks according to the priority of emergency scenarios. S23. Obtain real-time status data of the drone cluster, wherein the real-time status data includes at least the drone's payload capacity, cargo hold temperature control capability, remaining battery power, and current airspace location. S24. Input the food delivery priority rules and the real-time status data of the drone cluster into the AI ​​scheduling model, and generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks through a demand-resource two-way verification mechanism. The relationship includes task execution order, resource-specific binding and abnormal replacement mechanism.

[0030] In this embodiment, step S21 is the demand instruction parsing step. Its core is to extract the core input elements for constructing the priority rules for non-staple food delivery from the standardized demand instructions generated in S10, and at the same time obtain the key constraints and adaptation requirements required for drone resource matching, providing data support for subsequent rule construction. First, the demand instructions are structured and parsed to extract three types of core information: first, the delivery constraints of non-staple foods, such as "50 boxes of refrigerated low-sugar milk need to maintain a cargo hold temperature of 0-4℃ throughout the process and be delivered to the core resettlement point within 6 hours" and "100 boxes of ready-to-eat dry food need to be delivered in batches and have no temperature control requirements"; second, the priority of emergency scenarios, such as "3 core resettlement points are first-level priority and peripheral villages are second-level priority"; and third, the resource adaptation requirements of implicit needs, such as "refrigerated milk needs to be compatible with drones with independent temperature-controlled cargo holds" and "low-sugar food needs to be compatible with drones with cargo hold classification storage compartments". This extracted information will serve as the core basis for constructing priority rules in step S22, and is a prerequisite for the rules to have scenario adaptability and demand matching.

[0031] Step S22 is the step of constructing the priority rules for the delivery of non-staple foods. Its core is to establish a delivery task priority system adapted to emergency scenarios based on the three types of core information extracted in S21, so as to ensure that the delivery needs of high urgency scenarios are given priority. During the rule-making process, the emergency scenario priority extracted from S21 is used as the core benchmark. It is clear that the weight of the emergency scenario level is higher than the weight of the food category. For example, based on the first-level / second-level scenario division of S21, the weight of the first-level emergency scenario is set to 0.7, the weight of the refrigerated food category is 0.3, the weight of the second-level emergency scenario is 0.5, and the weight of the room-temperature dry food category is 0.2. This avoids the problem of ignoring the urgency of the scenario due to simply considering the food category. At the same time, combined with the implicit demand resource adaptation requirements extracted from S21, they are set as the trigger threshold for priority adjustment. For example, if a drone does not have the 0-4℃ temperature-controlled cargo compartment specified in S21, its priority for undertaking the task of refrigerating milk at the core settlement point will be automatically reduced. In addition, according to the emergency scenario priority division results of S21, the delivery tasks are divided into two categories. The delivery of refrigerated low-sugar milk corresponding to the first-level priority scenario is a high-priority task, and the delivery of ordinary ready-to-eat dry food corresponding to the second-level priority scenario is an ordinary task. The rules clearly state that the high-priority task must lock high-quality drone resources first, while the ordinary task can flexibly match the remaining resources. The above content constitutes the complete priority rules for the delivery of non-staple foods. All of its setting logic originates from the prior information extraction in S21, achieving a precise connection between demand and rules.

[0032] Step S23 is the real-time status data acquisition step for the drone swarm. Its core is to collect the core performance and location data of the drones to provide real-time data support for resource matching. Through the ground control platform of the drone swarm, multi-dimensional real-time status data of each drone is acquired, specifically including: multi-rotor refrigerated drones with a payload capacity of 50kg, cargo compartment with precise temperature control from 0-4℃, 80% remaining battery power, and currently located at an emergency material distribution point; fixed-wing heavy-duty drones with a payload capacity of 200kg, no cargo compartment temperature control function, 90% remaining battery power, and currently located at a backup helipad at the distribution point; and small multi-rotor drones with a payload capacity of 10kg, no independent temperature control but with classified storage compartments, 75% remaining battery power, and currently located at a temporary supply point in a township. At the same time, the airspace location latitude and longitude of all drones are collected to ensure the comprehensiveness and real-time nature of the status data.

[0033] Step S24 is the collaborative scheduling scheme generation step, the core of which is to complete the precise binding of drones and delivery tasks through the demand-resource two-way verification mechanism of the AI ​​scheduling model. The delivery priority rules constructed in S22 and the real-time drone status data in S23 are input into the AI ​​scheduling model. The model uses a demand-resource two-way verification mechanism to first verify whether the drone resources meet the task requirements (e.g., the temperature control capability of the refrigerated drone matches the storage and transportation requirements of refrigerated milk), and then verify whether the task priority matches the drone resource level (e.g., a first-priority task matches a refrigerated drone with high battery power and high adaptability). Finally, a collaborative scheduling scheme containing multiple relationships is generated. The task execution order is clearly defined: "The high-priority core resettlement point refrigerated milk task must take off within 1 hour, while the ordinary peripheral village dry food task can be delayed by 30 minutes." The exclusive resource binding is clearly defined: "The temperature-controlled cargo compartment of the refrigerated drone is only exclusively for the delivery of refrigerated non-staple foods and cannot be used for the transportation of room-temperature dry food." The abnormal backup mechanism is clearly defined: "One backup small multi-rotor drone is bound to the core resettlement point refrigerated milk delivery task. If the main drone has insufficient battery power or temperature control failure, the backup drone can be immediately activated to take over the task," ensuring that the scheduling scheme has both accuracy and emergency backup capabilities.

[0034] In one embodiment, step S30 specifically includes the following steps: S31. Obtain multi-source dynamic path influencing factors for emergency scenarios. The multi-source dynamic path influencing factors include at least real-time meteorological data, airspace control information, coordinates of temporary obstacles, and time constraints for the transportation of non-staple foods. S32. Perform data fusion processing on the multi-source dynamic path influencing factors, and generate standardized path constraint parameters through a weighted fusion algorithm of static basic data and dynamic real-time data; S33. Based on the food delivery priority rules in the collaborative scheduling scheme, the standardized path constraint parameters are input into the path planning model to generate a delivery path scheme that includes the main path and backup paths. In the delivery path scheme, the priority weight of the main path of high-priority tasks is higher than the priority weight of the main path of ordinary tasks. S34. Set up a hierarchical triggering mechanism for dynamic path replanning, with different replanning levels corresponding to the change range of multi-source dynamic path influencing factors. The replanning levels include local fine-tuning, path switching, and global replanning. The hierarchical triggering mechanism is associated with the abnormal replacement mechanism within the collaborative scheduling scheme.

[0035] In this embodiment, step S31 is the multi-source dynamic path influencing factor acquisition step. Its core is to comprehensively collect various dynamic and static factors affecting the drone delivery path in the emergency scenario, providing a complete data source for the subsequent generation of path constraint parameters. First, four types of core path influencing factors are acquired through the multi-source data acquisition terminal: First, real-time meteorological data, for example, real-time data obtained through meteorological monitoring base stations indicating that moderate rain and localized strong winds of level 7-8 will occur in the delivery area 3 hours later, and that there is short-term airflow disturbance in the airspace of the core resettlement point; Second, airspace control information, for example, control instructions obtained through the emergency airspace command center indicating that the airspace 100-200 meters above the disaster area is the exclusive airspace for temporary rescue helicopters, which must be detoured throughout the entire route, and that the airspace in the narrow alleys in the eastern part of the township is only open to small drones; Third, coordinates of temporary obstacles. For example, by using GIS data transmitted back from the early reconnaissance of drones, the specific latitude and longitude (XX°XX′XX″N, XX°XX′XX″E) of the collapsed buildings on the main road of the township, as well as the coordinates of the low-altitude obstacles formed by the fallen trees at the village entrance, can be obtained; fourth, there are time constraints on the transportation of non-staple foods, such as the time requirements in the demand instructions that "refrigerated milk at the core resettlement point must be delivered within 6 hours and the temperature control of the cargo compartment must not be interrupted" and "ordinary dry food can be delivered within 24 hours". The above-mentioned multi-source data together constitute the core influencing factors of route planning.

[0036] Step S32 is the step of multi-source path influencing factor fusion and standardized parameter generation. Its core is to integrate static and dynamic data through a weighted fusion algorithm to form a unified path constraint benchmark. First, the collected multi-source data is classified. Township geographic topology and basic distances between distribution points and resettlement sites are categorized as static basic data, while real-time weather and temporary airspace control data are categorized as dynamic real-time data. Then, a weighted fusion algorithm is used to assign differentiated weights to different data types. For example, considering the dynamic nature of emergency scenarios, dynamic real-time data is assigned a weight of 0.6, and static basic data is assigned a weight of 0.4. The weights of strong wind weather information and no-fly zone information within the dynamic data are further increased to a sub-weight of 0.7. The algorithm then normalizes the multi-source data to generate standardized path constraint parameters. For example, areas with level 8 strong winds are marked as high-constraint areas with a risk value of 0.8, no-fly zones of 100-200 meters are marked as absolute constraint areas with a risk value of 1.0, and the 6-hour time constraint for refrigerated milk is converted into a time-sensitive parameter where the path duration cannot exceed 5 hours. Simultaneously, obstacle coordinates are converted into rasterized obstacle constraint parameters recognizable by the GIS system, achieving standardization and unification of various path influencing factors.

[0037] Step S33 is the priority-differentiated primary and backup path generation step. Its core is to combine delivery priority rules to generate suitable primary and backup paths for tasks of different levels, ensuring the priority of high-priority task paths. First, based on the food delivery priority rules in the S20 collaborative scheduling scheme, differentiated weights are assigned to path planning, clarifying that the primary path priority weight for high-priority tasks is higher than that for ordinary tasks. For example, a weight of 0.9 is assigned to the primary path for the delivery of refrigerated milk to the core resettlement point, and a weight of 0.6 is assigned to the primary path for the delivery of ordinary dry food to the surrounding villages. Then, standardized path constraint parameters are input into the path planning model. The model will prioritize planning the optimal path for high-priority tasks. For example, for refrigerated drones, it generates a path like "take off from the distribution point → detour 100-2". The main route for the fixed-wing drones delivering ordinary dry rations is "00-meter controlled airspace → avoiding collapsed buildings on main roads → passing through 3 meteorologically stable airspaces → directly reaching the core resettlement point". A backup route is also planned to "switch to the low-altitude airspace behind the mountain when encountering strong winds". The main route avoids areas with moderate rain throughout to ensure temperature control in the cargo hold. The main route for the fixed-wing drones delivering ordinary dry rations is "take off from the distribution point → along the outer ring airspace of the town → bypass the obstacles in the narrow alley in the east → arrive at the outer villages" and a backup route to "switch to the low-altitude route of rural roads when the outer ring airspace is controlled". This allows for differentiated adaptation of the routes for different priority tasks.

[0038] Step S34 is the setting step for the hierarchical path replanning mechanism. Its core is to set hierarchical trigger rules based on the magnitude of changes in path influencing factors and link them with an anomaly backup mechanism to form an emergency safety net. First, three replanning levels are divided according to the magnitude of changes in multi-source dynamic path influencing factors: First, local fine-tuning, triggered by conditions such as "real-time weather shows airflow disturbance at a waypoint on the original path but wind force ≤ 6" and "track deviation ≤ 5 meters," with the adjustment method being to only correct the coordinates of that waypoint without changing the overall path framework; second, path switching, triggered by conditions such as "strong winds of 7-8 on the original path" and "expansion of local airspace control area," with the adjustment method being to directly switch to the backup path; and third, global replanning, triggered by conditions such as "widespread heavy rain on the original path" and "adjustment of the overall airspace control plan," with the adjustment method being to recalculate the entire link path. Simultaneously, this hierarchical triggering mechanism is linked to the anomaly backup mechanism of the collaborative scheduling scheme. For example, when a high-priority task triggers path switching or global replanning, the standby procedure of the backup drone will be activated simultaneously. If the main drone experiences insufficient endurance due to path adjustment, the backup drone can be immediately called upon to take over the task, ensuring coordinated linkage between path adjustment and resource backup.

[0039] In one embodiment, before step S40, the following step is further included: S301a. Based on the generated delivery route plan and the priority rules for non-staple food delivery, obtain the task path execution parameters corresponding to the tasks bound to each drone in the drone cluster. The task path execution parameters include the total path mileage, estimated energy consumption and timeliness requirements. S302a. Obtain the dynamic load parameters of each drone in the drone cluster, wherein the dynamic load parameters include real-time remaining power, available cargo space resources, and cumulative load of executed tasks; S303a. Construct a cluster load balancing evaluation model, wherein the path load weight corresponding to high priority tasks is higher than the path load weight of ordinary tasks in the cluster load balancing evaluation model. S304a. Input the task path execution parameters and dynamic load parameters corresponding to each UAV into the cluster load balancing evaluation model, and output the load saturation of each UAV. S305a. If the load saturation of at least one drone bound to a high-priority task and a normal task exceeds a preset load threshold, and the load saturation of at least one drone bound to a normal task is lower than a preset idle threshold, then the cluster load is determined to be unbalanced. S306a When a cluster load imbalance is determined, the collaborative scheduling scheme is dynamically adjusted according to the priority rules for non-staple food delivery. The ordinary tasks carried on drones whose load saturation exceeds the preset load threshold and are simultaneously bound to high-priority tasks and ordinary tasks are transferred to ordinary task drones whose load saturation is below the preset idle threshold. During the transfer, priority is given to ensuring that the exclusive resource binding between the high-priority task and the original drone and the corresponding delivery path remain unchanged. After the transfer, it is necessary to ensure that the comprehensive load of the drone undertaking the ordinary task does not exceed the preset load threshold in order to update the collaborative scheduling scheme.

[0040] In this embodiment, step S301a is the task path execution parameter extraction step. Its core is to combine the delivery path scheme and priority rules to obtain the core execution indicators of each drone-bound task, so as to provide task-side data support for load balancing assessment. First, based on the delivery route plan generated by S30 and the food delivery priority rules of S20, corresponding task path execution parameters are matched for the tasks bound to each drone. For example, for the multi-rotor refrigerated drone bound to the high-priority task of refrigerated milk at the core resettlement point, the task path execution parameters are extracted as "total path mileage 80km, estimated energy consumption of 60% of total power, and delivery time requirement within 6 hours"; for the two fixed-wing heavy-duty drones bound to the task of ordinary dry food in the surrounding villages, the parameters are extracted as "total path mileage 120km, estimated energy consumption of 50% of total power, and delivery time requirement within 24 hours" and "total path mileage 100km, estimated energy consumption of 45% of total power, and delivery time requirement within 24 hours"; for the small multi-rotor drone bound to the task of scattered low-sugar food at the temporary supply point in the township, the parameters are extracted as "total path mileage 30km, estimated energy consumption of 20% of total power, and delivery time requirement within 12 hours". These parameters will serve as the core task dimension basis for load assessment.

[0041] Step S302a is the step of acquiring dynamic load parameters for drones. Its core is to collect the real-time operating status parameters of each drone to provide equipment-side data support for load balancing assessment. Through the real-time monitoring module of the drone cluster ground control platform, three types of dynamic load parameters for each drone are acquired: First, the real-time remaining power, for example, the multi-rotor refrigerated drone has 80% power remaining, the first fixed-wing drone has 90% power remaining, the second fixed-wing drone has 70% power remaining, and the small multi-rotor drone has 75% power remaining; Second, the available resources in the cargo hold, for example, the refrigerated drone's cargo hold has only 50 boxes of dedicated storage space for refrigerated milk (no additional redundancy), the first fixed-wing drone's cargo hold can hold 100 boxes of ready-to-eat dry food (fully loaded), and the second fixed-wing drone's cargo hold only carries 40 boxes of dry food (60 boxes of storage space remaining); Third, the cumulative load of executed tasks, for example, if all drones are executing a task for the first time, the cumulative load is 0. If it is a multi-batch delivery scenario, the energy consumption and cargo hold occupancy of historical tasks need to be added to ensure the real-time and comprehensiveness of the dynamic load parameters.

[0042] Step S303a is the construction step of the cluster load balancing assessment model. Its core is to establish a load assessment system adapted to emergency delivery priorities to ensure that the path load of high-priority tasks is given priority. When constructing the model, it is clear that the path load weight corresponding to high-priority tasks is higher than that of ordinary tasks. For example, the path load weight of high-priority tasks is set to 0.7, and the path load weight of ordinary tasks is 0.3. At the same time, the model has built-in logic for calculating load saturation. Load saturation needs to comprehensively consider the matching degree between task path execution parameters (path mileage, estimated energy consumption, timeliness requirements) and dynamic load parameters (remaining power, cargo space resources, cumulative load). Among them, the matching weight of estimated energy consumption and remaining power is 0.5, the matching weight of cargo space resources and task requirements is 0.3, and the matching weight of path mileage and timeliness requirements is 0.2.

[0043] The cluster load balancing assessment model is built using a four-layer architecture and has a clear quantitative assessment logic: First, the data input layer receives the task path execution parameters of the S301a and the dynamic load parameters of the S302a drones, and performs data format standardization conversion, for example, converting "remaining battery percentage" and "estimated energy consumption ratio" into values ​​in the 0-1 range; second, the weight allocation layer, in addition to assigning differentiated weights to task types (high priority 0.7, normal 0.3), also assigns weights to various dimensions of load assessment indicators (energy consumption matching 0.5, cargo compartment matching 0.3, time-based mileage matching 0.2); third, the load saturation calculation layer. The core calculation formula is: Load saturation = (High priority task load percentage × 0.7 + Ordinary task load percentage × 0.3) × (Energy consumption matching degree × 0.5 + Cargo hold matching degree × 0.3 + Time and mileage matching degree × 0.2), where, Energy consumption matching degree = Estimated energy consumption percentage / Remaining power percentage, Cargo hold matching degree = Task cargo hold occupancy / Total available cargo hold, Time and mileage matching degree = Estimated path time / Task required timeliness; Fourth, the evaluation output layer is responsible for classifying and labeling the load saturation values ​​output by the calculation layer (low load ≤ 30%, normal load 30%-80%, high load > 80%), providing a quantitative basis for subsequent imbalance judgment.

[0044] Step S304a is the drone load saturation calculation step. Its core is to output the quantitative value of the load status of each drone through the evaluation model, providing a basis for determining load imbalance. The task path execution parameters from S301a and the dynamic load parameters from S302a are input into the cluster load balancing evaluation model. The model calculates the load saturation of each drone according to preset weights and formulas. For example, a multi-rotor refrigerated drone is only bound to a high-priority refrigeration task. Its energy consumption matching degree = 60% / 80% = 0.75, cargo hold matching degree = 50 boxes / 50 boxes = 1, and time-mileage matching degree = 5 hours (estimated path time) / 6 hours (time requirement) ≈ 0.83. Substituting these values ​​into the formula yields: Load saturation = (1×0.7+0×0.3)×(0.75×0.5+1×0.3+0.83×0.2)≈0.7×(0.375+0.3+0.166)≈0.7×0.841≈58.87%; If the drone is simultaneously bound to both a high-priority refrigerated food mission and a regular low-sugar food mission, and the estimated energy consumption for the regular mission accounts for 10%, then the overall energy consumption matching degree = 70% / 80% = 0.875, the cargo hold matching degree = 55 boxes / 50 boxes = 1.1 (the excess is calculated as 1), and the time-mileage matching degree = 5.5 hours / 6 hours ≈ 0.92. At this point: Load saturation = (1×0.7+1×0.3)×(0.875×0.5+1×0.3+0.92×0.2)=1×(0.4375+0.3+0.184)≈92.15%; The first fixed-wing UAV is only bound to ordinary missions, with energy consumption matching degree = 50% / 90%≈0.56, cargo hold matching degree = 100 boxes / 100 boxes = 1, and time-to-mileage matching degree = 8 hours / 24 hours≈0.33; Load saturation = (0 × 0.7 + 1 × 0.3) × (0.56 × 0.5 + 1 × 0.3 + 0.33 × 0.2) ≈ 0.3 × (0.28 + 0.3 + 0.066) ≈ 0.3 × 0.646 ≈ 19.38%; The load saturation of the second fixed-wing UAV was calculated to be 17.2%, and the load saturation of the small multi-rotor UAV was 12.5%. The load status of each UAV was visualized and quantified.

[0045] Step S305a is the cluster load imbalance determination step. Its core is to accurately identify the imbalance state based on the load saturation threshold, while clearly defining the core premise of the determination as not affecting high-priority tasks. First, preset load thresholds and idle thresholds are set. For example, the drone load threshold is set to 80% (if it exceeds this, it is determined to be overloaded), and the preset idle threshold is set to 35% (if it is below this, it is determined to be idle). The determination is made by combining the load saturation of each drone with the type of task it is bound to: In the above scenario, if a multi-rotor refrigeration drone is bound to both high-priority and ordinary tasks, and the load saturation is 92.15% > 80% of the load threshold, and the load saturation of a small multi-rotor drone is 12.5% ​​< 35% of the idle threshold and it is only bound to an ordinary task, then the condition of "high-priority + ordinary task drone overloaded, ordinary task drone idle" is met, and it can be determined to be a cluster load imbalance; if a drone is only bound to a single high-priority task and the load does not exceed the threshold, or the load of an ordinary task drone is within the normal range, then it is determined to be a load balance.

[0046] Step S306a is the dynamic adjustment step of the scheduling scheme when there is a load imbalance. Its core is to complete the cross-drone transfer of ordinary tasks while ensuring that high-priority tasks are not affected, so as to achieve cluster load balancing. When a load imbalance is determined, the adjustment process is initiated according to the priority rules of food delivery. For example, in the above scenario, the multi-rotor refrigerated drone has a load saturation of 92.15% > 80% load threshold and is bound to both high-priority refrigerated tasks and ordinary low-sugar food tasks. The small multi-rotor drone has a load saturation of 12.5% ​​< 35% idle threshold and is only bound to ordinary tasks. At this time, the ordinary low-sugar food tasks on the multi-rotor refrigerated drone are transferred to the small multi-rotor drone. During the transfer, it is necessary to strictly ensure that the exclusive resource binding between the high-priority refrigerated task and the refrigerated drone remains unchanged (the refrigerated cargo compartment still exclusively serves refrigerated milk), and the original delivery route of the refrigerated drone remains unchanged. At the same time, the small multi-rotor drone that takes over the task is verified. The overall load after the drone transfer, with its original load saturation of 12.5% ​​plus the estimated load of the low-sugar food task (energy consumption matching degree 20% / 75%≈0.27, cargo hold matching degree 5 boxes / 10 boxes=0.5, time-mileage matching degree 2 hours / 12 hours≈0.17), is calculated to have a load saturation of ≈ (0×0.7+1×0.3)×(0.27×0.5+0.5×0.3+0.17×0.2)+12.5%≈0.3×0.329+12.5%≈22.37%, which does not exceed the preset load threshold of 80%. Finally, the collaborative scheduling scheme was updated, which not only solved the cluster load imbalance problem, but also ensured that the delivery guarantee of high-priority tasks was not affected.

[0047] In one embodiment, before step S40, the following step is further included: S301b. Construct a path pre-flight simulation model. Input the real-time performance parameters of the UAV, the real-time multi-source dynamic path influencing factors, and the delivery path scheme including the main path and the backup path into the path pre-flight simulation model. The real-time performance parameters of the UAV include at least the power response speed, attitude adjustment sensitivity, and remaining power decay rate. S302b: The potential risks during the entire path execution process are deduced through the path pre-flight simulation model. The potential risks include at least the risk of excessive track deviation, endurance breakpoint, and attitude instability in strong turbulent airspace, and high-risk sections are marked. S303b If the pre-simulation results show that high-risk road sections will affect the reliability of delivery task execution, the hierarchical triggering mechanism of dynamic route replanning will be triggered to enable backup routes for high-risk road sections; if there is no high risk, the final executable delivery route plan will be output.

[0048] In this embodiment, step S301b is the path pre-flight simulation model construction and data input step. Its core is to build a pre-simulation model adapted to the flight characteristics of the UAV and input full-dimensional path and equipment data to provide a foundation for risk simulation. First, a path pre-flight simulation model is constructed. This model adopts a three-layer architecture: "hardware performance layer - environment coupling layer - risk simulation layer." The hardware performance layer maps the actual flight performance of the UAV, the environment coupling layer simulates the interaction between the path environment and the UAV, and the risk simulation layer outputs the risk distribution of the entire path. Then, three types of core data are input into the model: one is the real-time performance parameters of the UAV, such as the dynamic response speed of a multi-rotor refrigerated UAV (0.5m / m²). The acceleration), attitude adjustment sensitivity (attitude correction rate of ±5° / s), remaining battery power decay rate (power consumption rate of 1% / 10km range), and dynamic response speed of fixed-wing heavy-payload UAVs (1.2m / The data includes: 1) acceleration, attitude adjustment sensitivity (correction rate of ±3° / s), and remaining power decay rate (power consumption rate of 1% / 15km range); 2) real-time multi-source dynamic path influencing factors, including meteorological data of moderate rain and localized strong winds of level 7-8 in the delivery area 3 hours later, control information of temporary no-fly zones of 100-200 meters, obstacle coordinates of collapsed buildings on main roads in towns and villages, and time constraints for refrigerated milk delivery within 6 hours; 3) delivery route schemes generated by S30, namely the main route of refrigerated drones "distribution point → bypass controlled airspace → avoid collapsed buildings → core resettlement point" and the backup route of "low-airflow airspace behind the mountain", and the main and backup routes of fixed-wing drones "outer ring airspace → rural roads", to ensure the comprehensiveness and real-time nature of the model input data.

[0049] Step S302b is the full-path risk simulation and high-risk section marking step. Its core is to identify potential risks during flight through model simulation, providing accurate basis for path optimization. The path pre-flight simulation model will simulate the full path execution process of each UAV according to the actual working conditions at a 1:1 scale, focusing on identifying three types of core potential risks: First, the risk of excessive track deviation. For example, the model simulation found that when the refrigerated UAV passes through the airspace of the eastern valley of the township, it is easy to have a track deviation of more than 8 meters due to airflow disturbance (exceeding the preset deviation threshold of 5 meters); second, the risk of endurance breakpoint. For example, if the simulation shows that the fixed-wing UAV flies along the original main path, it will only have 15% battery left when it arrives at the outer village, which cannot support the return trip, forming an endurance breakpoint; third, the risk of attitude instability in strong turbulent airspace. For example, the simulation found that the core There is short-term strong turbulence in the airspace north of the resettlement site. When multi-rotor refrigerated drones pass through this area, their attitude adjustment sensitivity cannot offset the effects of turbulence, which may lead to a chain reaction of risks such as cargo compartment temperature control equipment failure. The model will classify and mark the above-mentioned risky road sections, with strong turbulent airspace marked as "Level 1 High Risk Road Section", valley airflow disturbance airspace marked as "Level 2 High Risk Road Section", and the end point of the fixed-wing drone's original main path marked as "Level 1 High Risk Road Section". The model will also output an assessment of the impact of the risks on the delivery task (e.g., Level 1 High Risk Road Section may cause interruption of refrigerated milk temperature control and delay in delivery time).

[0050] Step S303b is the risk management and final route plan output step. Its core is to link the route replanning mechanism based on the pre-simulation risk results to ensure the reliability of the delivery route execution. First, assess the impact of the pre-simulation risks on the delivery task: If the pre-simulation results show that high-risk road sections will affect the reliability of task execution (e.g., refrigerated drones passing through a level-one high-risk strong turbulent airspace will cause temperature control interruption, violating the requirement of cold chain delivery within 6 hours; the endurance break point of the original main path of fixed-wing drones will cause them to be unable to return), then immediately trigger the path dynamic replanning hierarchical triggering mechanism set in S30, and activate the backup path for high-risk road sections. For example, adjust the path of refrigerated drones to "borrow the backup path of low airflow behind the mountain to avoid the strong turbulent airspace on the north side", and adjust the path of fixed-wing drones to "shorten the backup path of the outer ring airspace segment to ensure the return safety threshold of ≥20% remaining power". If the pre-simulation results show that there are no high-risk road sections (e.g., the track deviation of all drone paths is ≤3 meters, the remaining power is sufficient, and there is no risk of attitude instability), then directly output the final executable delivery path plan. This plan can be directly used to generate S40 control commands, providing reliable path guidance for drones to accurately execute delivery tasks.

[0051] In one embodiment, step S40 specifically includes the following steps: S41. Based on the delivery route scheme and the collaborative scheduling scheme, extract the task execution instruction elements corresponding to each UAV. The instruction elements include at least the flight track coordinates, task execution sequence, cargo hold environmental control parameters, airspace access permission certificate and emergency communication channel. S42. Construct a differentiated instruction generation model and configure exclusive instruction templates for different levels of tasks according to the priority rules for non-staple food delivery. Among them, the instruction templates for high-priority tasks include priority airspace passage instructions, cargo hold environment real-time monitoring instructions, and abnormal emergency circuit breaker instructions, while the instruction templates for ordinary tasks include basic flight control instructions and routine status feedback instructions. S43. Input the instruction elements into the differentiated instruction generation model to generate a collaborative control instruction set for each UAV. The collaborative control instruction set has an instruction priority identifier, and the control instruction corresponding to a high-priority task has a higher priority than that of a normal task. S44. Perform multimodal verification on the collaborative control instruction set. Through a triple verification mechanism of instruction syntax and logic verification, UAV performance matching verification, and path compliance verification, invalid or conflicting instructions are eliminated. S45. The verified collaborative control command set is sent to the UAV cluster in order of command priority. At the same time, a command sending receipt and breakpoint resume mechanism is established to ensure that all UAVs receive and confirm the command before starting the task execution process.

[0052] In this embodiment, step S41 is the task execution instruction element extraction step. Its core is to extract the full-dimensional instruction parameters required for the UAV to execute the task from the delivery route plan and the collaborative scheduling plan, providing core input for the generation of differentiated instructions. First, based on the final delivery route plan output by S30 and the collaborative scheduling plan optimized by load balancing and pre-flight simulation, five types of core instruction elements are accurately extracted for each UAV. For the multi-rotor refrigerated UAV bound to the high-priority task of refrigerated milk at the core settlement point, the extracted instruction elements are: flight track coordinates (distribution point latitude XX°XX′XX″N, longitude XX°XX′XX″E → backup airspace of low airflow behind the mountain latitude XX°XX′XX″N, longitude XX°XX′XX″E → core settlement point latitude XX°XX′XX″N, longitude XX°XX′XX″E), task execution sequence (take-off within 1 hour, arrival within 6 hours, status data transmitted every 10 minutes), cargo hold environmental control parameters ( The system maintains precise temperature control between 0-4℃ throughout the entire process, with an immediate alarm triggered if the temperature fluctuation exceeds 3℃. It also includes airspace access permission credentials (temporary rescue airspace exclusive access code JYZ-2025-001) and an emergency communication channel (command center exclusive communication frequency band 433MHz). For fixed-wing, heavy-payload UAVs tasked with delivering ordinary dry rations to surrounding villages, the extracted command elements are: flight path coordinates (gathering point → township outer ring airspace → surrounding village material receiving point), mission execution sequence (takeoff within 1.5 hours, delivery within 24 hours, basic status feedback every 30 minutes), cargo hold environmental control parameters (no mandatory temperature control requirements, anti-turbulence protection mode enabled throughout), airspace access permission credentials (ordinary delivery airspace code PSD-2025-003), and emergency communication channel (general communication frequency band 430MHz). All command elements are precisely matched with mission priority, UAV performance, and path constraints, providing complete data support for subsequent command generation.

[0053] Step S42 is the differentiated instruction generation model construction step. Its core is to establish a dedicated instruction template system adapted to task priorities, ensuring that high-priority task instructions have special emergency support mechanisms. During model construction, based on the food delivery priority rules in S20, differentiated dedicated instruction templates are configured for different levels of tasks. Among them, the instruction templates for high-priority tasks, in addition to basic flight and status feedback instructions, integrate three additional types of dedicated instructions: priority airspace passage instructions (which can prioritize the use of backup channels in temporary rescue airspace, without being restricted by the regular airspace passage order), and real-time cargo hold environment monitoring instructions (which actively report cargo hold temperature and humidity, refrigeration equipment, etc. every 5 minutes). The system includes operational status and requires simultaneous uploading of data verification codes. It also includes an emergency circuit breaker command (when cargo hold temperature exceeds the threshold or battery power drops sharply, it can autonomously trigger a nearby emergency landing and directly request ground support without waiting for regular command approval). For ordinary missions, the command template only retains basic flight control commands (flying sequentially according to preset waypoints, without unauthorized deviation) and regular status feedback commands (reporting position and remaining battery power at a fixed frequency; delays are allowed when the signal is weak, but interruptions are not permitted). It does not configure dedicated emergency support commands. Furthermore, the model has built-in automatic matching logic between command elements and templates, which can automatically call the corresponding template based on the mission level bound to the UAV, providing a framework for standardized command generation.

[0054] Step S43 is the collaborative control instruction set generation step. Its core is to output an instruction set with priority identifiers through a differentiated instruction generation model to achieve priority differentiation of cluster instruction execution. The instruction elements of each UAV extracted in S41 are input into the differentiated instruction generation model. The model integrates the elements according to the instruction template of the corresponding task level and adds clear priority identifiers to the instruction set. For high-priority tasks of refrigerated UAVs, the priority identifier of the generated collaborative control instruction set is "Level 1 Instruction". The instruction set clearly states that "priority airspace passage instructions can cover the regular restrictions of ordinary airspace control" and "the circuit breaker instruction is triggered immediately when the cargo hold temperature control is abnormal, without waiting for the approval of regular instructions". For ordinary tasks of fixed-wing UAVs, the priority identifier of the generated instruction set is "Level 2 Instruction". The instruction set clearly states that "during flight, the flight airspace of Level 1 instruction UAVs must be actively avoided" and "status feedback can be delayed for up to 10 minutes when the airspace signal is weak, but key data must not be missed". The final generated collaborative control instruction set contains both the independent execution instructions of each UAV and the priority rules of cluster collaboration, ensuring that high-priority task instructions can be received and executed first.

[0055] Step S44 is the multimodal verification step of the collaborative control instruction set. Its core is to eliminate invalid or conflicting instructions through a triple verification mechanism to ensure the feasibility and compliance of the instructions. First, the syntax and logic of the instructions are verified to check the format standardization and logical consistency of the instructions. For example, it verifies whether the timing of "take off within 1 hour" and "arrive within 6 hours" in the refrigerated drone instructions matches, and whether there is a conflict between the triggering logic of "return temperature control data every 10 minutes" and "emergency circuit breaker instruction". Instructions with syntax errors or timing contradictions are eliminated. Second, the drone performance matching is verified to check the compatibility between the instruction parameters and the drone hardware performance. For example, it is confirmed whether the cargo hold temperature control system of the refrigerated drone can support the instruction requirement of "0-4℃ throughout the entire process" and whether the remaining power can meet the energy consumption requirements of the flight path, thus eliminating instructions such as "requiring the fixed-wing drone to enter narrow alley low-altitude airspace" that are inconsistent with the equipment performance; finally, the path compliance is verified, verifying whether the track coordinates in the instruction comply with airspace control and path constraints. For example, it is confirmed whether the backup flight path of the refrigerated drone avoids the temporary no-fly zone of 100-200 meters and avoids high-risk sections of strong winds, ensuring that the instruction track is completely consistent with the final path plan. The full compliance of the instruction is achieved through triple verification.

[0056] Step S45 is the instruction priority issuance and task execution initiation step. Its core is to issue instructions according to priority and establish a guarantee mechanism to ensure the reliability of instruction transmission. First, instructions are issued in descending order of priority. The first-level instruction for the refrigerated drone is issued first. After the drone sends a "instruction received and parsed" receipt within 10 seconds, the second-level instruction for the fixed-wing drone is issued. At the same time, two guarantee mechanisms are established: one is the instruction issuance receipt mechanism, after the drone receives the instruction, it must send a receipt within a specified time limit. If no receipt is received within 5 seconds, the retransmission process is immediately initiated to ensure that no instruction is missed; the other is the breakpoint resume transmission mechanism, if the airspace signal is interrupted and the instruction issuance is interrupted, the untransmitted instruction fragment can be sent from the breakpoint after the signal is restored, without retransmitting the complete instruction set. After all drones send instruction confirmation receipts, the food delivery task execution process can be officially started to ensure the synchronization and reliability of instruction execution.

[0057] In one embodiment, step S50 specifically includes the following steps: S51. Use a multimodal monitoring module to collect flight status data of each drone in the drone cluster and status data of the delivered food. The flight status data includes at least the drone's real-time trajectory deviation and the rate of change of remaining battery power. The food status data includes at least the cargo hold temperature fluctuation and the integrity of the cargo packaging. S52. Construct a multimodal state fusion judgment model, and perform weighted fusion processing on the flight state data and food status data corresponding to each UAV. Among them, the weight of food status data is higher than that of flight state data. Based on the fusion result, generate the comprehensive state health of each UAV. S53. A three-level threshold range for the overall health status is preset. The level is dynamically adjusted to adapt to the differentiated activation and path replanning hierarchical triggering mechanism corresponding to different threshold ranges. Specifically: Slightly abnormal range: Only minor adjustments are made to the flight path of the corresponding UAV; Moderately abnormal range: Trigger the abnormal replacement mechanism in the collaborative scheduling scheme, activate the backup drone to take over the original task, and make adaptive adjustments to the delivery route of the original drone; Severe anomaly zone: Globally suspend delivery tasks within the current zone and regenerate the collaborative scheduling plan and corresponding delivery route plan; S54. After completing the dynamic adjustment of the corresponding level, continuously collect status information through the multimodal monitoring module and update the overall health status in real time.

[0058] In this embodiment, step S51 is a multimodal state data acquisition step. Its core is to simultaneously acquire dual data on the UAV's flight status and the status of the supplementary food using a multimodal monitoring module, providing a comprehensive data source for overall state determination. First, using the GPS positioning module, power sensor, cargo hold temperature and humidity sensor, high-definition image recognition module onboard the UAV, and the multimodal monitoring terminal at the ground command center, two types of core state data are collected. Flight status data includes real-time trajectory deviations of each UAV (e.g., the deviation between the real-time trajectory and preset waypoints of a multi-rotor refrigerated UAV, and the trajectory offset angle of a fixed-wing UAV), and the rate of change of remaining battery power (e.g., the battery decay rate of a refrigerated UAV every 10 minutes, and the real-time remaining percentage of the fixed-wing UAV's range). It can also expand to collect UAV attitude stability data (e.g., fuselage vibration amplitude, rotor speed fluctuations). Supplies... The cargo status data includes cargo hold temperature fluctuations (such as the deviation of the real-time temperature of the refrigerated milk cargo hold from the standard value of 0-4℃, and the changes in ambient temperature and humidity in the normal temperature dry food cargo hold), and cargo packaging integrity (such as determining whether the refrigerated milk packaging box is damaged through the image recognition module, and whether the seal of the ready-to-eat dry food is intact). For refrigerated drones with high priority tasks, additional data on the operating status of the cargo hold cold chain equipment will be collected (such as the working power of the refrigeration unit and the calibration status of the temperature control sensor). For example, if the refrigerated drone's flight path deviation reaches 6 meters, the battery decay rate is 1.5% / 10 minutes, the cargo hold temperature briefly rises to 5℃, and the cargo packaging is undamaged, and the fixed-wing drone's flight path deviation is 2 meters, the battery decay rate is 0.8% / 10 minutes, and the dry food packaging is intact, all collected data will be synchronously transmitted back to the ground command center at a frequency of 1 minute / time to ensure the real-time and completeness of the status data.

[0059] Step S52 involves constructing a multimodal state fusion judgment model and generating health status. Its core is establishing a state fusion judgment system adapted to emergency delivery needs, quantifying the overall health status of the drone. During model construction, the weight of food status data is explicitly set higher than that of flight status data to prioritize food delivery safety. For example, the weight of food status data is set to 0.6, and the weight of flight status data is 0.4. Within the food status data, the sub-weights for cargo compartment temperature fluctuations are 0.7 and cargo packaging integrity is 0.3. Within the flight status data, the sub-weights for real-time track deviation are 0.6 and the remaining battery power change rate are 0.4. The model incorporates data standardization processing logic, first converting various status data into 0-1 quantified values ​​(e.g., track deviation ≤ 5 meters is 0, 5-10 meters is 0.5, > 10 meters is 1; cargo compartment temperature is 0-4℃, 4-6℃ is 0.3, > 6℃ is 1). Then, a weighted fusion algorithm is used to calculate the overall health status. The calculation formula is: Overall health status = 1 - (quantified value of non-staple food status × 0.6 + quantified value of flight status × 0.4); The health score ranges from 0 to 1, with higher values ​​indicating a healthier state. Substituting the data from the refrigerated drone collected by the S51 into the formula, the quantification value for the food status is 0.3 (temperature only briefly exceeds the threshold) and the quantification value for the flight status is 0.5 (track deviation of 6 meters). Therefore, the overall health score = 1 - (0.3 × 0.6 + 0.5 × 0.4) = 1 - (0.18 + 0.2) = 0.62. For the fixed-wing drone, the quantification value for the food status is 0 and the quantification value for the flight status is 0.2 (track deviation of 2 meters). Therefore, the overall health score = 1 - (0 × 0.6 + 0.2 × 0.4) = 0.92. The model will simultaneously output the health score classification label, providing a quantitative basis for subsequent adjustments.

[0060] Step S53 involves setting three threshold ranges and implementing differentiated dynamic adjustments. Its core is establishing a tiered adjustment system adapted to the path replanning mechanism to achieve precise handling of abnormal states. First, three threshold ranges for overall health are preset: a slightly abnormal range (health 0.6-0.8), a moderately abnormal range (health 0.3-0.6), and a severely abnormal range (health < 0.3). Differential adjustment levels adapted to the path dynamic replanning tiered triggering mechanism are configured for each range. For the slightly abnormal range, only the flight path of the corresponding UAV is locally fine-tuned. For example, if a refrigerated UAV has an overall health of 0.62 within this range and only exhibits track deviation and minor temperature fluctuations, local track fine-tuning is initiated to correct the deviated waypoint coordinates. Simultaneously, a cargo hold temperature control calibration command is issued, without adjusting the overall path or scheduling plan. For the moderately abnormal range, the abnormal backup mechanism in the collaborative scheduling plan is triggered, activating the backup mechanism. The drones take over the original mission, while the delivery routes of the original drones are adaptively adjusted. For example, if the health of a refrigerated drone drops to 0.5 (cargo compartment temperature rises to 7°C and battery depletion rate reaches 2% / 10 minutes), a backup small multi-rotor drone is immediately activated and put on standby. At the same time, the original refrigerated drone's route is adjusted to the nearest alternate landing point, and delivery continues after the fault is resolved, or the backup drone directly takes over the remaining mission. For severely abnormal areas, delivery missions in the current area are suspended globally, and a new collaborative scheduling plan and corresponding delivery route plan are generated. For example, if multiple drones in the area have a health of <0.3 (track deviation exceeds 10 meters, cargo compartment temperature control fails, packaging is damaged) due to sudden strong winds, delivery must be suspended globally, the airspace environment is reassessed, the mission allocation of the drone cluster is adjusted, and a new safe route is planned. All adjustments prioritize the delivery needs of high-priority missions to avoid spoilage of non-staple foods or delivery delays.

[0061] Step S54 involves continuous monitoring and health updates of the adjusted status. Its core is to achieve closed-loop status control throughout the entire delivery process, ensuring the effectiveness of adjustment measures. After completing the corresponding level of dynamic adjustment, the multimodal monitoring module will maintain a high-frequency data collection rhythm. For refrigerated drones that have completed trajectory fine-tuning, it will focus on monitoring whether their trajectory deviation has fallen back to within 5 meters and whether the cargo compartment temperature has stabilized in the 0-4℃ range. For scenarios where backup drones are activated, it will simultaneously monitor the flight status of the backup drones and the status of the non-staple food after transfer. At the same time, the multimodal status fusion judgment model will continuously update the comprehensive health status of each drone based on the real-time collected data. For example, after trajectory fine-tuning, if the trajectory deviation of the refrigerated drone drops to 3 meters and the cargo compartment temperature recovers to 3℃, its comprehensive health status rises to 0.85, moving out of the slightly abnormal range. If the adjusted status does not improve or even deteriorates, a higher-level adjustment mechanism will be triggered to ensure that the status of the drones and non-staple foods remains within a controllable range throughout the entire process from task initiation to material delivery, ensuring the final completion of the emergency delivery task.

[0062] It should be noted that the basic technical operations involved in each embodiment of the present invention are all conventional technical means well known to those skilled in the art, requiring no further explanation and easy to implement. Specifically, these include: data processing operations such as noise reduction, normalization, coordinate standardization, and feature vector dimension calibration of multimodal training data / real-time interactive data; conventional matching processes for label binding; model-related operations such as learning rate adjustment and batch size setting for spatiotemporal attention network training; basic algorithm selection and debugging for models such as cluster load balancing assessment and multimodal state fusion determination; basic simulation algorithm selection for path pre-flight simulation models; and configuration of basic template editing tools for differentiated instruction generation models; equipment and tools such as demand instruction structured parsing tools, UAV status / performance parameter acquisition sensors and terminals, multi-source path influencing factor acquisition terminals, general tools for GIS system rasterization processing, instruction multimodal verification algorithm engines, general communication protocols for instruction breakpoint resumption, and conventional hardware for multimodal monitoring modules; and threshold and weight-related operations such as debugging and calibration of initial values ​​for demand matching degree thresholds, delivery priority weights, load thresholds and idle thresholds, path risk level thresholds, and comprehensive status health thresholds. Meanwhile, all the specific parameters listed in the various embodiments, such as power response speed, power decay rate, weight ratio, and health measurement value, are merely exemplary descriptions to facilitate intuitive understanding of the technical solution of the present invention and do not constitute a limitation on the scope of protection of the present invention. Those skilled in the art can make flexible and dynamic adjustments according to different situations such as the complexity of the actual emergency scenario, airspace environmental conditions, performance configuration of the UAV cluster, and adaptability to the emergency communication environment.

[0063] In one embodiment, an AI-powered remote collaborative control system for emergency food delivery using a drone based on multimodal interaction is provided. This system corresponds to the AI-powered remote collaborative control method for emergency food delivery using a drone based on multimodal interaction described in the previous embodiment. The AI-powered remote collaborative control system for emergency food delivery using a drone based on multimodal interaction includes: The instruction generation module is used to acquire demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios using a multimodal interaction method, verify the demand data through a pre-trained multimodal verification model, and output demand instructions containing implicit demands, non-staple food attributes, emergency scenario correlations and emergency scenario priorities based on the verification results. The task generation module is used to construct priority rules for the delivery of non-staple foods based on demand instructions, and to match drone resources that are suitable for delivery needs through an AI scheduling model to generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. The route generation module is used to obtain the route influencing factors in emergency scenarios, input the collaborative scheduling plan and the route influencing factors into the route planning model, generate a delivery route plan that can avoid the route influencing factors, and set a dynamic route replanning trigger mechanism. The command transmission module is used to generate control commands based on the collaborative scheduling scheme and delivery route scheme, and transmit the control commands to the drone to execute the food delivery task; The feedback and adjustment module is used to acquire real-time status information, including the status of the non-staple food and the flight status of the drone, during the drone delivery process using the multimodal monitoring module. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

[0064] Specific limitations regarding the AI-based drone emergency remote collaborative control system for food delivery based on multimodal interaction can be found in the above-described limitations of the AI-based drone emergency remote collaborative control method for food delivery based on multimodal interaction, and will not be repeated here. Each module in the aforementioned AI-based drone emergency remote collaborative control system for food delivery based on multimodal interaction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0066] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for emergency remote collaborative control of AI-powered drones for non-staple food delivery based on multimodal interaction, characterized in that: Includes the following steps: S10. Use a multimodal interaction method to obtain demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios. Verify the demand data through a pre-trained multimodal verification model. Based on the verification results, output demand instructions containing implicit demands, non-staple food attributes, emergency scenario relationships and emergency scenario priorities. S20. Construct priority rules for the delivery of non-staple foods based on demand instructions, match drone resources that are compatible with delivery needs through an AI scheduling model, and generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. S30. Obtain the path influencing factors in the emergency scenario, input the collaborative scheduling plan and path influencing factors into the path planning model, generate a delivery route plan that can avoid path influencing factors, and set a dynamic path replanning trigger mechanism. S40. Based on the collaborative scheduling scheme and delivery route scheme, generate control commands and transmit the control commands to the drone to execute the non-staple food delivery task; S50. The multimodal monitoring module is used to obtain real-time status information including the status of non-staple food and the flight status of the drone during the drone delivery process. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

2. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 1, characterized in that: Step S10, the training and construction of the pre-trained multimodal verification model, specifically includes the following steps: S101. Obtain multimodal training data, wherein the multimodal training data includes at least speech time-series data, visual image data, and gesture data; S102. Construct an associated tag set, which includes emergency scenario attribute tags, multi-dimensional attribute tags for non-staple foods, implicit demand tags, and association rules between the tags; bind the multimodal training data with the associated tag set to form labeled training samples; S103. Input the training samples into the spatiotemporal attention network architecture, extract speech temporal features through the temporal attention layer, extract visual spatial features through the spatial attention layer, extract gesture spatiotemporal features through the spatiotemporal fusion layer, and perform cross-modal fusion with the speech temporal features and visual spatial features to obtain the basic fusion feature vector. S104. Based on the association rules and basic fusion feature vectors of the association label set, the spatiotemporal attention network architecture is trained to obtain a trained multimodal verification model, which is used to verify the requirement data and output the requirement matching degree and implicit requirement completion results.

3. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 2, characterized in that, Step S10 specifically includes the following steps: S11. Collect multimodal real-time interactive data, including at least voice time-series data, visual image data, and gesture data, and extract real-time demand data containing non-staple food attribute information and emergency scenario demand information from the multimodal real-time interactive data. S12. Input the multimodal real-time interaction data corresponding to the real-time demand data into the trained multimodal verification model, extract speech temporal features through the time attention layer, extract visual spatial features through the spatial attention layer, extract gesture spatiotemporal features through the spatiotemporal fusion layer, and complete cross-modal fusion to generate a real-time fusion feature vector. S13. Based on the real-time fusion feature vector and the association rules of the associated tag set, determine the consistency between the non-staple food attribute information carried by the multimodal real-time interactive data and the emergency scenario demand information, and output the demand matching degree; and based on the association rules between the emergency scenario attribute tags and the non-staple food full-dimensional attribute tags, output the implicit demand completion result. S14. If the demand matching degree is greater than or equal to a preset threshold, based on the real-time demand data and the implicit demand completion result, a demand instruction containing implicit demand, non-staple food attributes, emergency scenario association relationship and emergency scenario priority is generated.

4. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 1, characterized in that, Step S20 specifically includes the following steps: S21. Parse the demand instruction and extract the delivery constraints, emergency scenario priorities, and resource adaptation requirements of implicit needs for the non-staple food. S22. Construct a priority rule for the delivery of non-staple foods based on the priority of emergency scenarios and delivery constraints. In the priority rule for the delivery of non-staple foods, the weight of the emergency scenario level is higher than the weight of the non-staple food category. The resource adaptation requirements of implicit needs are the trigger threshold for adjusting the priority of non-staple food delivery. The delivery tasks are divided into high-priority tasks and ordinary tasks according to the priority of emergency scenarios. S23. Obtain real-time status data of the drone cluster, wherein the real-time status data includes at least the drone's payload capacity, cargo hold temperature control capability, remaining battery power, and current airspace location. S24. Input the food delivery priority rules and the real-time status data of the drone cluster into the AI ​​scheduling model, and generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks through a demand-resource two-way verification mechanism. The relationship includes task execution order, resource-specific binding and abnormal replacement mechanism.

5. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 4, characterized in that, Step S30 specifically includes the following steps: S31. Obtain multi-source dynamic path influencing factors for emergency scenarios. The multi-source dynamic path influencing factors include at least real-time meteorological data, airspace control information, coordinates of temporary obstacles, and time constraints for the transportation of non-staple foods. S32. Perform data fusion processing on the multi-source dynamic path influencing factors, and generate standardized path constraint parameters through a weighted fusion algorithm of static basic data and dynamic real-time data; S33. Based on the food delivery priority rules in the collaborative scheduling scheme, the standardized path constraint parameters are input into the path planning model to generate a delivery path scheme that includes the main path and backup paths. In the delivery path scheme, the priority weight of the main path of high-priority tasks is higher than the priority weight of the main path of ordinary tasks. S34. Set up a hierarchical triggering mechanism for dynamic path replanning, with different replanning levels corresponding to the change range of multi-source dynamic path influencing factors. The replanning levels include local fine-tuning, path switching, and global replanning. The hierarchical triggering mechanism is associated with the abnormal replacement mechanism within the collaborative scheduling scheme.

6. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 5, characterized in that, Before step S40, the following steps are also included: S301a. Based on the generated delivery route plan and the priority rules for non-staple food delivery, obtain the task path execution parameters corresponding to the tasks bound to each drone in the drone cluster. The task path execution parameters include the total path mileage, estimated energy consumption and timeliness requirements. S302a. Obtain the dynamic load parameters of each drone in the drone cluster, wherein the dynamic load parameters include real-time remaining power, available cargo space resources, and cumulative load of executed tasks; S303a. Construct a cluster load balancing evaluation model, wherein the path load weight corresponding to high priority tasks is higher than the path load weight of ordinary tasks in the cluster load balancing evaluation model. S304a. Input the task path execution parameters and dynamic load parameters corresponding to each UAV into the cluster load balancing evaluation model, and output the load saturation of each UAV. S305a. If the load saturation of at least one drone bound to a high-priority task and a normal task exceeds a preset load threshold, and the load saturation of at least one drone bound to a normal task is lower than a preset idle threshold, then the cluster load is determined to be unbalanced. S306a When a cluster load imbalance is determined, the collaborative scheduling scheme is dynamically adjusted according to the priority rules for non-staple food delivery. The ordinary tasks carried on drones whose load saturation exceeds the preset load threshold and are simultaneously bound to high-priority tasks and ordinary tasks are transferred to ordinary task drones whose load saturation is below the preset idle threshold. During the transfer, priority is given to ensuring that the exclusive resource binding between the high-priority task and the original drone and the corresponding delivery path remain unchanged. After the transfer, it is necessary to ensure that the comprehensive load of the drone undertaking the ordinary task does not exceed the preset load threshold in order to update the collaborative scheduling scheme.

7. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 5, characterized in that, Before step S40, the following steps are also included: S301b. Construct a path pre-flight simulation model. Input the real-time performance parameters of the UAV, the real-time multi-source dynamic path influencing factors, and the delivery path scheme including the main path and the backup path into the path pre-flight simulation model. The real-time performance parameters of the UAV include at least the power response speed, attitude adjustment sensitivity, and remaining power decay rate. S302b: The potential risks during the entire path execution process are deduced through the path pre-flight simulation model. The potential risks include at least the risk of excessive track deviation, endurance breakpoint, and attitude instability in strong turbulent airspace, and high-risk sections are marked. S303b If the pre-simulation results show that high-risk road sections will affect the reliability of delivery task execution, the hierarchical triggering mechanism of dynamic route replanning will be triggered to enable backup routes for high-risk road sections; if there is no high risk, the final executable delivery route plan will be output.

8. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 1, characterized in that, Step S40 specifically includes the following steps: S41. Based on the delivery route scheme and the collaborative scheduling scheme, extract the task execution instruction elements corresponding to each UAV. The instruction elements include at least the flight track coordinates, task execution sequence, cargo hold environmental control parameters, airspace access permission certificate and emergency communication channel. S42. Construct a differentiated instruction generation model and configure exclusive instruction templates for different levels of tasks according to the priority rules for non-staple food delivery. Among them, the instruction templates for high-priority tasks include priority airspace passage instructions, cargo hold environment real-time monitoring instructions, and abnormal emergency circuit breaker instructions, while the instruction templates for ordinary tasks include basic flight control instructions and routine status feedback instructions. S43. Input the instruction elements into the differentiated instruction generation model to generate a collaborative control instruction set for each UAV. The collaborative control instruction set has an instruction priority identifier, and the control instruction corresponding to a high-priority task has a higher priority than that of a normal task. S44. Perform multimodal verification on the collaborative control instruction set. Through a triple verification mechanism of instruction syntax and logic verification, UAV performance matching verification, and path compliance verification, invalid or conflicting instructions are eliminated. S45. The verified collaborative control command set is sent to the UAV cluster in order of command priority. At the same time, a command sending receipt and breakpoint resume mechanism is established to ensure that all UAVs receive and confirm the command before starting the task execution process.

9. The AI-powered remote collaborative control method for emergency food delivery using unmanned aerial vehicles based on multimodal interaction as described in claim 5, characterized in that, Step S50 specifically includes the following steps: S51. Use a multimodal monitoring module to collect flight status data of each drone in the drone cluster and status data of the delivered food. The flight status data includes at least the drone's real-time trajectory deviation and the rate of change of remaining battery power. The food status data includes at least the cargo hold temperature fluctuation and the integrity of the cargo packaging. S52. Construct a multimodal state fusion judgment model, and perform weighted fusion processing on the flight state data and food status data corresponding to each UAV. Among them, the weight of food status data is higher than that of flight state data. Based on the fusion result, generate the comprehensive state health of each UAV. S53. A three-level threshold range for the overall health status is preset. The level is dynamically adjusted to adapt to the differentiated activation and path replanning hierarchical triggering mechanism corresponding to different threshold ranges. Specifically: Slightly abnormal range: Only minor adjustments are made to the flight path of the corresponding UAV; Moderately abnormal range: Trigger the abnormal replacement mechanism in the collaborative scheduling scheme, activate the backup drone to take over the original task, and make adaptive adjustments to the delivery route of the original drone; Severe abnormality Interval: Pause all delivery tasks within the current area and regenerate the collaborative scheduling plan and the corresponding delivery route plan; S54. After completing the dynamic adjustment of the corresponding level, continuously collect status information through the multimodal monitoring module and update the overall health status in real time.

10. A multimodal interaction-based AI drone emergency remote collaborative control system for food delivery, used to implement the steps of the multimodal interaction-based AI drone emergency remote collaborative control method for food delivery as described in any one of claims 1-9, characterized in that, include: The instruction generation module is used to acquire demand data containing information on the attributes of non-staple foods and the demand information of emergency scenarios using a multimodal interaction method, verify the demand data through a pre-trained multimodal verification model, and output demand instructions containing implicit demands, non-staple food attributes, emergency scenario correlations and emergency scenario priorities based on the verification results. The task generation module is used to construct priority rules for the delivery of non-staple foods based on demand instructions, and to match drone resources that are suitable for delivery needs through an AI scheduling model to generate a collaborative scheduling scheme that includes the relationship between drones and delivery tasks. The route generation module is used to obtain the route influencing factors in emergency scenarios, input the collaborative scheduling plan and the route influencing factors into the route planning model, generate a delivery route plan that can avoid the route influencing factors, and set a dynamic route replanning trigger mechanism. The command transmission module is used to generate control commands based on the collaborative scheduling scheme and delivery route scheme, and transmit the control commands to the drone to execute the food delivery task; The feedback and adjustment module is used to acquire real-time status information, including the status of the non-staple food and the flight status of the drone, during the drone delivery process using the multimodal monitoring module. When the change of the status information exceeds the preset change threshold range, feedback information is generated and the collaborative scheduling scheme and delivery route scheme are dynamically adjusted.

Images