Air hospital intelligent logistics distribution method based on deep reinforcement learning

Abstract

The invention provides an air hospital intelligent logistics distribution method based on deep reinforcement learning, and the method comprises the steps: receiving all target tasks, determining the priorities of the tasks through a hierarchical task decision model, sorting all the tasks according to the priorities, and entering a scheduling stage; constructing a task and aircraft resource matching model to perform single-task and single-aircraft matching between tasks and aircrafts, performing multi-aircraft cooperative scheduling task allocation in combination with the cooperation cost of the aircrafts, and performing optimization adjustment on the priority based on real-time monitoring; generating a flight instruction based on the optimized and adjusted priority; an optimized flight instruction is generated by adopting a fuzzy logic control method, and dynamic path adjustment is performed on the total flight path to ensure that the aircraft always keeps the optimal flight path in the task execution process; feedback of the flight instruction for executing optimization is collected, and dynamic optimization is conducted on the flight task.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep reinforcement learning, and particularly relates to an intelligent logistics distribution method for airborne hospitals based on deep reinforcement learning. Background Art

[0002] With the rapid development of drone technology, airborne hospitals, as a new type of emergency medical rescue mode, have gradually become an important tool for improving the efficiency of emergency treatment. Airborne hospitals realize the distribution of emergency medical supplies and the referral of patients through a drone platform, which can significantly shorten the rescue time. Especially in the case of limited ground transportation or post-disaster emergency rescue, timely treatment and transportation services can be provided for critically ill patients. However, the existing airborne hospital systems still face a series of technical problems, especially in the multi-task coordination and scheduling of emergency medical supplies and referred patients.

[0003] First of all, in the existing technology, although significant progress has been made in drone path planning technology, in practical applications, most existing path planning systems still focus on the optimization of a single task, such as the flight path calculation of a single drone or the distribution path optimization of a certain type of supplies. This single-task optimization method is particularly insufficient in complex airborne hospital application scenarios. The operation of an airborne hospital requires the simultaneous coordination of the distribution of emergency medical supplies and the referral of patients, and these tasks have different priorities and variable environmental constraints. It is difficult for existing drone path planning systems to simultaneously consider the scheduling and path planning of multiple tasks, resulting in low task execution efficiency and inability to meet emergency medical needs in a timely manner.

[0004] Secondly, the existing emergency medical supply distribution and patient referral scheduling systems lack an effective mechanism for dynamically adjusting task priorities. For example, in the face of complex environments, such as when multiple drones are performing tasks simultaneously or sudden bad weather occurs, traditional systems often cannot flexibly adjust the priorities of various tasks. For the patient referral task, especially the referral of critically ill patients, its priority should be higher than that of general medical supply distribution tasks, but existing systems often do not take into account the dynamic changes of tasks, resulting in the referral task not being given priority, thus delaying the treatment of patients.

[0005] In addition, the flight path planning in existing systems is usually based on static algorithms and fails to consider real-time environmental changes, such as flight altitude, meteorological factors, ground obstacles, etc. In the actual flight process, these factors have important impacts on flight safety, efficiency, and time. Especially during the patient referral process, the smoothness and safety of flight are crucial. The existing technology is slow to respond to these dynamic factors and cannot achieve real-time adaptive adjustment, posing certain flight risks.

[0006] In summary, the biggest technical challenge faced by the airborne hospital is how to coordinate the execution of material distribution and patient referral tasks in a multi-task, multi-constraint, and dynamic environment. Existing technologies cannot effectively integrate multiple tasks, resulting in low efficiency and untimely response of the system when facing complex and urgent tasks. Summary of the Invention

[0007] The object of the present invention is to propose an intelligent logistics distribution method for an airborne hospital based on deep reinforcement learning, which realizes the intelligent distribution of emergency medical supplies and patient referrals in the airborne hospital by combining hierarchical deep reinforcement learning (HDRL) and fuzzy logic control (FLC), and solves the deficiencies in multi-task scheduling and path planning in the existing technologies, especially how to efficiently coordinate task priorities and real-time path adjustment in a multi-task environment.

[0008] To achieve the above object, the present invention provides an intelligent logistics distribution method for an airborne hospital based on deep reinforcement learning, and the method includes: Receiving all target tasks, determining the priorities of the tasks through a hierarchical task decision model, all tasks will be sorted according to the priorities and enter the scheduling stage; the hierarchical task decision model is trained based on the hierarchical deep reinforcement learning method, and high-level task evaluation and low-level task optimization are respectively performed; Constructing a task and aircraft resource matching model to match tasks with aircraft on a one-task and one-aircraft basis, and combining the cooperation costs of the aircraft to perform multi-aircraft cooperation scheduling task allocation, and optimizing and adjusting the priorities based on real-time monitoring; the task and aircraft resource matching model is determined based on the remaining battery power of the aircraft, the flight distance required for the task, and the load requirements of the task; the multi-aircraft cooperation scheduling task allocation is used to minimize communication delay, energy consumption, and task synchronization problems; Based on the optimized and adjusted priorities, determining the path planning optimization objective function to minimize the energy consumption of the total flight path, and simultaneously satisfying the priorities of task scheduling and the dependencies between tasks, and generating flight instructions; the flight instructions satisfy the constraint conditions of the aircraft; the constraint condition is that the maximum flight distance of the aircraft cannot be exceeded; Adopting the fuzzy logic control method to generate optimized flight instructions, and performing dynamic path adjustment on the total flight path to ensure that the aircraft always maintains the optimal flight path during the task execution process; Collecting the feedback of executing the optimized flight instructions, and dynamically optimizing the flight tasks.

[0009] Furthermore, the target tasks are real-time task information obtained from the operation platform of the airborne hospital and the external sensor network, Among them, the real-time task information includes the task type, task deadline, and specific requirements of the task; and the external sensor network will obtain the resource status and environmental information of the aircraft in real time; The high-level task evaluation preliminarily sorts the tasks according to the task type and deadline to obtain the preliminary priority; the low-level task optimization optimizes the scheduling order of the tasks according to the real-time state of the aircraft to obtain the final priority; The hierarchical task decision model is associated with the preliminary priority, task deadline, current time, and maximum acceptable time window of the task.

[0010] Furthermore, the task-aircraft resource matching model is quantified based on the task-aircraft matching degree function, expressed as: ; Wherein, is the matching degree of task and aircraft ; is the remaining battery power of aircraft ; is the flight distance required for task ; is the load requirement of task ; is the task interdependence adjustment coefficient, indicating the mutual dependence during task execution; is the dependence degree of task ; is a regulation parameter that controls the influence of task priority and dependence degree on the matching degree.

[0011] Furthermore, by matching the task and the aircraft based on the task-aircraft matching degree function, and at the same time meeting the task priority, on this basis, it is also necessary to combine the resource utilization efficiency of the aircraft, task time delay, and task collaboration requirements to obtain the multi-objective collaborative optimization function of the task and the aircraft, expressed as: ; Wherein, is whether task is assigned to aircraft , a decision variable of 0 or 1; is the priority of task ; , are the weight coefficients of task priority and resource consumption; is the flight distance required for the execution of task ; is the remaining battery power of aircraft .

[0012] Furthermore, the collaborative cost of the aircraft is considered for the multi-aircraft collaborative scheduling task allocation, including: The collaborative cost of the aircraft includes communication delay, energy consumption, and task synchronization issues among aircraft, as well as whether a single task is allocated to an aircraft and the aircraft ; then the multi-aircraft collaborative scheduling task allocation is used to minimize the collaborative cost of the aircraft.

[0013] Furthermore, the path planning optimization objective function is expressed as: ; where is a decision variable of 0 or 1 indicating whether task is executed collaboratively by aircraft and aircraft ; is the flight distance when task is executed by aircraft ; is the remaining battery power of aircraft ; is the priority of task ; is the dependency between task and aircraft . If task must be completed before task , then is 1, otherwise it is 0; , are the weight coefficients of task priority and task dependency, controlling the influence of each factor on path planning.

[0014] Furthermore, the flight distance when task is executed by aircraft is also dynamically adjusted according to the real-time battery status of the aircraft and the execution progress of the task, generating the adjusted flight distance when task is executed by aircraft , and the adjusted flight distance when task is executed by aircraft meets the constraint conditions.

[0015] Among them, the optimized flight instructions include: Flight path, task allocation order, task execution duration.

[0016] Furthermore, the adoption of the fuzzy logic control method includes: The input variables are transformed into fuzzy sets through fuzzification; fuzzy rules for battery power, task progress, and environmental changes are set; Based on the fuzzy rules, fuzzy inference is carried out to obtain the control output; The output of the fuzzy inference is transformed into a specific path adjustment value; Then, a fuzzy logic control method is adopted to generate the adjustment amount of the path: If the battery power is low, the task progress lags behind, and the environmental change factor is large, the path should be shortened and the flight speed should be reduced; If the battery power is high, the task progress is normal, and the environmental change factor is small, the path remains unchanged; The fuzzy logic control method is used to generate an optimized flight instruction to perform dynamic path adjustment on the total flight path. Specifically: ; Wherein, is the adjusted flight path of the aircraft ; is the aircraft 's initial path; is the aircraft 's maximum battery power; are the coefficients of the fuzzy control, respectively representing the weights of battery power, task progress, and environmental changes on path adjustment.

[0017] Furthermore, the feedback for executing the optimized flight instruction includes battery power change, task progress change, and environmental change.

[0018] Furthermore, the dynamic optimization of the flight task is associated with task priority adjustment, environmental impact adjustment, and task progress correction; Based on the dynamic adjustment factor, the finally adjusted flight path is determined; the dynamic adjustment factor combines task progress, environmental impact, and battery status to determine the adjustment amplitude of the path; the final path is expressed as: ; Wherein, is the finally adjusted flight path; is the aircraft 's adjusted flight path; is the battery power change amount, representing the change in the remaining battery power; is the task progress change, representing the deviation of the current task progress; is the environmental change amount, representing the impact of external factors on the path; are the weight coefficients, respectively used to adjust the influence degrees of battery power, task progress, and environmental factors; is a global adjustment factor used to control the intensity of path adjustment.

[0019] The beneficial technical effects of the present invention are at least as follows: Multi-task scheduling and dynamic priority adjustment: By introducing hierarchical deep reinforcement learning, the present invention enables the system to dynamically adjust the task scheduling strategy according to the urgency and priority of real-time tasks. In the case of multi-task parallel execution, the system can intelligently evaluate the priority of each task to ensure that the task of referring patients can be processed first, while the task of delivering emergency medical supplies is reasonably allocated according to factors such as the type of supplies and the transportation distance. Especially in the face of sudden tasks or changes, the system can automatically adjust the task scheduling to ensure that the task priority can be optimized in real time according to factors such as the environment, the patient's condition, and the drone's status.

[0020] Efficient flight path planning and adaptive adjustment: By combining fuzzy logic control, the present invention can dynamically optimize the flight path during flight according to environmental changes, flight status, and task types (such as patient referral or medical supply delivery). The fuzzy logic system can adjust the flight path in real time to avoid a decrease in flight efficiency caused by environmental changes (such as weather, obstacles, etc.) or changes in the drone's own status (such as power consumption, load, etc.). Especially for the referral task, the system can ensure the smoothness and safety of flight and avoid the deterioration of the patient's condition caused by violent flight.

[0021] Co-optimization of task scheduling and path planning: The key innovation of the present invention is to co-optimize task scheduling and flight path planning. Previous technologies often independently handle task scheduling and path planning, but the present invention closely combines these two modules. Through the scheduling strategy of deep reinforcement learning and the path planning system of fuzzy logic, the system can adjust the drone's path planning in real time according to the priority of the task to ensure that each task can be completed in the shortest time, while avoiding conflicts between path planning and task scheduling and improving the operation efficiency of the entire system.

[0022] Real-time feedback and adaptive adjustment ability: The present invention also introduces a real-time feedback mechanism, enabling the system to adjust the task priority and flight path in a timely manner according to the dynamic changes during flight. For example, during flight, if encountering bad weather, sudden obstacles, or aircraft failures, the system can immediately adjust the task priority or re-plan the optimal path for the task, thereby ensuring the flexibility and emergency response ability of the airborne hospital system. Brief Description of the Drawings

[0023] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the following drawings.

[0024] Figure 1 This is the flowchart of the intelligent logistics distribution method for the air hospital based on deep reinforcement learning of the present invention. Detailed implementation manners

[0025] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0026] As Figure 1 described, the intelligent logistics distribution method for the air hospital based on deep reinforcement learning provided by the embodiments of the present invention includes the following: S1. Receive all target tasks, determine the priority of the tasks through a hierarchical task decision model, and all tasks will be sorted according to the priority and enter the scheduling stage; the hierarchical task decision model is trained based on the hierarchical deep reinforcement learning method, and high-level task evaluation and low-level task optimization are carried out respectively.

[0027] Specifically, the goal of this step is to preliminarily evaluate all the received tasks and assign priorities to each task. The priority evaluation of the tasks not only depends on the urgency of the tasks themselves, but also needs to comprehensively consider various information such as the resource status of the aircraft and environmental factors to ensure that the most urgent and important tasks can be processed first and completed on time.

[0028] In this step, the system first obtains real-time task information from the operation platform of the air hospital and the external sensor network, including task types (such as first-aid material distribution, patient referral, etc.), task time limits, and specific requirements of the tasks. At the same time, the system will also obtain the resource status of the aircraft (such as power, flight speed, load, etc.) and environmental information (such as weather conditions, obstacles, air traffic, etc.) in real time. These data provide a basis for subsequent priority evaluation and task scheduling.

[0029] Furthermore, after the task information and environmental data are collected, the system will conduct a preliminary priority evaluation of all tasks through the high-level task decision model. This evaluation model is trained based on the hierarchical deep reinforcement learning (HDRL) method and can preliminarily sort the tasks according to factors such as task types and deadlines. For example, for the first-aid material distribution task, the system will assign a higher priority to it according to the task time limit and urgency, while for non-urgent tasks, such as general material transportation, the priority is lower.

[0030] In addition, the system will also dynamically adjust the task priorities in combination with the resource status of the aircraft, such as battery power and current flight distance. If the deadline of a certain task is approaching and the current battery power of the aircraft is insufficient to complete the task, the system will automatically assign a higher priority to this task to ensure that the task can be executed in a timely manner. The calculation formula for task priority is as follows: ; where is the final priority of task . is the preliminary priority of task calculated by the high-level decision-making model. is the deadline of task . is the current time. is the maximum acceptable time window of task . and are the weighting coefficients of different factors in task priority calculation.

[0031] Furthermore, after completing the high-level task evaluation, the system will enter the low-level task optimization stage. The main goal of this stage is to further optimize the scheduling order of tasks according to the real-time state of the aircraft (such as remaining battery power, current flight state, flight distance, etc.). If the flight distance of a certain task is too long, the battery power of the aircraft is insufficient or the current state is not suitable for executing the task, the system will make dynamic adjustments according to the available state of the aircraft. The low-level task optimization module dynamically adjusts the task priorities by combining the aircraft state and real-time environmental data.

[0032] Furthermore, when adjusting the task priorities, the system also needs to consider the impact of external environmental factors on task execution. For example, severe weather such as strong winds and thunderstorms may cause the UAV to be unable to fly safely, thus affecting the task priorities and scheduling order. In addition, the load capacity of the aircraft and environmental conditions will also be dynamically considered. If the weather conditions are not suitable for executing a certain task, the priority of this task will be adjusted or postponed to ensure that the aircraft completes the most urgent tasks under the best conditions.

[0033] Furthermore, after comprehensive decision-making and dynamic optimization at the high level and low level, all tasks will be sorted according to the final priorities and enter the scheduling stage. In this stage, the system will allocate the aircraft resources to the most urgent tasks according to the task priorities. The priority sorting of tasks will change dynamically, and tasks with high urgency and tight deadlines will be given priority to ensure that the aerial hospital logistics distribution system can operate efficiently in complex environments and complete all tasks in a timely manner.

[0034] S2. Build a task and aircraft resource matching model to match tasks with aircraft on a one-to-one basis, and perform multi-aircraft collaborative scheduling task allocation considering the collaboration cost of aircraft. Optimize and adjust the priority based on real-time monitoring. The task and aircraft resource matching model is determined based on the remaining battery power of the aircraft, the flight distance required for the task, and the load requirement of the task. The multi-aircraft collaborative scheduling task allocation is used to minimize communication delay, energy consumption, and task synchronization issues.

[0035] Specifically, the core of this step is to establish a matching function between tasks and aircraft to evaluate the suitability of resources between each task and aircraft. The priority of the task has been calculated by the model in step S1, and the resources of the aircraft include the remaining battery power, battery consumption rate, flight distance, and load capacity.

[0036] Furthermore, the present invention designs an innovative matching function that not only considers factors such as the battery, power consumption, and flight distance of the aircraft but also introduces an adjustment coefficient to reflect the dependency between tasks and the possibility of collaboration between aircraft. This can solve the deficiencies of traditional methods that ignore the timing and collaboration issues between tasks.

[0037] The task and aircraft matching function is as follows: ; Where is the matching degree between task and aircraft . is the remaining battery power of aircraft . is the flight distance required for task . is the load requirement of task . is the task dependency adjustment coefficient, indicating the mutual dependency during task execution (for example, some tasks must be executed after the previous task is completed). is the dependency degree of task (i.e., whether task depends on the completion of other tasks). is a regulation parameter that controls the influence of task priority and dependency degree on the matching degree. Here, the exponential term in the matching function ensures the influence of task dependency on scheduling, preventing order conflicts and resource competition between tasks. By adjusting and , the task priority and dependency relationship can be flexibly controlled to achieve dynamic scheduling.

[0038] Furthermore, after calculating the matching degree between tasks and the aircraft, the present invention needs to allocate all tasks to suitable aircraft. The goal is to achieve the best scheduling of tasks within the capabilities of the aircraft while meeting the priorities.

[0039] The present invention designs an objective function that combines the resource consumption of the aircraft and the task priorities for global optimization. This objective function takes into account the resource utilization efficiency of the aircraft, the time delay of the tasks, and the collaborative requirements between tasks. The optimization expression of the objective function is as follows: ; Where, is a decision variable indicating whether task is allocated to the aircraft , which is either 0 or 1. is the priority of task (output from step 1). , are the weight coefficients of task priority and resource consumption. is the flight distance required for the execution of task . is the remaining battery power of the aircraft .

[0040] This objective function combines the task priorities and the resource consumption of the aircraft to optimize the task scheduling arrangement. By introducing and , the system can flexibly adjust the balance between task priorities and aircraft resource consumption.

[0041] Furthermore, during the task execution process, problems such as a decrease in the aircraft's battery power and task delays may occur. To address these uncertainties, the present invention designs a dynamic task adjustment mechanism that can provide real-time feedback and adjustment based on the task execution situation and the aircraft's status.

[0042] The dynamic adjustment is based on the real-time battery power of the aircraft and the task execution progress, and adjusts the task priorities in a timely manner. The dynamic adjustment is carried out through the following formula: ; Where, is the new priority of task . is the original priority of task . is the adjustment coefficient. is the real-time battery power of the aircraft . is the aircraft The primary battery power. This formula dynamically adjusts the task priority by monitoring the battery consumption of the aircraft. If the battery power is too low, the system will appropriately reduce the task priority to ensure the stability of the system.

[0043] Furthermore, for some tasks that require multiple aircraft to cooperate, the system needs to achieve resource sharing and collaborative operations among the aircraft. To coordinate the resources of multiple aircraft, the present invention designs a collaborative task scheduling model. This model optimizes the task allocation among the aircraft according to the resource sharing situation among them.

[0044] In this model, the present invention determines the task allocation method by calculating the cooperation cost between the aircraft. The cooperation cost takes into account the communication delay, energy consumption, and task synchronization issues between the aircraft.

[0045] The objective of the collaborative scheduling optimization model is: ; Wherein, is whether the task is simultaneously allocated to aircraft and aircraft , and it is a decision variable of 0 or 1. is the cooperation cost between aircraft and aircraft .

[0046] By minimizing the cooperation cost between the aircraft, the system can achieve effective resource sharing among the aircraft and maximize the task execution efficiency and cooperation performance.

[0047] Furthermore, during the process of the aircraft executing tasks, it is very important to monitor the completion situation of the tasks in real time. If it is found that the execution progress of the task does not meet the expectation, or the resources of the aircraft change, the system will adjust the task priority and aircraft allocation strategy in real time according to the execution situation of the task. Through these feedback mechanisms, the system can flexibly respond to emergencies and ensure the smooth completion of the tasks.

[0048] The mechanism of real-time feedback and task adjustment is based on the battery state, power consumption, and task execution progress of the aircraft. The system dynamically adjusts the scheduling strategy according to the task delay and the resource consumption situation of the aircraft to ensure that the task can be successfully completed according to the adjusted priority. The feedback adjustment rules are as follows: ; Wherein, is the new priority of task . is the original priority of task . is for task to aircraft Time requirements. For the aircraft And the aircraft The collaboration cost between. Is the feedback adjustment coefficient. This mechanism can adjust the task priority and aircraft resource allocation in real time during the task execution process to ensure that the task is completed on time in a changing environment.

[0049] Through the above task scheduling and resource allocation optimization model, the present invention can accurately schedule tasks based on task priority, aircraft resource status, and task interdependencies. This solution introduces new elements into the traditional scheduling method, such as the adjustment of task interdependencies, the collaboration cost between aircraft, and the real-time feedback mechanism, ensuring the dynamics and flexibility of scheduling. In practical applications, this model can effectively solve the resource allocation problem in a multi-task and multi-UAV collaboration environment, achieve the efficient utilization of UAV resources, and ensure the timely completion of tasks.

[0050] S3. Based on the optimized adjusted priority, determine the path planning optimization objective function to minimize the energy consumption of the total flight path, and at the same time satisfy the priority of task scheduling and the dependency relationship between tasks, and generate flight instructions; the flight instructions satisfy the constraint conditions of the aircraft; the constraint condition is that the maximum flight distance of the aircraft cannot be exceeded.

[0051] Specifically, the core objective of this step is to design the flight path planning of the aircraft based on the task priority, aircraft resource status, and task interdependencies output by the task scheduling optimization model. This path planning not only needs to optimize the flight route of the aircraft but also consider the execution order and timeliness of tasks. Path planning is a key part of aircraft task scheduling and resource allocation, and it is necessary to accurately calculate the flight route of the aircraft to meet the task requirements and maximize resource utilization. To achieve this goal, the following is a detailed step description.

[0052] The input data for path planning comes from the optimization results of step S2, including: Task priority ( ): The priority of each task obtained through task scheduling optimization.

[0053] Aircraft resources ( ): The remaining battery power of each aircraft.

[0054] Task interdependency ( ): Whether there is a dependency relationship between tasks (for example, task Must be executed after task Is completed).

[0055] Flight distance and load requirements ( , ): The flight distance required for task execution and the payload requirements of the task.

[0056] Furthermore, to ensure path optimization, the objective function needs to comprehensively consider factors such as the energy consumption of the aircraft, flight distance, task priority, and dependencies between tasks. The present invention designs a new path planning optimization objective function with the goal of minimizing the energy consumption of the total flight path while satisfying the priority of task scheduling and the dependencies between tasks. The objective function is as follows: ; where, is a decision variable of 0 or 1 indicating whether task is executed collaboratively by aircraft and aircraft . is the flight distance of task when executed by aircraft . is the remaining battery power of aircraft . is the priority of task (given by the optimization result of step 2). is the dependency between task and aircraft . If task must be completed before task , then is 1, otherwise it is 0. , are the weight coefficients of task priority and task dependency, controlling the influence of each factor on path planning.

[0057] This objective function comprehensively considers the remaining battery of the aircraft, power consumption, task priority, and task dependencies. The relationship between the flight distance of the aircraft and the battery power is reflected by the term . Task priority is introduced through to ensure that high-priority tasks can be arranged for execution first. Task dependency is ensured through for the execution order of tasks. By adjusting and , the influence of task priority and task dependency on path planning can be flexibly adjusted.

[0058] Furthermore, in addition to the objective function of minimizing the total path, path planning also needs to consider the capacity limitations of the aircraft. Each aircraft is restricted by its flight capabilities, such as battery power and payload capacity, when executing tasks. Therefore, constraints on the aircraft's capabilities must be added to the path planning to ensure that the aircraft can complete the tasks assigned to it.

[0059] Let the aircraft perform a mission The flight distance during the mission is , and the maximum flight distance of the aircraft is , then the present invention has the following constraint conditions: ; Wherein, is the decision variable (0 or 1) for whether the mission is performed by the aircraft . is the flight distance required for the aircraft to perform the mission . is the maximum flight distance limit of the aircraft . This constraint condition ensures that the flight distance of the aircraft does not exceed its maximum capacity, avoiding the situation where the aircraft cannot complete the mission due to battery power or other resource limitations.

[0060] Furthermore, during the flight path planning process, due to environmental changes, mission delays, or changes in the aircraft's state (such as a decrease in battery power), it may be necessary to dynamically adjust the path. The present invention designs a real-time feedback mechanism to dynamically adjust the flight path according to the real-time battery state of the aircraft and the progress of the mission execution.

[0061] The real-time adjustment mechanism adjusts the path planning based on the battery consumption of the aircraft and the mission progress, using the following feedback formula: ; Wherein, is the adjusted flight distance during the execution of the mission by the aircraft . is the initial flight distance during the execution of the mission by the aircraft . is the dynamic adjustment coefficient, which is used to control the amplitude of path adjustment. is the change in the battery power of the aircraft , that is, the difference between the real-time battery power and the initial battery power.

[0062] Through this formula, the system can dynamically adjust the path planning according to the battery consumption of the aircraft. If the battery power of the aircraft drops rapidly, the path planning is adjusted to reduce unnecessary flight distance and ensure that the aircraft can complete the current mission.

[0063] Furthermore, the ultimate goal of path planning is to generate flight instructions for the aircraft. These instructions generate the specific flight paths for each aircraft based on the scheduling priorities of the tasks, the flight capabilities of the aircraft, and the dependencies of the tasks. These path instructions not only need to consider the timeliness of the tasks but also ensure the reasonable allocation of the resources of the aircraft. Finally, the system will generate the following flight instructions according to the optimization results: Flight path: The optimal path for each aircraft from the starting point to the task target.

[0064] Task assignment order: Ensure that the aircraft execute tasks in the optimal order to meet task dependencies.

[0065] Task execution duration: Allocate a reasonable time window for each task to ensure that the tasks are completed on time.

[0066] The generation of flight instructions is based on the optimization results and constraint conditions, ultimately ensuring that each aircraft can complete the tasks according to the optimal path while meeting the resource limitations of the aircraft.

[0067] S4. Adopt the fuzzy logic control method to generate optimized flight instructions and perform dynamic path adjustment on the total flight path to ensure that the aircraft always maintains the optimal flight path during the task execution process.

[0068] Specifically, this step aims to combine the aforementioned path planning model with the real-time control system, apply the fuzzy logic control method, and perform dynamic adjustment on the flight path according to the real-time state of the aircraft (such as battery power, aircraft position, task progress, and environmental changes). Through this solution, it is possible to effectively respond to the dynamic changes in the environment and the fluctuations in the aircraft state, ensuring the stable progress of the flight mission. The following are the specific steps and detailed solutions.

[0069] The present invention designs a path adjustment scheme based on fuzzy logic. The core of fuzzy logic control is to dynamically adjust the key parameters of the flight path according to the real-time state of the aircraft and environmental changes. The specific control process is divided into the following steps: Fuzzify the input variables: Convert the input variables into fuzzy sets through fuzzification and set the fuzzy rules for battery power, task progress, and environmental changes.

[0070] Fuzzy inference: Based on the fuzzy rules, perform fuzzy inference to obtain the control output (i.e., the path adjustment value).

[0071] Defuzzify: Convert the output of the fuzzy inference into a specific path adjustment value.

[0072] Furthermore, through the following fuzzy rules, the present invention determines the adjustment amount of the path: If the battery power is low, and the task progress Lag, and environmental change factors are large, then the path should be shortened and the flight speed reduced.

[0073] If the battery power is high, the mission progress is normal, and the environmental change factors are small, then the path can remain unchanged or be optimized and extended in some cases.

[0074] This fuzzy logic control can respond flexibly to the state of the aircraft, ensuring real-time optimization of the flight path.

[0075] Furthermore, the fuzzy control output is the flight path adjustment amount ( ), and the adjusted path is dynamically corrected based on the input real-time data (battery power, mission progress, environmental change). The present invention proposes an innovative formula that combines the priorities of the aircraft battery power and mission progress to dynamically adjust the weight of the flight path. The path adjustment formula is as follows: ; Wherein, is the adjusted flight path of the aircraft (output from fuzzy control). is the initial path of the aircraft (path planning result from step 3). is the maximum battery power of the aircraft . are the coefficients of fuzzy control, respectively representing the weights of battery power, mission progress, and environmental change on path adjustment.

[0076] Through this path adjustment formula, the present invention can flexibly shorten or optimize the path according to real-time situations, enabling the aircraft to quickly adapt and efficiently complete tasks in different environments.

[0077] Furthermore, as the flight mission progresses, the real-time state of the aircraft will continuously change, and the battery power, environmental factors, and mission progress may all fluctuate. To respond to these changes in real time, the present invention introduces a real-time feedback mechanism to dynamically correct the flight path. The specific method is to monitor the change in the aircraft's battery power ( ) and the environmental change factor ( ), and further adjust the flight path based on these changes.

[0078] To enhance the adaptive ability of the path, the present invention designs the following correction formula: ; Wherein, is the corrected flight path. It is the adjusted path after fuzzy control. is the correction coefficient used to adjust the intensity of path correction. is the aircraft The change in battery power of the aircraft. This correction formula enables the aircraft to dynamically correct the path according to the real-time change in battery power, so as to avoid the aircraft deviating from the target path or failing to complete the task when the battery power is too low.

[0079] Further, finally, based on the corrected path ( ), the system will generate real-time control instructions, including control parameters such as the traveling direction, speed, and flight altitude of the aircraft. The generation of control instructions is based on the path planning in step 3 and the dynamic adjustment in step 4, ensuring that the aircraft can adjust its flight strategy according to real-time feedback information in any state, so as to complete the task.

[0080] These control instructions will be executed in real time through the control system of the aircraft to ensure that the aircraft always maintains the optimal flight path during the task execution.

[0081] S5. Collect the feedback of the optimized flight instructions and optimize the flight task dynamically.

[0082] This step aims to optimize the flight task dynamically through a feedback mechanism. By combining the path adjustment results of the previous step 4, the present invention will optimize the flight path according to the real-time state of the aircraft (including battery power, task progress, environmental factors, etc.) to ensure the smooth completion of the task. The core idea of this process is real-time feedback and dynamic path adjustment to cope with uncertainties.

[0083] Specifically, in order to cope with dynamic changes, this step designs a feedback-based task optimization mechanism. By evaluating the state of the aircraft (including battery power, task progress, and environmental factors) in real time, the present invention calculates the amplitude of path adjustment that needs to be made in real time.

[0084] Among them, the goal is to dynamically adjust the path priority according to the urgency of the task and the current state of the aircraft (such as battery power, power consumption, etc.).

[0085] Environmental impact adjustment: Consider the impact of environmental changes on the path into the adjustment decision, such as weather factors like strong wind, rain, and snow.

[0086] Task progress correction: Judge whether the aircraft is traveling as planned according to the real-time task progress. If there is a delay or early arrival, adjust the flight path.

[0087] Further, the present invention introduces a dynamic adjustment factor, which comprehensively considers the task progress, environmental impact, and battery state to determine the amplitude of path adjustment. The specific formula is: ; Among them, is the finally adjusted flight path. is the adjusted path obtained in step 4. is the change in battery power, indicating the change in the remaining battery power. is the change in task progress, indicating the deviation of the current task progress. is the environmental change amount, indicating the influence of external factors (such as wind speed, air pressure, etc.) on the path. is the weight coefficient, which is used to adjust the influence degrees of battery power, task progress, and environmental factors respectively. is a global adjustment factor, which is used to control the intensity of path adjustment.

[0088] This formula integrates the state of the aircraft (such as battery, power, etc.) and external environmental factors (such as wind speed, air pressure, etc.) for weighted adjustment to ensure that the flight path adapts to the actual needs under different conditions.

[0089] Furthermore, according to the adjusted path, the present invention will formulate control decisions for the aircraft: Path correction: The adjusted flight path will be corrected according to factors such as environmental changes and task progress to ensure that the aircraft continues to execute the task with the most appropriate path.

[0090] Speed and flight strategy adjustment: While adjusting the path, the present invention also needs to adjust the speed, heading, and flight mode (such as energy-saving mode, priority task mode, etc.) of the aircraft.

[0091] Task priority reallocation: If the battery power of the aircraft is not sufficient to support the completion of the task, the system will adjust the task priority according to the urgency of the task and skip some secondary tasks if necessary.

[0092] Through this feedback mechanism, the aircraft can adjust the path according to real-time data to ensure that the task can be executed as expected, whether in a normal flight environment or when encountering unforeseen external interferences.

[0093] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0094] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a division of logical functions. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0095] If the described functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, and other various media that can store program codes.

[0096] Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions, and deformations can be made to these embodiments without departing from the principles and purposes of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. An intelligent logistics distribution method for air hospitals based on deep reinforcement learning, characterized in that The method includes: Receiving all target tasks, determining the priority of the tasks through a hierarchical task decision model. All tasks will be sorted according to the priority and enter the scheduling phase. The hierarchical task decision model is trained based on the hierarchical deep reinforcement learning method, and high-level task evaluation and low-level task optimization are carried out respectively. Constructing a task and aircraft resource matching model to match tasks with aircraft on a one-to-one basis, and combining the cooperation cost of the aircraft to perform multi-aircraft cooperative scheduling task allocation, and optimizing and adjusting the priority based on real-time monitoring. The task and aircraft resource matching model is determined based on the remaining battery power of the aircraft, the flight distance required for the task, and the load requirements of the task. The multi-aircraft cooperative scheduling task allocation is used to minimize communication delay, energy consumption, and task synchronization problems. Based on the optimized and adjusted priority, determining the path planning optimization objective function to minimize the energy consumption of the total flight path, and at the same time satisfying the priority of task scheduling and the dependencies between tasks, and generating flight instructions. The flight instructions satisfy the constraint conditions of the aircraft. The constraint condition is that the maximum flight distance of the aircraft cannot be exceeded. Adopting a fuzzy logic control method to generate optimized flight instructions, and performing dynamic path adjustment on the total flight path to ensure that the aircraft always maintains the optimal flight path during the task execution process. Collecting the feedback of executing the optimized flight instructions and dynamically optimizing the flight task.

2. The intelligent logistics distribution method of the airborne hospital based on deep reinforcement learning according to claim 1, wherein, The target tasks are real-time task information obtained from the operation platform of the air hospital and the external sensor network. Among them, the real-time task information includes task type, task time limit, and specific requirements of the task. And the external sensor network will obtain the resource status and environmental information of the aircraft in real time. The high-level task evaluation preliminarily sorts the tasks according to the task type and deadline to obtain the preliminary priority. The low-level task optimization optimizes the scheduling order of the tasks according to the real-time state of the aircraft to obtain the final priority. The hierarchical task decision model is associated with the preliminary priority, the deadline of the task, the current time, and the maximum acceptable time window of the task.

3. The intelligent logistics distribution method of the air hospital based on deep reinforcement learning according to claim 1, wherein The task and aircraft resource matching model is quantified based on the task and aircraft matching degree function, expressed as: ； wherein, is the matching degree between the mission and the aircraft; is the remaining battery power of the aircraft; is the flight distance required for the mission; is the payload requirement of the mission; is the inter - mission dependency adjustment coefficient, indicating the mutual dependency during mission execution; is the dependency degree of the mission; is an adjustment parameter that controls the influence of mission priority and dependency degree on the matching degree.

4. The intelligent logistics distribution method of the airborne hospital based on deep reinforcement learning according to claim 2 or 3, characterized in that By matching tasks with aircraft based on the task and aircraft matching degree function, and at the same time satisfying the priority of the tasks. On this basis, it is also necessary to combine the resource utilization efficiency of the aircraft, the time delay of the tasks, and the collaborative requirements between tasks to obtain the multi-objective collaborative optimization function of tasks and aircraft, expressed as: ； Among them, is the task whether it is assigned to the aircraft , a decision variable of 0 or 1; is the task priority; , are the weight coefficients of task priority and resource consumption; is the flight distance required for the execution of the task ; is the remaining battery power of the aircraft .

5. The intelligent logistics distribution method for an air hospital based on deep reinforcement learning according to claim 4, characterized in that The multi-aircraft cooperative scheduling task allocation by combining the cooperation cost of the aircraft includes: The collaborative costs of the aircraft include communication delays, energy consumption, and task synchronization issues among aircraft, as well as whether a single task is assigned to aircraft simultaneously and the aircraft ; then the multi-aircraft collaborative scheduling task assignment is used to minimize the collaborative costs of the aircraft.

6. The intelligent logistics distribution method for an air hospital based on deep reinforcement learning according to claim 5, characterized in that The path planning optimization objective function is expressed as: ； Among them, is the task whether it is executed collaboratively by the aircraft and the aircraft is a decision variable of 0 or 1; is the task the flight distance during the execution of the aircraft ; is the remaining battery power of the aircraft ; is the task priority; is the task and the aircraft the dependency relationship between them; if the task must be completed before the task then is 1, otherwise it is 0; , are the weight coefficients of task priority and task dependency, controlling the influence of each factor on path planning.

7. The intelligent logistics distribution method of the air hospital based on deep reinforcement learning according to claim 6, characterized in that, The said task During the execution of the flight mission of the aircraft The flight distance is also dynamically adjusted according to the real-time battery status of the aircraft and the progress of the task execution, and a task During the execution of the flight mission of the aircraft The adjusted flight distance is generated, and the said task During the execution of the flight mission of the aircraft The adjusted flight distance meets the said constraint conditions; Among them, the optimized flight instructions include: Flight path, task allocation order, task execution duration.

8. The intelligent logistics distribution method of the air hospital based on deep reinforcement learning according to claim 1, characterized in that The adoption of the fuzzy logic control method includes: Converting the input variables into fuzzy sets through fuzzy processing; setting fuzzy rules for battery power, task progress, and environmental changes. Based on fuzzy rules, perform fuzzy inference to obtain the control output; Convert the output of the fuzzy inference into a specific path adjustment value; Then adopt the fuzzy logic control method to generate the adjustment amount of the path: If the battery power is low, the task progress lags behind, and the environmental change factor is large, the path should be shortened and the flight speed should be reduced; If the battery power is high, the task progress is normal, and the environmental change factor is small, the path remains unchanged; The above-mentioned fuzzy logic control method is used to generate an optimized flight instruction to perform dynamic path adjustment on the total flight path, specifically: ； Wherein, is the adjusted flight path of the aircraft; is the initial path of the aircraft; is the maximum battery power of the aircraft; is the initial path of the aircraft; is the aircraft is the maximum battery power of the aircraft; are the coefficients of fuzzy control, respectively representing the weights of battery power, mission progress, and environmental changes on path adjustment.

9. The intelligent logistics distribution method for the air hospital based on deep reinforcement learning according to claim 1, wherein The feedback of executing the optimized flight instruction includes changes in battery power, task progress, and environmental changes.

10. The intelligent logistics distribution method for the air hospital based on deep reinforcement learning according to claim 9, wherein The above-mentioned dynamic optimization of the flight task is associated with task priority adjustment, environmental impact adjustment, and task progress correction; Determine the finally adjusted flight path based on the dynamic adjustment factor; the dynamic adjustment factor combines the task progress, environmental impact, and battery status to determine the adjustment amplitude of the path; the final path is expressed as: ； Among them, is the finally adjusted flight path; is the aircraft 's adjusted flight path; is the change in battery power, indicating the change in the remaining battery power; is the change in mission progress, indicating the deviation of the current mission progress; is the environmental change amount, indicating the impact of external factors on the path; is the weight coefficient, which is used to adjust the influence degrees of battery power, mission progress and environmental factors respectively; is a global adjustment factor, which is used to control the intensity of path adjustment.

Images