Multi-modal large model driven unmanned aerial vehicle computing power adaptive scheduling method and system

Abstract

This invention relates to the field of UAV onboard computing and resource scheduling technology, and discloses a multimodal large model-driven UAV computing power adaptive scheduling method and system. This multimodal large model-driven UAV computing power adaptive scheduling method calculates the dynamic priority weights of each AI perception task in real time; dynamically determines the minimum computing power reservation value for flight control tasks; calculates the actual computing power requirement by monitoring the actual computing power consumed by each task and compares it with the theoretical allocation value to obtain the scheduling error; further, it operates under the same computing power quota, and reversely calculates the current available computing power level by measuring the completion frequency, thereby correcting the scheduling error; simultaneously, it establishes an environmental factor influence coefficient to correct the computing power quota; and finally, it allocates the computing power through a joint computing power budget. This invention significantly improves the utilization efficiency of UAV onboard computing power and the fairness of task scheduling while ensuring flight safety, and possesses long-term adaptive capabilities.

Description

Technical Field

[0001] This invention relates to the field of UAV onboard computing and resource scheduling technology, specifically to a multimodal large model-driven UAV computing power adaptive scheduling method and system. Background Technology

[0002] In recent years, multimodal large models (fusing data from vision, LiDAR, and voice) have been widely used in UAV environmental perception, target recognition, and autonomous navigation. However, UAV onboard computing platforms face hard constraints in computing power, power consumption, and real-time performance, which are fundamentally different from general edge computing environments: UAVs rely on limited batteries, and high computing power leads to high power consumption, directly shortening flight range and affecting flight safety; flight control tasks (attitude control, obstacle avoidance response) have millisecond-level hard real-time constraints, and computational delays may cause crashes; the flight environment (obstacle density, lighting, airflow) changes rapidly, and the computing power requirements for perception tasks fluctuate drastically.

[0003] However, existing UAV scheduling technologies mostly employ fixed priorities or simple equal allocation, failing to dynamically adjust computing power allocation based on real-time sensing data, leading to computing power mismatch. This invention addresses this problem by providing a UAV computing power adaptive scheduling method and system. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal, large-model-driven UAV computing power adaptive scheduling method and system, thereby solving the above-mentioned technical problems:

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A multimodal large model-driven UAV computing power adaptive scheduling method, the method comprising the following steps:

[0007] Step S1: Acquire real-time data from the drone's onboard multimodal sensors and calculate dynamic priority weights for each AI perception task;

[0008] Step S2: Monitor the real-time flight status and flight phase of the UAV, and dynamically calculate the minimum computing power reserve value for the flight control task according to the preset safety strategy;

[0009] Step S3: In each scheduling cycle, record the curve of the actual computing power resources occupied by each task changing over time, and calculate the actual computing power consumption by time integration; calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data; and use the difference between the actual consumption and the theoretical requirement as the scheduling error.

[0010] Step S4: Periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and back-calculate the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value.

[0011] Step S5: Collect at least one of the following environmental data: airborne temperature, battery voltage, and air pressure; and generate a comprehensive environmental correction coefficient based on the environmental factor impact table.

[0012] Step S6: Use the benchmark correction value and environmental correction coefficient to calibrate and correct the total available computing power. After deducting the minimum computing power reservation value for the flight control mission, the allocable computing power pool is obtained. Allocate the allocable computing power pool according to the dynamic priority weight ratio, and use scheduling error to correct the allocation result to generate the target computing power quota.

[0013] As a further technical solution, the real-time acquisition of airborne multimodal sensor data from the UAV, and the calculation of dynamic priority weights for each AI perception task, are as follows:

[0014] Real-time acquisition of data from at least the drone's onboard visual camera, LiDAR, millimeter-wave radar, and microphone array multimodal sensors;

[0015] For each AI perception task, the timeliness urgency, safety impact, and scene activity are calculated. The formula for calculating the timeliness urgency is: Timeliness urgency = Maximum allowable delay / (Maximum allowable delay - Time already consumed). The safety impact is the degree of impact of the target task output on the flight safety of the drone, with a preset fixed level. The scene activity is the number of targets, the rate of change of target movement speed, and the signal-to-noise ratio in the current sensor input.

[0016] The dynamic priority weight of the task is obtained by weighting and integrating the urgency of timeliness, the impact of security, and the activity of the scene. After weight normalization, the sum of the weights of all tasks is 1.

[0017] As a further technical solution, the actual computing power consumption is calculated through time integration, using the following formula:

[0018]

[0019] in, Let be the percentage of computing resources used by the i-th task at time t. The integration interval is one scheduling cycle, and it is obtained through the resource monitoring interface of the onboard operating system at a fixed sampling frequency. This is the start time of the scheduling cycle. This is the end time of the scheduling cycle.

[0020] As a further technical solution, the method also includes:

[0021] Step S7: Divide the allocable computing power pool into multiple computing power gradients, monitor the available computing power level in real time, and determine the number of AI perception tasks that need to be reduced when the computing power gradient decreases.

[0022] All AI perception tasks currently awaiting scheduling are virtualized into task targets evenly distributed on a circle. A dynamic ring with gaps is formed based on the current number of effective tasks. The deflection angle is calculated as 360° / the total number of AI perception tasks currently awaiting scheduling. The dynamic ring is rotated on the task targets by the deflection angle at a preset adjustment period. The tasks corresponding to the units where the dynamic ring overlaps with the task targets are active tasks and are allocated computing power. The tasks corresponding to the gaps are dormant tasks and are suspended from execution.

[0023] Step S8: Allocate computing power quotas to the activity task based on the output of step S7 and execute it, monitor the flight safety threat level, and trigger emergency preemption and recovery.

[0024] As a further technical solution, the method for dividing the computing power gradient is as follows:

[0025] Divide the available computing power pool into equal intervals according to a percentage. Each gradient corresponds to a fixed percentage reduction in the number of tasks. The number of tasks to be reduced = the total number of AI perception tasks currently to be scheduled × the percentage of gradient descent, rounded up.

[0026] As a further technical solution, the method for constructing the task target is as follows:

[0027] Create a virtual circle and divide its circumference into M arc segments, where M is the total number of AI perception tasks to be scheduled. The midpoint of each arc segment represents an AI perception task. The tasks are arranged in order of their dynamic priority weight from high to low or in order of task ID. The central angle of each arc segment is 360° / M.

[0028] As a further technical solution, the adjustment period is a fixed time length, which can be one scheduling cycle or several scheduling cycles; after each rotation, the active task set moves backward by one task position, and when the cumulative rotation angle reaches 360°, the active task set returns to the initial position in a loop.

[0029] As a further technical solution, it also includes: when the computing power gradient increases, reducing the number of tasks that need to be paused k, increasing the number of effective units of the dynamic ring N, re-initializing the position of the dynamic ring, and gradually restoring the concurrent execution of all tasks.

[0030] This invention also provides a multimodal large model-driven UAV computing power adaptive scheduling system, comprising the following modules:

[0031] The dynamic priority weight calculation module is used to acquire data from the UAV's onboard multimodal sensors in real time and calculate dynamic priority weights for each AI perception task.

[0032] The minimum computing power reservation value calculation module is used to monitor the real-time flight status and flight stage of the UAV, and dynamically calculate the minimum computing power reservation value for the flight control task according to the preset safety strategy.

[0033] The scheduling error calculation module is used to record the change curve of the actual computing resources occupied by each task over time in each scheduling cycle, calculate the actual computing power consumption through time integration, calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data, and use the difference between the actual consumption and the theoretical requirement as the scheduling error.

[0034] The benchmark correction value acquisition module is used to periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and back-calculate the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value.

[0035] The environmental correction coefficient acquisition module is used to collect at least one type of environmental data, such as airborne temperature, battery voltage, and air pressure, and generate a comprehensive environmental correction coefficient based on the environmental factor influence table.

[0036] The target computing power quota generation module is used to calibrate and correct the total available computing power using the benchmark correction value and the environmental correction coefficient. After deducting the minimum computing power reservation value for flight control tasks, the allocable computing power pool is obtained. The allocable computing power pool is allocated according to the dynamic priority weight ratio, and the allocation result is corrected by using scheduling error to generate the target computing power quota.

[0037] This invention also provides a computing device for a multimodal large model-driven UAV computing power adaptive scheduling method, comprising:

[0038] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0039] The beneficial effects of this invention are:

[0040] This invention relates to a multimodal large model-driven UAV computing power adaptive scheduling method and system. By comparing the time integral of actual computing power consumption with theoretical requirements, the scheduling error is obtained. Then, the actual computing power level is inferred through standardized comparison tasks, effectively eliminating scheduling deviations caused by sensor drift, chip aging, and environmental changes. The impact of environmental factors such as temperature and battery voltage on computing power performance is quantified into correction coefficients, ensuring that the scheduling strategy remains optimal under different environmental conditions. Attached Figure Description

[0041] The invention will now be further described with reference to the accompanying drawings.

[0042] Figure 1 This is a flowchart of the UAV computing power adaptive scheduling method driven by multimodal large model in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the target for the task in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the dynamic ring in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the combination of the task target and the dynamic ring in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the action of the target and the dynamic ring in an embodiment of the present invention;

[0047] Figure 6 This is a block diagram of a multimodal large model-driven UAV computing power adaptive scheduling system in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Please see Figure 1 This invention provides a multimodal large model-driven adaptive scheduling method for UAV computing power, which includes:

[0050] Step S1: Acquire real-time data from the drone's onboard multimodal sensors and calculate dynamic priority weights for each AI perception task. The specific process is as follows:

[0051] The system receives real-time data from multimodal sensors onboard the drone, including visual cameras, LiDAR, millimeter-wave radar, and microphone arrays. For each AI perception task (e.g., target detection, obstacle avoidance trajectory planning, semantic segmentation, and voice command recognition), the following sub-steps are executed:

[0052] The time urgency is calculated based on the maximum allowable delay for the task (e.g., 10 milliseconds for obstacle avoidance and 50 milliseconds for object detection) and the processing time already consumed. The formula is: Time Urgency = Maximum Allowable Delay / (Maximum Allowable Delay - Consumed Time). The larger this value, the more urgent the task.

[0053] Calculate the safety impact level: Based on the degree of impact of the task output on the flight safety of the drone, a fixed level is preset (e.g., 0.9 for obstacle avoidance, 0.5 for target detection, and 0.2 for speech recognition).

[0054] Calculate scene activity: Analyze the number of targets in the current sensor input, the rate of change of target motion speed, signal-to-noise ratio, etc. For example, when 20 obstacles are detected in the visual image, the scene activity is higher than when there is only 1 obstacle;

[0055] The three metrics mentioned above are weighted and fused to obtain the dynamic priority weight of the task. After weight normalization, the sum of the weights of all tasks is 1.

[0056] Step S2: Monitor the real-time flight status and flight phase of the UAV, and dynamically calculate the minimum computing power reserve value for the flight control task based on the preset safety strategy. The specific process is as follows:

[0057] The system monitors the drone's flight status (attitude angles, speed, altitude, battery level) and flight phases (takeoff, cruise, landing, emergency obstacle avoidance) in real time. Based on preset safety strategies, it dynamically calculates the minimum computing power reserve for flight control tasks. For example:

[0058] Cruise phase: 20% of computing power is reserved for flight control tasks; Takeoff / landing phase: 40% is reserved; Emergency obstacle avoidance phase: 80% is reserved.

[0059] The reserved value increases linearly or piecewise with the level of security threat (e.g., the distance to an obstacle detected by radar).

[0060] Step S3: Within each scheduling cycle, record the curve of the actual computing resources occupied by each task over time, and calculate the actual computing power consumption through time integration; calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data; use the difference between the actual consumption and the theoretical requirement as the scheduling error. The specific process is as follows:

[0061] Within each scheduling cycle (e.g., every 100 milliseconds), the system records the percentage of computing resources actually used by each AI perception task over time via the operating system resource monitoring interface. Integrating this curve over time yields the actual computing power consumption of the task within that cycle, using the following formula:

[0062]

[0063] in, Let be the percentage of computing resources used by the i-th task at time t. The integration interval is one scheduling cycle, and it is obtained through the resource monitoring interface of the onboard operating system at a fixed sampling frequency. This is the start time of the scheduling cycle. This is the end time of the scheduling cycle.

[0064] Meanwhile, based on the complexity of the input data for the task (such as image resolution, number of targets, and point cloud density), the system calculates the theoretical computing power required to complete the task using a pre-trained lightweight regression model (such as random forest or linear regression).

[0065] The scheduling error is obtained by subtracting the theoretical demand from the actual consumption. A positive error indicates that the actual consumption is greater than the theoretical demand (possibly due to resource contention or performance degradation), while a negative error indicates that the actual consumption is less than the theoretical demand.

[0066] Step S4: Periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and deduce the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value. The specific process is as follows:

[0067] The system continuously runs a lightweight, standardized control reference task in the background, the content of which is fixed. Every certain period of time (e.g., 30 seconds), the system allocates a standard computing power quota (e.g., 5% of the total CPU+GPU capacity) to the control task and measures the number of times it completes per unit time (e.g., 1 second), which is called the completion frequency.

[0068] In a ground-based laboratory environment, the system has pre-calibrated the completion frequency of the control task under different computing power quotas, and fitted a mapping curve. Based on the currently measured completion frequency, the ratio of the actual computing power level that the current airborne computing platform can provide to the nominal value is deduced, which is called the baseline correction value. For example, if the nominal 10% quota should generate 100 times / second, but only 85 times / second is actually measured, then the baseline correction value is 0.85.

[0069] Step S5: Collect at least one environmental data point from airborne temperature, battery voltage, and air pressure, and generate a comprehensive environmental correction coefficient based on the environmental factor impact table. The specific process is as follows:

[0070] The system collects data in real time from the airborne temperature sensor, battery voltage monitoring module, and barometric altimeter. Based on a preset environmental factor impact table, it generates a comprehensive environmental correction coefficient, calculated as: Environmental Correction Coefficient = Temperature Coefficient × Voltage Coefficient × Barometric Pressure Coefficient. When any environmental factor exceeds the safe operating range, the environmental correction coefficient is forcibly set to the minimum safe factor (e.g., 0.70), and an alarm is triggered.

[0071] Step S6: The total available computing power is calibrated and corrected using the baseline correction value and environmental correction coefficient. After deducting the minimum computing power reservation value for the flight control mission, the allocable computing power pool is obtained. The allocable computing power pool is allocated according to the dynamic priority weight ratio, and the allocation result is corrected using scheduling errors to generate the target computing power quota. The specific process is as follows:

[0072] The system first calculates the total effective available computing power: Total effective computing power = Total available computing power × Baseline correction value × Environmental correction factor;

[0073] Then, after deducting the minimum computing power reservation value for the flight control task, the allocable computing power pool is obtained: Allocable computing power pool = Total effective computing power - Flight control reserved value;

[0074] Next, based on the dynamic priority weight ratio of each AI perception task, the computing power in the allocable computing power pool is allocated to each task to obtain the initial quota: Initial Quota i = Allocable computing power pool × (weight) i (Sum of weights for all tasks)

[0075] Finally, using the scheduling error obtained in step S3, the initial quota is corrected by feedback through the proportional-integral controller: final quota i =Initial Quota i + K p ×Current Error i +K i ×Cumulative error i Among them, K p and K i These are the preset control parameters.

[0076] Step S7: Divide the allocable computing power pool into multiple computing power gradients, monitor the available computing power level in real time, and determine the number of AI perception tasks that need to be reduced when the computing power gradient decreases.

[0077] Virtualize all current AI perception tasks to be scheduled as evenly distributed task targets on a circumference, form a dynamic ring with a notch according to the current number of valid tasks, and calculate a deflection angle, where the deflection angle = 360° / total number of current AI perception tasks to be scheduled. The dynamic ring is rotated on the task targets by the deflection angle at a preset adjustment period, the tasks corresponding to the units where the dynamic ring overlaps with the task targets are active tasks and are allocated computing power, and the task corresponding to the notch is a dormant task and is suspended from execution. The specific process is as follows:

[0078] When the value of the allocatable computing power pool changes (for example, the total effective computing power decreases due to an increase in ambient temperature, or the flight control reserved value increases due to emergency obstacle avoidance), the system judges whether a preset computing power gradient threshold is crossed. In the present invention, the allocatable computing power pool is divided into 10 gradients from 100% to 0%, and each 10% decrease is defined as one gradient.

[0079] When the computing power gradient decreases (for example, dropping from the 50%~40% interval to the 40%~30% interval), the system determines the number of concurrently running AI perception tasks that need to be reduced. The number of tasks to be reduced is related to the magnitude of the gradient drop: for each gradient drop, the number of tasks to be reduced = total number of tasks × 10% (rounded up).

[0080] Then, the system executes the dynamic ring rotation scheduling algorithm, and the specific steps are as follows:

[0081] Assume that there are a total of M AI perception tasks (excluding flight control tasks) at present. The system virtualizes these M tasks into a circular task target. The circumference of the task target is evenly divided into M equal parts, each equal part corresponds to one task, and each task occupies a fixed angular position on the circumference (for example, task 1 is at 0°, task 2 is at 360° / M, task 3 is at 2×360° / M, and so on). Logically, the task target can be regarded as a fixed circular queue.

[0082] Assume that the number of tasks that need to be suspended due to the drop of the computing power gradient is k (k<M). Then the number of tasks that need to run simultaneously in the current scheduling period (the number of valid units) is N = M-k.

[0083] The dynamic ring is a circular ring with N valid units (active tasks) and a notch of length k. The radius of the dynamic ring is the same as that of the task target, the valid units on the ring are distributed at equal intervals, and the interval is consistent with the interval between adjacent tasks on the task target.

[0084] The deflection angle refers to the central angle of each rotation of the dynamic ring, and the calculation formula is: deflection angle = 360° / M (or in radian system: deflection angle = 2π / M)

[0085] Upon first entering this computing power gradient, the starting position of the gap in the dynamic ring aligns with the first task (task 1) of the task target. At this time, the N effective units on the dynamic ring overlap with tasks 1 to N on the task target, and these tasks become the active tasks of the current cycle; tasks N+1 to M on the task target corresponding to the gap become dormant tasks (suspended from execution).

[0086] The system is set to a fixed adjustment period T (e.g., every scheduling cycle, or every 5 seconds). After each adjustment period T, the dynamic ring rotates clockwise once according to the deflection angle. After rotation, the effective units of the dynamic ring establish a new overlap relationship with the tasks on the target:

[0087] Newly overlapping tasks become the activity tasks for the next cycle;

[0088] The task corresponding to the gap becomes a dormant task.

[0089] Since the deflection angle equals the angle difference between adjacent tasks, each rotation shifts the active task set one task position to the right. For example, after the first rotation, the active tasks become tasks 2 to N+1; after the second rotation, they become tasks 3 to N+2, and so on. When the rotation exceeds one full turn, the cycle returns to the starting point.

[0090] Within each adjustment cycle, the system allocates computing power only to active tasks in the current cycle. Active tasks execute according to the final quota calculated in step six; dormant tasks are not allocated computing power, their processing is suspended, but their task context is retained.

[0091] When the computing power gradient recovers (e.g., the ambient temperature decreases and the effective computing power rebounds), the system reduces the number of tasks that need to be paused, k, increases the number of effective units in the dynamic ring, N, and reinitializes the dynamic ring position, gradually restoring the concurrent execution of all tasks.

[0092] To more clearly illustrate the working principle of the mission target and dynamic loop, this embodiment uses... Figures 2-5 Detailed explanation:

[0093] Please see Figure 2 The target is a virtual circle. When M=8, eight task units (triangles numbered 1 to 8) are evenly distributed on the circumference, each occupying a fixed angular position (45° each). The target remains stationary, and each task has a fixed angular coordinate.

[0094] Please see Figure 3 Assuming a total number of tasks M=8, and k=2 tasks need to be paused due to the gradient descent of computing power, then the number of effective units N=6. The dynamic ring is a ring with a gap, on which 6 effective units (small circles) are evenly distributed, and the gap is empty.

[0095] Please see Figure 4 Align the starting edge of the gap in the dynamic ring with 0° of the target (Task 1). At this point, the effective elements of the dynamic ring cover Tasks 1 to 6, and the gaps correspond to Tasks 7 and 8.

[0096] Please see Figure 5 The deflection angle is 360° / M, where M = the total number of tasks. Each rotation causes the dynamic ring to rotate clockwise by the deflection angle.

[0097] Since the central angle between adjacent tasks on the target is exactly equal to the deflection angle, each rotation shifts the active task set by one task (i.e., discarding the first task before the current gap and adding the first task after the gap). Each task is active for 6 out of 8 consecutive cycles and dormant for 2 cycles, achieving fair allocation.

[0098] Please see Figure 5 The dynamic ring is rotated 45° clockwise (one task interval). In fact, after each rotation, the gap will move backward by one task position.

[0099] The result of the rotation and overlap is as follows: The original active task set is shifted one position to the right: New active tasks = {task 2, task 3, task 4, task 5, task 6, task 7}, and new dormant tasks = {task 8, task 1}.

[0100] Step S8: Allocate computing power quotas to the activity task based on the output of step S7 and execute it; monitor the flight safety threat level; and trigger emergency preemption and recovery. The specific process is as follows:

[0101] When the flight safety baseline management module detects an emergency obstacle avoidance or an extremely high safety threat, it triggers an emergency preemption mode. At this time, the system ignores the dynamic loop scheduling results, immediately suspends all AI perception tasks, and allocates all computing power to flight control tasks. Once the safety threat is resolved, the system resumes scheduling based on the current computing power gradient and dynamic loop position.

[0102] In some embodiments, the method of the present invention further includes:

[0103] The system records the baseline correction values ​​measured in each reference mission, forming a time series. An exponentially weighted moving average method is used for smoothing filtering. When the baseline correction value is below 0.9 for 10 consecutive times, the airborne computing platform is deemed to be aging or damaged, a maintenance alarm is generated, and the alarm is sent to the ground station via the data transmission link.

[0104] Another embodiment of the present invention provides a multimodal large model-driven UAV computing power adaptive scheduling system, comprising the following modules:

[0105] The dynamic priority weight calculation module is used to acquire data from the UAV's onboard multimodal sensors in real time and calculate dynamic priority weights for each AI perception task.

[0106] The minimum computing power reservation value calculation module is used to monitor the real-time flight status and flight stage of the UAV, and dynamically calculate the minimum computing power reservation value for the flight control task according to the preset safety strategy.

[0107] The scheduling error calculation module is used to record the change curve of the actual computing resources occupied by each task over time in each scheduling cycle, calculate the actual computing power consumption through time integration, calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data, and use the difference between the actual consumption and the theoretical requirement as the scheduling error.

[0108] The benchmark correction value acquisition module is used to periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and back-calculate the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value.

[0109] The environmental correction coefficient acquisition module is used to collect at least one type of environmental data, such as airborne temperature, battery voltage, and air pressure, and generate a comprehensive environmental correction coefficient based on the environmental factor influence table.

[0110] The target computing power quota generation module is used to calibrate and correct the total available computing power using the benchmark correction value and the environmental correction coefficient. After deducting the minimum computing power reservation value for flight control tasks, the allocable computing power pool is obtained. The allocable computing power pool is allocated according to the dynamic priority weight ratio, and the allocation result is corrected by using scheduling error to generate the target computing power quota.

[0111] Furthermore, the system also includes:

[0112] The dynamic ring rotation scheduling module is used to divide the allocable computing power pool into multiple computing power gradients, monitor the available computing power level in real time, and determine the number of AI perception tasks that need to be reduced when the computing power gradient decreases.

[0113] All AI perception tasks currently awaiting scheduling are virtualized into task targets evenly distributed on a circle. A dynamic ring with gaps is formed based on the current number of effective tasks. The deflection angle is calculated as 360° / the total number of AI perception tasks currently awaiting scheduling. The dynamic ring is rotated on the task targets by the deflection angle at a preset adjustment period. The tasks corresponding to the units where the dynamic ring overlaps with the task targets are active tasks and are allocated computing power. The tasks corresponding to the gaps are dormant tasks and are suspended from execution.

[0114] The execution and monitoring module is used to allocate computing power quotas to activity tasks and execute them based on the output of the dynamic ring rotation scheduling module, monitor flight safety threat levels, and trigger emergency preemption and recovery.

[0115] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0117] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0118] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0119] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0120] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A multimodal large model-driven UAV computing power adaptive scheduling method, characterized in that, The method includes the following steps: Step S1: Acquire real-time data from the drone's onboard multimodal sensors and calculate dynamic priority weights for each AI perception task. The specific process is as follows: Real-time acquisition of data from at least the onboard visual camera, lidar, millimeter-wave radar, and microphone array multimodal sensors of the drone; For each AI perception task, the timeliness urgency, safety impact, and scene activity are calculated. The formula for calculating the timeliness urgency is: Timeliness urgency = Maximum allowable delay / (Maximum allowable delay - Time already consumed). The safety impact is the degree of impact of the target task output on the flight safety of the drone, with a preset fixed level. The scene activity is the number of targets, the rate of change of target movement speed, and the signal-to-noise ratio in the current sensor input. The dynamic priority weight of the task is obtained by weighting and integrating the urgency of timeliness, the impact of security, and the activity of the scene. After the weights are normalized, the sum of the weights of all tasks is 1. Step S2: Monitor the real-time flight status and flight phase of the UAV, and dynamically calculate the minimum computing power reserve value for the flight control task according to the preset safety strategy; Step S3: Within each scheduling cycle, record the curve of the actual computing power resources occupied by each task changing over time, and calculate the actual computing power consumption through time integration; calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data; use the difference between the actual consumption and the theoretical requirement as the scheduling error; wherein, the formula for calculating the actual computing power consumption through time integration is as follows: in, Let be the percentage of computing resources used by the i-th task at time t. The integration interval is one scheduling cycle, and it is obtained through the resource monitoring interface of the onboard operating system at a fixed sampling frequency. This is the start time of the scheduling cycle. This is the end time of the scheduling cycle; Step S4: Periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and back-calculate the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value. Step S5: Collect at least one of the following environmental data: airborne temperature, battery voltage, and air pressure; and generate a comprehensive environmental correction coefficient based on the environmental factor impact table. Step S6: Use the benchmark correction value and environmental correction coefficient to calibrate and correct the total available computing power. After deducting the minimum computing power reservation value for the flight control mission, the allocable computing power pool is obtained. Allocate the allocable computing power pool according to the dynamic priority weight ratio, and use scheduling error to correct the allocation result to generate the target computing power quota.

2. The UAV computing power adaptive scheduling method driven by a multimodal large model according to claim 1, characterized in that, The method further includes: Step S7: Divide the allocable computing power pool into multiple computing power gradients, monitor the available computing power level in real time, and determine the number of AI perception tasks that need to be reduced when the computing power gradient decreases. All AI perception tasks currently awaiting scheduling are virtualized into task targets evenly distributed on a circle. A dynamic ring with gaps is formed based on the current number of effective tasks. The deflection angle is calculated as 360° / the total number of AI perception tasks currently awaiting scheduling. The dynamic ring is rotated on the task targets by the deflection angle at a preset adjustment period. The tasks corresponding to the units where the dynamic ring overlaps with the task targets are active tasks and are allocated computing power. The tasks corresponding to the gaps are dormant tasks and are suspended from execution. Step S8: Allocate computing power quotas to the activity task based on the output of step S7 and execute it, monitor the flight safety threat level, and trigger emergency preemption and recovery.

3. The UAV computing power adaptive scheduling method driven by multimodal large model according to claim 2, characterized in that, The method for dividing the computing power gradient is as follows: Divide the available computing power pool into equal intervals according to a percentage. Each gradient corresponds to a fixed percentage reduction in the number of tasks. The number of tasks to be reduced = the total number of AI perception tasks currently to be scheduled × the percentage of gradient descent, rounded up.

4. The UAV computing power adaptive scheduling method driven by multimodal large model according to claim 3, characterized in that, The method for constructing the task target is as follows: Create a virtual circle and divide its circumference into M arc segments, where M is the total number of AI perception tasks to be scheduled. The midpoint of each arc segment represents an AI perception task. The tasks are arranged in order of their dynamic priority weight from high to low or in order of task ID. The central angle of each arc segment is 360° / M.

5. The UAV computing power adaptive scheduling method driven by a multimodal large model according to claim 4, characterized in that, The adjustment period is a fixed time length, which can be one scheduling cycle or several scheduling cycles; after each rotation, the active task set moves backward by one task position, and when the cumulative rotation angle reaches 360°, the active task set returns to the initial position in a loop.

6. The UAV computing power adaptive scheduling method driven by multimodal large model according to claim 5, characterized in that, Also includes: As the computing power gradient increases, the number of tasks that need to be paused (k) is reduced, the number of effective units in the dynamic ring (N) is increased, and the position of the dynamic ring is reinitialized to gradually restore the concurrent execution of all tasks.

7. A multimodal large model-driven UAV computing power adaptive scheduling system, used to implement the adaptive scheduling method according to any one of claims 1-6, characterized in that, Includes the following modules: The dynamic priority weight calculation module is used to acquire data from the UAV's onboard multimodal sensors in real time and calculate dynamic priority weights for each AI perception task. The minimum computing power reservation value calculation module is used to monitor the real-time flight status and flight stage of the UAV, and dynamically calculate the minimum computing power reservation value for the flight control task according to the preset safety strategy. The scheduling error calculation module is used to record the change curve of the actual computing resources occupied by each task over time in each scheduling cycle, calculate the actual computing power consumption through time integration, calculate the theoretical computing power requirement based on the task theoretical model and the complexity of the input data, and use the difference between the actual consumption and the theoretical requirement as the scheduling error. The benchmark correction value acquisition module is used to periodically allocate standard computing power quotas to standardized reference tasks, measure their completion frequency, and back-calculate the actual available computing power level based on the pre-calibrated mapping curve to obtain the benchmark correction value. The environmental correction coefficient acquisition module is used to collect at least one type of environmental data, such as airborne temperature, battery voltage, and air pressure, and generate a comprehensive environmental correction coefficient based on the environmental factor influence table. The target computing power quota generation module is used to calibrate and correct the total available computing power using the benchmark correction value and the environmental correction coefficient. After deducting the minimum computing power reservation value for flight control tasks, the allocable computing power pool is obtained. The allocable computing power pool is allocated according to the dynamic priority weight ratio, and the allocation result is corrected by using scheduling error to generate the target computing power quota.

8. A computing device for a multimodal large model-driven UAV computing power adaptive scheduling method, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1-6.

Images