Lightweight construction method of multi-task AI prediction model in low-altitude economic aviation field

Abstract

The present application relates to the technical field of artificial intelligence model optimization, in particular to a lightweight construction method of a multi-task AI prediction model in the field of low-altitude economic aviation, comprising: obtaining the historical execution frequency proportion of a plurality of sub-tasks in the model, and calculating the complexity entropy value of each sub-task based on the frequency proportion; for the to-be-compressed parameters of the model, combining the static weight amplitude and the dynamic gradient sensitivity, calculating the importance score of each to-be-compressed parameter; combining the entropy value and the importance score, generating a dynamic compression threshold; and structurally pruning the to-be-compressed parameters with an importance score lower than the dynamic compression threshold from the model to obtain a lightweight model. The present application solves the technical problem that the real-time change of task complexity cannot be adapted due to the use of a fixed compression threshold, resulting in resource waste in simple task scenarios and precision damage in complex task scenarios.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence model optimization technology, and in particular to a lightweight construction method for multi-task AI prediction models in the field of low-altitude economic aviation. Background Technology

[0002] In multi-task AI prediction scenarios within the low-altitude economic aviation sector, such as those involving flight path planning, air traffic management, or real-time decision-making by multi-task AI prediction models, the complexity of the tasks is highly dynamic. For instance, prediction tasks may encompass short-term predictions (relatively simple patterns) that rely on unforeseen events, and long-term predictions (significantly more complex) that require capturing long-term trends. These sub-tasks with different time spans place drastically different demands on the model's capabilities.

[0003] Traditional static compression strategies (i.e., a "one-size-fits-all" approach using a fixed threshold) cannot adapt to this dynamic nature, and there is a fundamental contradiction with the task requirements:

[0004] 1. Waste of resources and unnecessary delays: If a low compression rate is set to handle the most complex tasks, the model will become too bloated when dealing with a large number of simple tasks, resulting in continuous waste of resources and unnecessary delays.

[0005] 2. Accuracy loss and security risks: If a high compression rate is set in order to achieve extreme lightweighting, the model may lose key information due to excessive compression when dealing with complex tasks, resulting in a significant drop in prediction accuracy.

[0006] To overcome the limitations of static strategies, existing technologies have proposed dynamic or adaptive compression methods, such as Chinese patent application CN117882376A. This patent discloses a technique for "adaptive image and video compression using instances in the network parameter subspace of a machine learning system".

[0007] The core approach is as follows: During model training, a low-dimensional parameter subspace is constructed using methods such as Principal Component Analysis (PCA). During inference, the system searches for an optimal set of coordinates within the subspace for each specific input data instance, generating a customized set of updated model parameters for that instance. Subsequently, the system transmits the compressed data along with this optimal set of "subspace coordinates" to the decoding end.

[0008] However, this technical solution still has the following shortcomings when applied to high-real-time, high-throughput multi-task scenarios such as low-altitude economic aviation:

[0009] 1. Overly fine adaptive granularity and high computational overhead: This method requires an optimization calculation for each independent input instance to determine its optimal coordinates in the parameter subspace. The overhead of this "case-by-case optimization" (which requires a considerable amount of computation) may offset the inference speed gain brought by compression, resulting in an increase in overall system latency instead of a decrease, making it difficult to meet the stringent requirements for low latency (e.g., less than 100ms).

[0010] 2. Lack of awareness of macro-level task complexity: The method's adaptation only addresses the micro-level characteristics of individual data instances, lacking awareness of the macro-level statistical properties of the entire task flow. It cannot proactively and forward-lookingly adjust the model's overall compression strategy at a macro level, and cannot identify when it is "currently in a simple task mode," thus resulting in wasted resources.

[0011] 3. Lagging response to changes in task distribution: Its core is based on a fixed subspace constructed from training data. When the task distribution in actual applications drifts significantly, the subspace may no longer be optimal, and the adaptive effect will decrease.

[0012] Therefore, there is an urgent need in this field for a novel technical solution that can quantify the overall complexity of the task flow in real time and accurately, and based on this, dynamically and efficiently adjust the compression intensity of the model, so as to save computing and storage resources to the maximum extent while meeting the requirements of high real-time performance and high precision. Summary of the Invention

[0013] The purpose of this invention is to overcome the aforementioned shortcomings of existing technologies and provide a lightweight construction method and corresponding system for multi-task AI prediction models in the field of low-altitude economic aviation. Specifically, this invention aims to address the technical problem that existing model compression strategies, especially static compression strategies, cannot adapt to real-time changes in task complexity due to the use of fixed compression thresholds, resulting in resource waste in simple task scenarios and loss of accuracy in complex task scenarios. Simultaneously, this invention also aims to address the technical challenge that existing instance-adaptive compression methods are unsuitable for high-throughput, high-real-time scenarios due to excessively fine adaptive granularity, high computational overhead, and lack of awareness of the macroscopic state of the task flow.

[0014] To achieve the above-mentioned objectives, the technical solution provided by this invention includes:

[0015] A lightweight construction method for multi-task AI prediction models in the field of low-altitude economic aviation includes:

[0016] Obtain the historical execution frequency percentage of multiple subtasks in the model, and calculate the complexity entropy value of each subtask based on the frequency percentage;

[0017] For the parameters to be compressed in the model, the importance score of each parameter to be compressed is calculated by combining the static weight magnitude and the dynamic gradient sensitivity.

[0018] By combining the entropy value and the importance score, a dynamic compression threshold is generated;

[0019] The parameters to be compressed that have an importance score below the dynamic compression threshold are structurally pruned from the model to obtain a lightweight model.

[0020] Preferably, the method for calculating the importance score of the parameter to be compressed includes: ;in, For parameters Importance rating; For parameters The squared L2 norm of the parameter to be compressed represents the parameter to be compressed. The static weight magnitude; For the loss function L with respect to parameters The gradient represents the parameter to be compressed. The dynamic gradient sensitivity; α and β are weighting coefficients.

[0021] Preferably, the dynamic compression threshold is generated by multiplying the entropy value, a preset compression strength coefficient, and the median of the importance scores of all parameters to be compressed.

[0022] Preferably, the preset compressibility strength coefficient is set to be negatively correlated with the entropy value.

[0023] Preferably, the multiple sub-tasks are divided according to the time span of the aircraft material demand forecast, including short-term demand forecast, medium-term demand forecast and long-term demand forecast.

[0024] Preferably, the complexity entropy value The methods include: ;in, denoted as the percentage of execution frequency for the i-th type of subtask, and n is the total number of subtask categories.

[0025] Preferably, when the model includes a long short-term memory network layer, the structured pruning includes removing an entire row or column of the weight matrix of the long short-term memory network layer.

[0026] Preferably, when the model contains a Transformer layer, the structured pruning includes deleting low-importance attention heads corresponding to the parameters to be compressed.

[0027] Beneficial effects

[0028] 1. This invention introduces information entropy to quantify the macroscopic complexity of task flows in real time and establishes a dynamic threshold generation mechanism, achieving intelligent and automatic adaptation of compression intensity to current task requirements. When processing a large number of concentrated tasks with low complexity (low entropy), it automatically switches to a "high compression ratio" mode to maximize the saving of computing resources and reduce energy consumption, effectively supporting the requirements of high efficiency and low power consumption in the low-altitude economic aviation field.

[0029] 2. When handling high-complexity (high entropy) critical tasks, the system automatically switches to a "low compression rate" mode to ensure that critical parameters are preserved. This dynamic protection mechanism based on task complexity effectively avoids the loss of critical information caused by over-compression in static pruning methods. Furthermore, its characteristic of prioritizing the removal of low-importance noise parameters while retaining critical signal parameters also serves as an effective model regularization, significantly reducing the risk of model overfitting. This has significant technical value for the low-altitude economic aviation field, where high reliability is required. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation, provided in a preferred embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0032] Example 1

[0033] This embodiment details the basic implementation of a lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation. This method aims to dynamically adjust the model's compression ratio based on the complexity of the real-time task flow to achieve an optimal balance between inference performance and resource consumption on edge devices. The specific application scenario involves the lightweight deployment of a multi-task AI prediction model in the low-altitude economic aviation field. A typical application is predicting the demand for aviation materials in a drone operation warehouse. The deep learning model used is a hybrid architecture model combining Long Short-Term Memory (LSTM) networks and Transformers, which is deployed on an edge computing device.

[0034] like Figure 1 As shown, it illustrates the overall flow of the method in this embodiment. Each step will be described in great detail below.

[0035] Step S201: Obtain the historical execution frequency ratio of multiple subtasks in the model, and calculate the complexity entropy value of each subtask based on the frequency ratio.

[0036] This step is the starting point of the dynamic adaptive mechanism of this invention, and its core objective is to provide an accurate and real-time quantitative indicator of the "overall complexity" of the series of tasks processed by the current system. In this embodiment, we use "information entropy" from information theory as this indicator.

[0037] Specifically, the detailed process for this step is as follows:

[0038] S301: Subtask definition and classification.

[0039] First, the continuous multi-task AI prediction task flow in the low-altitude economic aviation field needs to be logically divided. In this embodiment, taking aircraft material demand prediction as an example, the division is based on the time span covered by the prediction tasks. All prediction tasks T are divided into n=3 categories of sub-tasks:

[0040] Subtask Category 1 (T_short): Short-term demand forecasting. Forecasting aircraft material demand for the next 1-3 days. This type of task is usually closely related to sudden, unplanned maintenance events (AOG - Aircraft on Ground) or short-term flight schedule fluctuations. The data patterns are relatively straightforward, but extremely fast response times are required.

[0041] Subtask Category 2 (T_mid): Mid-term Demand Forecasting. This task forecasts aircraft material demand for the next 4-7 days or up to the next scheduled maintenance cycle. These tasks are typically related to routine line maintenance and replacement of consumable parts, requiring the model to consider factors such as flight frequency and aircraft type composition; the complexity is moderate.

[0042] Subtask Category 3 (T_long): Long-term demand forecasting. This involves forecasting aircraft material demand for more than 7 days in the future, such as a month, a quarter, or even a year. This type of task is highly correlated with macroeconomic factors such as long-term fleet maintenance plans, supplier delivery cycles, aircraft material inventory strategies, and seasonal factors (such as increased demand for de-icing fluid in winter). The data patterns are complex, and the model's ability to capture long-term dependencies is required to the highest degree.

[0043] S302: Real-time statistics of execution frequency.

[0044] To capture the dynamic changes in task flow complexity, it is necessary to count the frequency of occurrence of various subtasks in real time. This embodiment employs a first-in, first-out (FIFO) sliding window mechanism. The task analysis module 110 internally maintains a task history queue of a fixed size (e.g., window size N=5000). Whenever a new predicted task is classified, its category label is pushed to the tail of the queue; if the queue is full, the oldest task label at the head of the queue is removed. At any given time, the system can traverse this queue to accurately count the number of occurrences of each of the three types of subtasks in the most recent N tasks, denoted as count(T_short), count(T_mid), and count(T_long). Subsequently, the execution frequency percentage of each type of subtask can be immediately calculated.

[0045] The size N of this sliding window is an important hyperparameter. A smaller N value makes the system more sensitive to changes in task complexity, but may be affected by short-term noise; a larger N value results in smoother and more stable entropy calculations, but there will be some delay in response to changes. In the scenario of aircraft material forecasting, considering that flight schedules are usually adjusted on a daily or weekly basis, N=5000 (assuming hundreds of forecasting tasks are processed daily, the window covers several days to several weeks) is an empirical value that strikes a good balance between sensitivity and stability.

[0046] S303: Calculation of complexity entropy.

[0047] After obtaining the real-time frequency proportions of various subtasks, the complexity entropy value H(T) of the current task flow is calculated based on the Shannon entropy formula derived from information theory:

[0048] ;in, Let H(T) be the execution frequency percentage of the i-th subtask, and n be the total number of subtask categories. The physical meaning of this entropy value H(T) is to measure the uncertainty or "chaos" of the task flow. In this invention, we directly map it to the "complexity" of the task.

[0049] Lowest complexity scenario (H(T)->0): Suppose that at a certain period, the airport only handles urgent short-term prediction tasks, i.e., p_short approaches 1, while p_mid and p_long approach 0. In this case, H(T) approaches 0. This indicates that the task flow is highly predictable and singular, and the system can boldly adopt an extremely high compression ratio because the model only needs to focus on handling this single mode of short-term predictions.

[0050] The highest complexity scenario (H(T) = log2(3) ≈ 1.58): Assume that the three types of tasks are mixed evenly, i.e., p_short = p_mid = p_long = 1 / 3. At this point, the entropy reaches its maximum. This indicates that the task flow is most unpredictable; the system may receive any type of task at any time, and the model needs to be able to handle short-term, medium-term, and long-term complexities simultaneously. In this case, the system must adopt the most conservative compression strategy to preserve the model's full performance.

[0051] In general, during actual operation, the value of H(T) will fluctuate dynamically between 0 and 1.58. For example, in one embodiment, statistics show that short-term tasks account for 70%, medium-term tasks for 20%, and long-term tasks for 10%. Therefore, H(T) is calculated as -[0.7*log2(0.7) + 0.2*log2(0.2) + 0.1*log2(0.1)] ≈ 1.36. This value becomes the quantitative basis for dynamic decision-making in downstream steps.

[0052] Entropy calculations can be performed periodically (e.g., every 100 new tasks processed or every 5 minutes), or continuously updated in the background by a separate monitoring process.

[0053] Step S202: For the parameters to be compressed in the model, calculate the importance score of each parameter to be compressed by combining the static weight magnitude and the dynamic gradient sensitivity.

[0054] The goal of this step is to assign a numerical score S(w) to each parameter (weight w) in the model that reflects its current importance. Unlike existing technologies that rely solely on a single dimension of weight magnitude for evaluation, this invention employs a more comprehensive and accurate multi-dimensional evaluation method, specifically including:

[0055] S401: Static Importance Assessment.

[0056] The static importance of a parameter refers to its inherent importance, independent of specific input data, and is usually positively correlated with its numerical value. A common assumption is that in a well-trained model, weights with larger absolute values ​​carry more information and play a more dominant role in the model's decision-making.

[0057] This embodiment uses the squared L2 norm of the parameter ||w|| 2 As a measure of its static importance, a weight tensor w in the model is calculated by summing the squares of all elements in the tensor.

[0058] S402: Dynamic sensitivity assessment.

[0059] The dynamic sensitivity of a parameter refers to how sensitive the model's performance is to small changes in that parameter; it reflects the parameter's importance to the task at hand. A parameter's static value may be small, but it might happen to be located at a critical point on a decision boundary, where even a small change can lead to a large difference in the prediction results.

[0060] In this embodiment, the L2 norm ||∇wL|| of the gradient ∇wL of the loss function L with respect to the parameter w is used as a measure of its dynamic sensitivity.

[0061] The gradient can be calculated by someone skilled in the art using existing technology by selecting a small, representative dataset (called a "calibration dataset") and performing one forward and backward propagation of the model. The specific process is as follows:

[0062] 1. From the most recent task flow, categorize by the current subtask frequency as a percentage. Sample a small batch (e.g., 128) of representative data samples. For example, if p_short=0.7, p_mid=0.2, p_long=0.1, then the 128 samples should roughly include 90 short-term task samples, 25 medium-term task samples, and 13 long-term task samples.

[0063] 2. Input this batch of data into the current model, perform a forward propagation, and calculate the prediction results.

[0064] 3. Compare the predicted results with the true labels and calculate the value of the loss function L (e.g., mean squared error MSE).

[0065] 4. Perform one backpropagation and calculate the gradient ∇wL of the loss function L with respect to each parameter w in the model.

[0066] 5. Calculate the L2 norm of the gradient tensor of each parameter as its dynamic sensitivity score.

[0067] The computational cost of this process is manageable because it requires only a small batch of data and one backpropagation, which is far lower than the cost of retraining the model.

[0068] S403: Multi-dimensional scoring fusion and normalization.

[0069] After obtaining static importance ||w|| 2 After considering the dynamic sensitivity ||∇wL||, the parameter evaluation module 120 needs to fuse the information from these two different dimensions into a final importance score S(w). Direct addition is not feasible because the numerical scales of the two may differ significantly.

[0070] Preferably, the method for calculating the importance score of the parameter to be compressed includes: ;in, For parameters Importance rating; For parameters The squared L2 norm of the parameter to be compressed represents the parameter to be compressed. The static weight magnitude; For the loss function L with respect to parameters The gradient represents the parameter to be compressed. Dynamic gradient sensitivity.

[0071] In the above formula, normalization of the dynamic sensitivity term eliminates the scaling effect of gradient values, allowing the static and dynamic terms to be fused on a comparable scale. Furthermore, weighting coefficients α and β are used to balance the importance of static and dynamic information. α and β are hyperparameters, and their sum is not necessarily 1. Their values ​​determine which aspect the scoring function emphasizes. For example, if α > β, the scoring emphasizes the intrinsic magnitude of the parameter; if β > α, the scoring emphasizes the immediate impact of the parameter on the current task. According to the technical disclosure, the typical range of α and β is between [0.3, 0.8], which can be optimized by performing a grid search on the validation set.

[0072] Through the above steps, each parameter w to be compressed in the model is assigned a comprehensive and accurate importance score S(w), which includes both its "historical achievements" (static weights) and its "current performance" (dynamic gradients).

[0073] Step S203: Combine the entropy value with the importance score to generate a dynamic compression threshold.

[0074] This is the core decision-making step of the method of this invention. In some preferred embodiments, a specific method for obtaining the dynamic compression threshold is given, which is generated by multiplying the entropy value, a preset compression strength coefficient, and the median of the importance scores of all parameters to be compressed. The formula is expressed as follows:

[0075] θ = γ * H(T) * median({S(w)}), the three components of this formula each play a key role:

[0076] 1. Base Scale `median({S(w)})`: First, calculate the median of the importance scores `S(w)` for all parameters to be compressed. Using the median instead of the mean improves the robustness of threshold generation. The distribution of parameter importance in deep learning models is typically long-tailed, meaning a few parameters are extremely important (very high `S(w)` values), while a large number of parameters are less important. If the mean is used, the few extremely high `S(w)` values ​​will excessively inflate the average level, leading to an overly high threshold and potentially retaining too many unimportant parameters incorrectly. The median, unaffected by these extreme values, more stably reflects the "central tendency" of parameter importance, providing a more reasonable base scale for the threshold.

[0077] 2. Dynamic adjustment factor H(T): The complexity entropy value H(T) is the key to achieving dynamic adaptation. It is directly multiplied by the base scale to scale the threshold.

[0078] As the task becomes more complex, H(T) increases (e.g., from 1.0 to 1.5), and the product H(T)*median({S(w)}) also increases, leading to a higher final threshold θ. A higher threshold means a higher pruning threshold, where only the "least important" parameters are removed, thus preserving more model capacity to handle complex tasks.

[0079] As the task flow becomes simpler, H(T) decreases (e.g., from 1.0 to 0.5), the product decreases, and the threshold θ also decreases. A lower threshold means a lower barrier to pruning, and more "less important" parameters are removed, resulting in a higher compression ratio and saving resources.

[0080] 3. Global Strength Controller γ: The compressive strength coefficient γ is a global adjustment knob that controls the overall compressive strength. In some preferred embodiments, to further enhance the model's protection mechanism, γ is set to have a negative correlation with the entropy value H(T). For example, a simple inverse relationship can be set: γ=C / (1+k*H(T)), where C and k are constants.

[0081] When H(T) is very high (complex tasks), not only will H(T) itself raise the threshold, but the negative correlation will also make γ very small, thus "secondary suppression" of the threshold, ensuring that the compression process is extremely conservative.

[0082] When H(T) is low (for simple tasks), γ becomes larger, which further “stimulates” the threshold to decrease, resulting in more aggressive compression.

[0083] Step S204: Perform structured pruning on the parameters to be compressed that have an importance score lower than the dynamic compression threshold to obtain a lightweight model.

[0084] It should be understood that, in order to ensure a realistic hardware acceleration effect, this invention employs structured pruning. The pruning method varies depending on the model architecture:

[0085] For LSTM layers: For the weight matrix of an LSTM layer (e.g., the weight matrix of the input gate, forget gate, and output gate), if the average importance score of all weight parameters corresponding to a certain column (or row) in the matrix is ​​lower than a threshold θ, then the engine will remove the entire column (or row) from the weight matrix. This is equivalent to reducing the hidden state dimension of the LSTM unit, directly reducing the computational cost of subsequent matrix multiplications.

[0086] For the Transformer layer: The core of the Transformer model is the multi-head self-attention mechanism. Each "head" contains an independent set of query (Q), key (K), and value (V) projection matrices. If the average score of a head is below a threshold θ, then the entire attention head is considered pruneable. The pruning operation directly removes all computational paths and parameters associated with that head, resulting in an attention module with fewer heads. This directly reduces the computational cost of attention scores and the subsequent feature dimensions.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight construction method for multi-task AI prediction models in the field of low-altitude economic aviation, characterized in that, include: Obtain the historical execution frequency percentage of multiple subtasks in the model, and calculate the complexity entropy value of each subtask based on the frequency percentage; For the parameters to be compressed in the model, the importance score of each parameter to be compressed is calculated by combining the static weight magnitude and the dynamic gradient sensitivity. By combining the entropy value and the importance score, a dynamic compression threshold is generated; The parameters to be compressed that have an importance score below the dynamic compression threshold are structurally pruned from the model to obtain a lightweight model; The method for calculating the importance score of the parameter to be compressed includes: ;in, For parameters Importance rating; For parameters The squared L2 norm of the parameter to be compressed represents the parameter to be compressed. The static weight magnitude; For the loss function L with respect to parameters The gradient characterizes the parameter to be compressed. The dynamic gradient sensitivity; α and β are weighting coefficients; The dynamic compression threshold is generated by multiplying the entropy value, a preset compression strength coefficient, and the median of the importance scores of all parameters to be compressed.

2. The lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation as described in claim 1, characterized in that, The preset compressibility coefficient is set to be negatively correlated with the entropy value.

3. The lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation as described in claim 1, characterized in that: The multiple sub-tasks are divided according to the time span of the aircraft material demand forecast, including short-term demand forecast, medium-term demand forecast, and long-term demand forecast.

4. The lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation as described in claim 1, characterized in that: The complexity entropy value The methods include: ;in, denoted as the percentage of execution frequency for the i-th type of subtask, and n is the total number of subtask categories.

5. The lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation as described in claim 1, characterized in that: When the model contains a long short-term memory network layer, the structured pruning includes removing an entire row or column of the weight matrix of the long short-term memory network layer.

6. The lightweight construction method for a multi-task AI prediction model in the field of low-altitude economic aviation as described in claim 1, characterized in that: When the model contains a Transformer layer, the structured pruning includes removing low-importance attention heads corresponding to the parameters to be compressed.

Images