Lightweight Neural Network Model Optimization Method for UAV Airborne Platforms

Abstract

This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more particularly to a lightweight neural network model optimization method for UAV airborne platforms. The method includes: performing structured pruning on the acquired original neural network model to obtain a lightweight model; during the structured pruning process, dual supervision is applied using task loss and knowledge distillation loss; and based on feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized; a dedicated runtime engine is constructed based on the lightweight model; the dedicated runtime engine performs graph optimization and static memory planning on the lightweight model; the lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model to achieve an optimal balance between accuracy and speed. This invention provides a complete solution from algorithm compression, hardware co-optimization, dedicated runtime deployment to dynamic inference scheduling.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a lightweight neural network model optimization method for UAV airborne platforms. Background Technology

[0002] With the widespread application of drones in fields such as security inspection, precision agriculture, facility surveying, and logistics distribution, endowing them with onboard real-time intelligent perception capabilities (such as target detection, tracking, and semantic segmentation) has become crucial for improving operational autonomy and efficiency. This typically relies on deploying deep neural network models on the embedded computing units (such as NVIDIA Jetson series and ARM Cortex-A series processors) carried by the drone.

[0003] However, drone platforms have inherent and stringent limitations, such as limited computing power, memory bandwidth bottlenecks, power consumption and heat dissipation constraints. Therefore, directly deploying large models designed for servers to airborne platforms will result in high inference latency, high power consumption and poor stability.

[0004] Therefore, there is an urgent need for a deep collaborative optimization method that integrates algorithm compression, hardware characteristics, and runtime scheduling to achieve the best balance between accuracy, speed, power consumption, and resource consumption. Summary of the Invention

[0005] In view of the above problems, this disclosure provides a lightweight neural network model optimization method for UAV airborne platforms to overcome or at least partially solve the above problems. The purpose is to overcome problems such as the disconnect between compression and deployment, insufficient hardware utilization, and rigid allocation of computing resources through a tightly coupled technology chain, and finally achieve high-precision, low-latency, and high-energy-efficiency neural network inference on UAV airborne embedded platforms.

[0006] The objective of this invention can be achieved through the following technical solutions: This invention provides a lightweight neural network model optimization method for unmanned aerial vehicle (UAV) airborne platforms, comprising: The original neural network model is pruned in a structured manner to obtain a lightweight model; During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized. A dedicated runtime engine is built based on the lightweight model; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model. The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

[0007] Furthermore, during the structured pruning process, dual supervision is performed using both task loss and knowledge distillation loss, including: The dual supervision includes task loss based on real labels and knowledge distillation loss based on teacher model output; The original neural network model was used as the teacher model; In the fine-tuning phase of each pruning iteration, the total loss function of the lightweight model is: ; in, For task loss based on real labels, To compensate for the knowledge distillation loss from the teacher model output, This is a balancing coefficient used to adjust the weights of the two types of losses.

[0008] Furthermore, the joint search and optimization of the pruning strategy based on feedback information from the target hardware platform includes: Pre-build a performance cost model for the target hardware platform This is used to predict inference latency or power consumption under different layer configurations. The lightweight model structure parameters are represented by a vector m, which includes the number of channels per layer and the kernel size. t is the prediction delay, and p is the prediction power consumption; During the joint search using the pruning strategy, the optimization objective is to minimize the total cost.

[0009] Furthermore, the dedicated runtime engine includes: The graph optimization layer performs operator fusion, constant folding, and redundant subgraph elimination for lightweight models. The memory management layer performs lifecycle analysis based on a lightweight model data flow graph, and is used for cross-layer static memory allocation and reuse. Operator acceleration layers are used to provide convolution kernels for sparse weight matrices and optimized kernels for the target platform instruction set.

[0010] Furthermore, the operator acceleration layer for accelerating sparse convolution specifically includes: The convolution kernel weights are stored in a compressed sparse row format. The actual computational cost of sparse convolution is expressed as follows: ; in, Number of output channels Input the number of channels. convolution kernel The number of non-zero weighted numbers, and These represent the height and width of the convolution kernel, respectively. and These represent the height and width of the output feature map, respectively.

[0011] Furthermore, the adaptive inference scheduling mechanism includes a lightweight input complexity evaluation module; the lightweight input complexity evaluation module outputs a scalar complexity score s, calculated using the following formula: ;in, Where I is the current input image; is the feature extraction function; W and b are the trainable parameter vector and bias term; sigmoid is the activation function.

[0012] Furthermore, the adaptive inference scheduling mechanism dynamically selects exit points at different computational depths in the lightweight model, including: The lightweight model has multiple exits, which are multiple classification or regression branches introduced from the middle layers of the backbone network during the training phase; the total loss during training is the weighted sum of the losses of all exits. ;in, M represents the total quantity of exports; Let be the category probability distribution vector of the output of the i-th exit; y is the true label corresponding to the input data I; Let be the cross-entropy loss function for the i-th exit; Let be the weight of the export loss for the i-th export.

[0013] A second aspect of the present invention provides a lightweight neural network model optimization system for unmanned aerial vehicle (UAV) airborne platforms, comprising: Lightweight Model Module: Performs structured pruning on the acquired original neural network model to obtain a lightweight model; Pruning strategy module: During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized. Dedicated runtime engine module: Based on the lightweight model, a dedicated runtime engine is built; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model; Adaptive inference scheduling mechanism module: The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

[0014] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the lightweight neural network model optimization method for UAV airborne platforms as described in the first aspect.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lightweight neural network model optimization method for UAV airborne platforms as described in the first aspect.

[0016] The technical solution proposed in this application can bring the following beneficial effects: 1. This invention ensures that the reduction in theoretical computational cost of the model can be directly and effectively converted into a reduction in end-to-end inference latency through hardware feedback pruning and a dedicated runtime engine, thus avoiding deployment bottlenecks. The dual supervision mechanism effectively alleviates the performance drop caused by structural pruning, enabling the lightweight model to achieve significant compression with minimal loss of accuracy.

[0017] 2. The adaptive scheduling mechanism of this invention enables on-demand allocation of computing resources, which significantly reduces the average energy consumption of the system in the overall task flow and extends the effective operation time of the UAV. This invention provides a complete toolchain from model compression and engine generation to scheduling integration, which reduces the technical threshold and integration cost of deploying high-performance AI models on UAV platforms.

[0018] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the steps of a lightweight neural network model optimization method for unmanned aerial vehicle (UAV) airborne platforms, as provided in the embodiments of this specification. Figure 2 This is a schematic diagram of the structure of a lightweight neural network model optimization system for an unmanned aerial vehicle (UAV) airborne platform, as provided in the embodiments of this specification. Detailed Implementation

[0020] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. The technical solutions provided by various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram illustrating the structure of a lightweight neural network model optimization method for unmanned aerial vehicle (UAV) airborne platforms, provided for one or more embodiments of this specification.

[0022] like Figure 1 As shown, this application provides a lightweight neural network model optimization method for unmanned aerial vehicle (UAV) airborne platforms. This method includes: S101. Perform structured pruning on the obtained original neural network model to obtain a lightweight model.

[0023] First, select the baseline model and hardware platform, including: Teacher Model: A YOLOv5s model pre-trained on large datasets such as COCO was selected as the teacher model. With approximately 7.0M parameters, it achieves a good balance between detection accuracy and scale, providing rich supervisory signals for knowledge distillation.

[0024] Student Model Backbone: A more streamlined YOLOv5-tiny model is constructed as the initial backbone of the student model to be compressed; the number of channels in its backbone network and feature pyramid network (FPN) is significantly reduced compared to YOLOv5s, forming the "original neural network model" to be optimized.

[0025] Target hardware platform: The target deployment platform is the NVIDIA Jetson AGX Orin, a commonly used onboard computing unit for drones; it features a GPU, CPU, and DL accelerator. Dataset: The vehicle detection dataset, which integrates UA-DETRAC (vehicle view) and VisDrone (drone view), was used for training and validation to ensure that the model is adapted to the perspective and scene characteristics of drone ground observation.

[0026] S102. During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized.

[0027] Dual supervision includes task loss based on real labels and knowledge distillation loss based on teacher model output, including: First, knowledge distillation pre-training: using the original neural network model as the teacher model; first, on the training dataset, YOLOv5s (teacher model) is used to guide YOLOv5-tiny (student model) to perform a round of knowledge distillation pre-training; no pruning is performed at this stage, the purpose of which is to allow the student model to learn the output distribution and feature representation of the teacher model in the initial stage, obtain a good starting point, and alleviate the optimization difficulty of subsequent pruning.

[0028] Second, progressive structured pruning and fine-tuning: A progressive strategy is adopted to remove unimportant channels from the convolutional layers of the student model round by round; in the fine-tuning stage after each round of pruning, a fusion loss function is used for dual supervision, specifically including: In the fine-tuning phase of each pruning iteration, the total loss function of the lightweight model is: ; in, This is the task loss based on the ground truth labels, i.e., the loss of the object detection task itself; for YOLOv5, this typically includes bounding box regression loss (such as CIoU Loss), object confidence loss (Binary Cross-Entropy), and classification loss (Cross-Entropy). It ensures that the model predictions are consistent with the ground truth labels y (i.e., the labeled boxes and categories).

[0029] The knowledge distillation loss is used to measure the difference between the student model output and the teacher model output. The loss function based on Kullback-Leibler Divergence makes the classification probability distribution ps of the student model approximate the distribution pt of the teacher model, expressed as: This loss transfers the generalization ability and dark knowledge of the teacher model to the student model; the loss function of KL divergence is an existing technique and will not be elaborated further.

[0030] This is a balancing coefficient used to adjust the weights of the two types of losses; α is a hyperparameter with a value range between [0, 1]. Its optimal value was determined through experiments, and in this embodiment it is set to 0.7. When α is close to 1, training relies more on the real labels. When α is close to 0, it relies more on the guidance of the teacher model. By adjusting α, the trade-off between maintaining task performance and absorbing teacher knowledge can be controlled.

[0031] Through a dual oversight mechanism, This ensures the model's basic task capabilities, while As a powerful regularization term, it provides a stable supervisory signal from the high-performance teacher model during periods of abrupt changes in network structure caused by pruning, effectively preventing a precipitous drop in accuracy and making higher compression ratios possible.

[0032] Based on feedback information from the target hardware platform, a joint search and optimization of the pruning strategy is performed, including: Pre-build a performance cost model for the target hardware platform: Before pruning begins, a lightweight performance cost model needs to be pre-built for the target platform (Jetson AGX Orin). This is used to predict inference latency or power consumption under different layer configurations. Model input m: is a vector representing the configuration of the neural network layer. For convolutional layers, it may include: number of input channels (C_in), number of output channels (C_out), kernel height (K_h), width (K_w), stride, input feature map size, etc.

[0033] Model output (t, p): is the prediction latency t (in milliseconds) and power consumption p (in watts) of this layer performing one operation on the target hardware; power consumption information is crucial for the drone's endurance.

[0034] Methodology: By writing micro-benchmark programs on Jetson Orin, the actual runtime and power consumption under different combinations of configuration parameters are systematically measured (by sampling) to form a lookup table (LUT) or to train a simple regression model (such as multinomial regression); this process is offline and does not add online pruning overhead.

[0035] During the joint search using the pruning strategy, the optimization objective is to minimize the total cost.

[0036] Joint optimization search: In each round of pruning decision, "which channel should be pruned", the traditional approach is to sort the channels based solely on the weight magnitude or the channel's impact on the loss function (sensitivity); this invention introduces hardware feedback, combining sensitivity scores with changes in hardware costs.

[0037] For the channel j to be evaluated, calculate its traditional sensitivity S_sens(j), which in this embodiment is the gradient change of the loss function after pruning; Meanwhile, using the performance cost model f_hw, we predict the configuration changes of the layer and subsequent affected layers after the channel is cut off, and estimate the latency gain ΔT(j) brought about by this change (usually a negative value, indicating a reduction in latency).

[0038] Define a new joint optimization metric: The meaning of this indicator is: prioritize pruning those channels that have little impact on accuracy (small S_sens(j)) but can bring a large reduction in latency (large |ΔT(j)|).

[0039] In each round, all candidate channels are sorted according to Score(j), and the channels with the smallest Score(j) value (i.e. the highest cost-effectiveness) are removed.

[0040] In this way, the pruning process is explicitly guided not only to pursue smaller models, but also to achieve faster and more energy-efficient operation on the target hardware, thus realizing the collaborative optimization of algorithms and hardware.

[0041] S103. Based on the lightweight model, a dedicated runtime engine is constructed; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model. The dedicated runtime engine includes: The graph optimization layer performs operator fusion, constant folding, and redundant subgraph elimination for lightweight models. The memory management layer performs lifecycle analysis based on a lightweight model data flow graph, and is used for cross-layer static memory allocation and reuse. Operator acceleration layers are used to provide convolution kernels for sparse weight matrices and optimized kernels for the target platform instruction set.

[0042] Furthermore, the operator acceleration layer for accelerating sparse convolution specifically includes: The convolution kernel weights are stored in a compressed sparse row format. The actual computational cost of sparse convolution is expressed as follows: ; in, Number of output channels Input the number of channels. convolution kernel The number of non-zero weighted numbers, and These represent the height and width of the convolution kernel, respectively. and These represent the height and width of the output feature map, respectively.

[0043] In this embodiment, it is achieved by building the DroneRT engine: specifically including First, the DroneRT engine receives the lightweight model after S102 compression and fine-tuning, performs deep optimization, and generates a serialization engine file that can be efficiently executed on Jetson Orin; its operation is divided into three layers: The first layer, the graph optimization layer, involves the engine performing static analysis on the model's computational graph and then executing the following optimizations: Operator fusion: Identify fixed sequences such as "convolution (Conv) - batch normalization (BN) - activation function (ReLU)" and fuse them into a single composite operator Y = ReLU(BN(W*X + b))); After fusion, the intermediate results W*X+b and BN(...) do not need to be written back to global memory, and subsequent calculations can be completed directly in registers or caches, which greatly reduces memory bandwidth pressure.

[0044] Constant folding: The computation results of nodes in the network whose output is constant during inference (such as shape transformation layers) are pre-calculated and replaced with constants to reduce runtime computation.

[0045] Redundant subgraph elimination: To address the potential for all-zero channels or invalid connections that may result from structured pruning, the entire data flow graph is analyzed, and computational paths that are never activated are completely removed, simplifying the network structure.

[0046] The second layer: the memory management layer, which performs refined static memory planning to address the limited memory resources of embedded platforms. Lifecycle analysis: Traverse the entire optimized computation graph and analyze the generation (producer operator) and final use (consumer operator) locations of each intermediate tensor (feature map); Static memory pool allocation: A fixed, pre-allocated memory pool is allocated to tensors based on their lifecycles. When the lifecycles of two tensors do not overlap at all, they can safely reuse the same memory pool.

[0047] The third layer: operator acceleration layer; this is key to leveraging the engine's performance advantages; it provides a customized kernel tailored to the characteristics of the pruned model. General-purpose optimized kernel: Hand-written highly optimized INT8 quantized versions of commonly used operators such as convolution and matrix multiplication for Jetson Orin's CUDA cores and Tensor Cores, making full use of hardware features.

[0048] Sparse Convolution Acceleration: For the large number of sparse weights generated by pruning, Compressed Sparse Row (CSR) format is used for storage and computation. This embodiment uses CSR format: only the values ​​of non-zero weights and their corresponding column indices are stored, supplemented by an array of row pointers (row_ptr) to record the starting position of each row's non-zero elements. This greatly saves storage space.

[0049] Sparse convolution computation: In practice, dense multiplication and addition operations are no longer performed. For each location in the output feature map, only the corresponding non-zero weights in the input channels need to be calculated; the actual computational cost (FLOPs) is as follows: ; in, Number of output channels Input the number of channels. convolution kernel The number of non-zero weighted numbers, and These represent the height and width of the convolution kernel, respectively. and These represent the height and width of the output feature map, respectively.

[0050] A convolutional kernel connecting the j-th input channel and the i-th output channel. The number of zero-weighted numbers in China and Africa is a direct manifestation of sparsity.

[0051] because much smaller The total computational load was significantly reduced; by only accessing and computing the non-zero elements in the CSR format, a qualitative improvement in computational efficiency was achieved.

[0052] S104. The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

[0053] To enable the model to adapt dynamically, the lightweight model needs to be modified and a scheduling mechanism integrated: First, a lightweight input complexity evaluation module is constructed: this is a tiny network that runs in parallel with the main detection model to evaluate the "processing difficulty" of the current input frame in real time. The lightweight input complexity evaluation module consists of a small CNN with 5 layers.

[0054] The calculation formula is: ;in, Where I is the current input image; For a miniature CNN, Φ(I) is the feature extraction function, which is the feature vector after global average pooling in its last layer. W and b are trainable parameter vectors and bias terms; Sigmoid is an activation function that maps the linear output to the (0, 1) interval, serving as a normalized complexity score s. The closer s is to 1, the more complex the scene (e.g., many targets, severe occlusion, cluttered background); the closer s is to 0, the simpler the scene (e.g., sky background, single target).

[0055] Furthermore, the multi-exit model was modified and trained, including: In the model compression stage, the backbone network of YOLOv5-tiny is structurally modified. Inserting exits: Select two feature layers in the middle of the backbone network (e.g., after downsampling by 8x and 16x), and attach a lightweight detector head (consisting of 1-2 convolutional layers) to each, forming exit 1 denoted as Exit1 (shallowest) and exit 2 denoted as Exit 2 (middle); the original final detector head is used as exit M denoted as Exit M (deepest, M=3).

[0056] Multi-task training: During training, data is forward-propagated through all exits; each exit generates a detection prediction and calculates its own detection loss. ; The total training loss is expressed as: ;in, M represents the total quantity exported, which is 3 in this embodiment; Let be the category probability distribution vector of the i-th output, for example, confidence level or category; y is the true label corresponding to the input data I. Let L be the cross-entropy loss function for the i-th exit, which has the same form as the main task loss L_task. Let be the weight of the export loss for the i-th export.

[0057] It is the weighting coefficient for the i-th export loss; it is usually set as follows: < < Exemplary It is 0.5. It is 0.8. (To be 1.0), to ensure that the performance of the final exit Exit M is dominant, while allowing the shallow exit to learn useful features.

[0058] Furthermore, dynamic decision-making and offline threshold calibration include a set of optimal decision thresholds θ = (θ1, θ2) obtained through offline calibration.

[0059] The offline calibration process includes: Data collection: Run the complete system (including a lightweight input complexity evaluation module and a multi-exit model) on the validation set and record the complexity score s_n, prediction accuracy for each exit, and inference time t_{n,i} on the hardware for each sample n.

[0060] Define the performance function: For any set of candidate thresholds (θ1, θ2), the exit selection strategy for each sample n can be determined (s_n<θ1, select Exit1; θ1≤s_n<θ2, select Exit2; otherwise, select Exit M).

[0061] Then, the overall accuracy Acc(θ1, θ2) and average inference delay T_avg(θ1, θ2) of the system under this strategy are calculated.

[0062] The optimization problem is to find the combination of thresholds that minimizes the average delay T_avg(θ1, θ2) while satisfying the minimum precision requirement Acc_min (for example, set to 97% of the precision of Exit M alone).

[0063] Search for solutions: Since Acc and T_avg are piecewise constant functions of θ1 and θ2, the optimal solution θ can be found by performing an efficient grid search or dynamic programming on the sorted sequence of complexity scores {s_n} of the validation set samples.

[0064] Online dynamic reasoning: After deployment, for each frame of input, the lightweight input complexity evaluation module quickly calculates s.

[0065] Routing is performed using the offline-calibrated optimal threshold θ: if s < θ1, then Exit1 is used for inference and terminated early; if θ1 ≤ s < θ2, then Exit 2 is used; if s ≥ θ2, then the complete Exit M is used; the system outputs the detection results of the selected exit.

[0066] In this embodiment, the complete model package, which has undergone S101 and S102 pruning optimization, S103 DroneRT engine compilation, and integrated with the S104 adaptive scheduling mechanism, is deployed to the Jetson AGX Orin platform; during UAV missions, the system will automatically run the above entire process.

[0067] Figure 2 This is a schematic diagram of the structure of a lightweight neural network model optimization system for an unmanned aerial vehicle (UAV) airborne platform, as provided in the embodiments of this specification. Figure 2 As shown, a second aspect of the technical solution of the present invention provides a lightweight neural network model optimization system 200 for unmanned aerial vehicle (UAV) airborne platforms, comprising: Lightweight Model Module 201: Performs structured pruning on the acquired original neural network model to obtain a lightweight model; Pruning strategy module 202: During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized. Dedicated runtime engine module 203: Based on the lightweight model, a dedicated runtime engine is constructed; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model; Adaptive inference scheduling mechanism module 204: The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

[0068] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the lightweight neural network model optimization method for UAV airborne platforms as described in the first aspect.

[0069] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lightweight neural network model optimization method for UAV airborne platforms as described in the first aspect.

[0070] This embodiment can divide the lightweight neural network model optimization method for UAV airborne platforms into functional modules based on the above method example. For example, each function can be assigned to a separate module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0071] When using integrated units, the lightweight neural network model optimization method for UAV-borne platforms can include a processing module and a storage module. The processing module controls and manages the actions of the lightweight neural network model optimization method for UAV-borne platforms. The storage module supports the execution of program code and data by the lightweight neural network model optimization method for UAV-borne platforms.

[0072] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as disclosed in this application. The processor may also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module may be a memory.

[0073] This embodiment also provides a computer-readable storage medium (including but not limited to disk storage, CD-ROM, optical storage, etc.) storing computer program code. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement the lightweight neural network model optimization method for UAV airborne platforms provided in the above embodiment.

[0074] This embodiment also provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the aforementioned steps to implement the lightweight neural network model optimization method for UAV airborne platforms provided in the above embodiment. The beneficial effects of the above embodiments can be found in the corresponding methods described above, and will not be repeated here.

[0075] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0076] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. In the description of this disclosure, it should be understood that if terms such as "upper," "lower," "front," "rear," "left," and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, they are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the indicated position or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure.

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0078] The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A lightweight neural network model optimization method for unmanned aerial vehicle (UAV) airborne platforms, characterized in that, include: The original neural network model is pruned in a structured manner to obtain a lightweight model; During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized. A dedicated runtime engine is built based on the lightweight model; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model. The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

2. The lightweight neural network model optimization method for UAV airborne platforms according to claim 1, characterized in that, During the structured pruning process, dual supervision is provided by combining task loss and knowledge distillation loss, including: The dual supervision includes task loss based on real labels and knowledge distillation loss based on teacher model output; The original neural network model was used as the teacher model; In the fine-tuning phase of each pruning iteration, the total loss function of the lightweight model is: ； in, For task loss based on real labels, To compensate for the knowledge distillation loss from the teacher model output, This is a balancing coefficient used to adjust the weights of the two types of losses.

3. The lightweight neural network model optimization method for UAV airborne platforms according to claim 1, characterized in that, The joint search and optimization of the pruning strategy based on feedback information from the target hardware platform includes: Pre-build a performance cost model for the target hardware platform This is used to predict inference latency or power consumption under different layer configurations. The lightweight model structure parameters are represented by a vector m, which includes the number of channels per layer and the kernel size. t is the prediction delay, and p is the prediction power consumption; During the joint search using the pruning strategy, the optimization objective is to minimize the total cost.

4. The lightweight neural network model optimization method for UAV airborne platforms according to claim 1, characterized in that, The dedicated runtime engine includes: The graph optimization layer performs operator fusion, constant folding, and redundant subgraph elimination for lightweight models. The memory management layer performs lifecycle analysis based on a lightweight model data flow graph, and is used for cross-layer static memory allocation and reuse. Operator acceleration layers are used to provide convolution kernels for sparse weight matrices and optimized kernels for the target platform instruction set.

5. The lightweight neural network model optimization method for UAV airborne platforms according to claim 4, characterized in that, The operator acceleration layer, specifically for accelerating sparse convolution, includes: The convolution kernel weights are stored in a compressed sparse row format. The actual computational cost of sparse convolution is expressed as follows: ; in, Number of output channels Input the number of channels. convolution kernel The number of non-zero weighted numbers, and These represent the height and width of the convolution kernel, respectively. and These represent the height and width of the output feature map, respectively.

6. The lightweight neural network model optimization method for UAV airborne platforms according to claim 1, characterized in that, The adaptive inference scheduling mechanism includes a lightweight input complexity evaluation module; the lightweight input complexity evaluation module outputs a scalar complexity score s, calculated using the following formula: ； in, Where I is the current input image; For feature extraction functions; W and b are the trainable parameter vector and bias term; sigmoid is the activation function.

7. The lightweight neural network model optimization method for UAV airborne platforms according to claim 6, characterized in that, The adaptive inference scheduling mechanism dynamically selects exit points at different computational depths in the lightweight model, including: The lightweight model has multiple exits, which are multiple classification or regression branches introduced from the middle layers of the backbone network during the training phase; the total loss during training is the weighted sum of the losses of all exits. ;in, M represents the total quantity of exports; Let be the category probability distribution vector of the i-th output; y is the true label corresponding to the input data I. Let be the cross-entropy loss function for the i-th exit; Let be the weight of the export loss for the i-th export.

8. A lightweight neural network model optimization system for unmanned aerial vehicle (UAV) airborne platforms, characterized in that, include: Lightweight Model Module: Performs structured pruning on the acquired original neural network model to obtain a lightweight model; Pruning strategy module: During the structured pruning process, task loss and knowledge distillation loss are used together for dual supervision; and based on the feedback information from the target hardware platform, the pruning strategy is jointly searched and optimized. Dedicated runtime engine module: Based on the lightweight model, a dedicated runtime engine is built; the dedicated runtime engine optimizes the execution graph and static memory planning of the lightweight model; Adaptive inference scheduling mechanism module: The lightweight model integrates an adaptive inference scheduling mechanism; the adaptive inference scheduling mechanism dynamically selects exits at different computational depths in the lightweight model, so as to achieve the optimal balance between accuracy and speed in the lightweight model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images