A platform heterogeneous deployment method and system of an unmanned aerial vehicle visual navigation model

Abstract

The application discloses a kind of unmanned plane visual navigation model platform heterogenous deployment method and system, belong to embedded model deployment technical field.This method will end side algorithm execution link split into perception task and decision task, according to the function of each task, determine the corresponding configuration requirement, perception task is configured as high inference frame rate and low quantization precision, decision task is configured as low inference frame rate and high quantization precision;After hardware adaptability optimization to perception model, with low quantization precision, quantization is carried out to decision model with high quantization precision, two quantization models are deployed in the processing unit end of target platform, and the processing logic of each is deployed in the central processing unit end;Two quantization models are respectively allocated independent thread, and a heterogeneous asynchronous execution architecture including a shared ring buffer is constructed, two threads are executed asynchronously and in parallel, and data interaction is carried out through the shared ring buffer.The application realizes the consideration of real-time performance and robustness on low-power platform through the cooperation of differential quantization and asynchronous parallel execution.

Description

Technical Field

[0001] This invention relates to the field of embedded model deployment technology, specifically to a platform heterogeneous deployment method and system for UAV visual navigation models. Background Technology

[0002] With the widespread application of drone technology in emergency rescue, power line inspection, and field surveying, vision-based end-to-end autonomous navigation technology has become a core research direction in the industry. Navigation solutions that integrate semantic perception and deep reinforcement learning, in particular, have demonstrated significant advantages in extreme environments where there are no prior maps or GPS denial. However, the perception and decision-making models of such navigation solutions generally suffer from large parameter counts and high computational complexity, placing high demands on the computing power and memory of the operating platform. Drones, constrained by airborne payload, power consumption, and hardware costs, typically utilize low-cost, low-computing-power heterogeneous embedded platforms. Traditional deployment methods struggle to simultaneously meet the real-time, robust, and offline operation requirements of navigation systems with limited edge computing power. Especially in extreme scenarios such as disaster relief where communication is weak or nonexistent, real-time inference cannot be achieved using cloud computing power, and the performance bottleneck of edge deployment directly limits the engineering implementation of navigation systems.

[0003] Current edge deployment technologies for UAV navigation models still suffer from the following core shortcomings: First, the quantization strategy is too simplistic, failing to balance inference efficiency and decision-making performance. Existing solutions often employ uniform precision quantization adaptation. While low-precision quantization can significantly reduce inference latency, it leads to precision loss in deep reinforcement learning decision networks, which are highly sensitive to numerical accuracy, causing fatal problems such as obstacle avoidance failure and policy degradation. Conversely, high-precision quantization results in a significant decrease in the inference efficiency of visual perception models, failing to meet the frame rate requirements of real-time navigation, making it difficult to balance real-time performance and robustness. Second, the execution process is serially blocked, resulting in extremely low utilization of heterogeneous computing power. Existing deployment solutions often employ a synchronous serial execution process of preprocessing, perception inference, decision inference, and post-processing. The next step cannot start until the previous step is completed, easily leading to process blockage and accumulated latency. Furthermore, the heterogeneous computing power of embedded platforms' CPUs and NPUs cannot be utilized in parallel, resulting in severe waste of computing resources and further exacerbating the real-time performance issue. Third, the model and hardware compatibility is insufficient, making engineering implementation difficult. Existing YOLOv5 deployment solutions mostly adopt the native network structure and built-in post-processing logic. Their activation functions and complex decoding operators suffer from poor adaptability to embedded NPUs, low inference efficiency, and uncontrollable quantization accuracy, failing to fully leverage the hardware's computing power. Furthermore, the deep coupling between model inference and post-processing logic means that parameter adjustments require model recompilation, resulting in poor engineering flexibility. Fourth, there is high cloud dependency and insufficient offline operation capability. Most existing solutions still deploy the core inference process in the cloud, with the edge only responsible for image acquisition and action execution. This reliance on real-time communication is extremely high, making them completely unsuitable for extreme scenarios without communication, resulting in severely insufficient system reliability and environmental adaptability. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a platform heterogeneous deployment method and system for UAV visual navigation models. By decoupling modules and hybrid quantization, and multi-threaded heterogeneous asynchronous pipeline design, it solves the core problems of existing deployment methods, such as difficulty in balancing real-time performance and robustness, low utilization of heterogeneous computing power, poor hardware adaptability, and insufficient offline operation capability. This enables UAV visual navigation models to achieve high real-time performance and completely stable offline operation on low-computing-power embedded platforms.

[0005] This invention is achieved through the following technical solution: The first aspect of this application provides a platform heterogeneous deployment method for a UAV visual navigation model, characterized by comprising the following steps: The edge-side algorithm execution link is split into a perception task and a decision task. The perception task is used to generate navigation semantic perception results based on the real-time visual frames of the UAV, and the decision task is used to generate UAV navigation action commands based on the navigation semantic perception results. Determine the corresponding configuration requirements based on the functions of each task; The configuration requirements for the perception task include a first inference frame rate and a first quantization precision; The configuration requirements for the decision-making task include a second inference frame rate and a second quantization accuracy. The first inference frame rate is higher than the second inference frame rate, and the first quantization precision is lower than the second quantization precision. According to the configuration requirements, the perception model corresponding to the perception task is optimized for hardware adaptability, quantized with the first quantization precision to generate a first quantization model, which is then deployed on the processing unit of the target platform; the decision model corresponding to the decision task is quantized with the second quantization precision to generate a second quantization model, which is then deployed on the processing unit of the target platform; and the processing logic of the perception task and the decision task is deployed on the central processing unit of the target platform. A first thread is assigned to the first quantization model, and a second thread is assigned to the second quantization model. A heterogeneous asynchronous execution architecture is constructed, which includes the first thread, the second thread, and a shared circular buffer. The first thread and the second thread execute asynchronously in parallel and interact with each other through the shared circular buffer.

[0006] Preferably, the hardware adaptation optimization of the perception model corresponding to the perception task includes: The activation function in the perception model that does not meet the preset condition for adaptability to the processing unit of the target platform is adjusted to an activation function that matches the computing power characteristics of the processing unit of the target platform. The redundant decoding logic of the built-in detection head in the perception model is removed, and only the original detection head branches output by the backbone network and the feature fusion network are retained.

[0007] Preferably, the first quantization precision is 8-bit integer quantization, and the second quantization precision is 16-bit half-precision floating-point quantization.

[0008] Preferably, the processing logic of the perception task and the inference of the first quantization model are executed serially within the first thread, and the perception result is written into the shared circular buffer. The second thread serially executes the processing logic of the decision task and the inference of the second quantization model, and reads the perception result from the shared circular buffer.

[0009] Preferably, the shared circular buffer adopts a lock-free read-write mechanism, assigns a unique frame identifier to each frame of data, and determines the buffer depth based on the difference between the first inference frame rate and the decision frequency. When new data is written, the oldest data is overwritten.

[0010] Preferably, the perception model is a target detection model, the decision model is a deep reinforcement learning navigation decision model, and the target platform is an embedded platform equipped with a heterogeneous central processing unit and a neural network processing unit.

[0011] Preferably, the first quantization model and the second quantization model establish independent inference instances, and are respectively bound to different neural network processing unit cores or different inference execution queues of the target platform for execution.

[0012] Preferably, it further includes: storing the first quantization model, the second quantization model, and the processing logic locally on the target platform, loading them into memory when the system starts, and achieving offline operation without cloud dependency; The offline operation includes optimizing the memory usage during system operation by loading model weights on demand and reusing feature map memory, thereby controlling the peak memory usage of the system operation within a preset range.

[0013] The first aspect of this application provides a platform heterogeneous deployment system for a UAV visual navigation model, including: The task decoupling module is used to split the edge algorithm execution link into perception tasks and decision tasks, and determine the corresponding configuration requirements according to the function of each task. The configuration requirements of the perception task include a first inference frame rate and a first quantization accuracy, and the configuration requirements of the decision task include a second inference frame rate and a second quantization accuracy. The first inference frame rate is higher than the second inference frame rate, and the first quantization accuracy is lower than the second quantization accuracy. The quantization deployment module is used to optimize the hardware adaptability of the perception model corresponding to the perception task according to the configuration requirements, quantize it with the first quantization precision to generate a first quantization model, and deploy it on the processing unit of the target platform; quantize the decision model corresponding to the decision task with the second quantization precision to generate a second quantization model, and deploy it on the processing unit of the target platform; and deploy the processing logic of the perception task and the decision task respectively on the central processing unit of the target platform. An asynchronous execution module is used to allocate a first thread to the first quantization model and a second thread to the second quantization model, thereby constructing a heterogeneous asynchronous execution architecture that includes the first thread, the second thread, and a shared circular buffer. The first thread and the second thread execute asynchronously in parallel and interact with each other through the shared circular buffer.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the platform heterogeneous deployment method of the UAV visual navigation model.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects: This application proposes a platform heterogeneous deployment method for UAV visual navigation models. It splits the edge-side algorithm execution chain into perception and decision tasks. Based on the different requirements of inference frame rate and quantization accuracy for different tasks, it performs quantization with different precision levels, constructing a heterogeneous asynchronous execution architecture to achieve efficient parallelism of the two tasks. This platform heterogeneous deployment method does not uniformly compress and deploy a single model. Instead, it sets quantization accuracy, inference frequency, and thread execution mode separately for the perception and decision tasks in the UAV visual navigation chain, based on the differences in execution frequency, accuracy sensitivity, and data dependencies, thus forming a heterogeneous collaborative deployment process oriented towards the navigation closed loop. At the task decoupling level, this method configures the perception task with a high inference frame rate and low quantization accuracy, and the decision task with a low inference frame rate and high quantization accuracy, avoiding the contradiction that a single quantization strategy cannot simultaneously achieve real-time performance and robustness from the source. At the quantization deployment level, low-precision quantization is performed on the perception model after hardware adaptation optimization. The activation function is adjusted to match the computing power characteristics of the target platform's processing unit, redundant decoding logic is removed, and the integer computation advantage of the processing unit is fully utilized to improve the inference frame rate. High-precision quantization is performed on the decision model to preserve numerical precision and avoid policy degradation. The inference of the two models is physically isolated at the processing unit level, and their respective processing logic is deployed at the central processing unit. At the execution architecture level, independent threads are allocated to the two quantization models, and lock-free asynchronous data interaction is achieved through a shared circular buffer, breaking the blocking bottleneck of the traditional serial process and enabling parallel utilization of heterogeneous computing power. This method solves the core problem of balancing real-time performance and robustness on low-computing-power platforms through the synergy of differentiated quantization and asynchronous parallel execution.

[0016] This application also proposes a platform heterogeneous deployment system for UAV visual navigation models, an electronic device, and a computer storage medium, which possess all the advantages of the aforementioned platform heterogeneous deployment method for UAV visual navigation models. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the platform heterogeneous deployment method for the UAV visual navigation model of the present invention; Figure 2 This is a schematic diagram of the architecture of the CPU-NPU heterogeneous asynchronous pipeline of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] A lightweight heterogeneous deployment method for a low-computing-power platform of a UAV visual navigation model includes the following steps: Step 1: Decouple the functional boundaries and execution links of the edge-side algorithm execution link of UAV visual navigation from image input to navigation action output, and split it into two independent execution units: perception pipeline and decision pipeline. Determine the corresponding configuration requirements based on the function of each execution unit.

[0022] The specific implementation method of step 1 will be described in detail below.

[0023] S11: The perception pipeline is responsible for visual image preprocessing, target detection inference, detection result decoding, and semantic mask generation.

[0024] The perception pipeline is an environmental semantic perception unit based on a target detection network, which is adapted to the real-time environmental perception needs of UAVs in scenarios where there is no prior map or GPS denial.

[0025] Based on the function of the perception pipeline, its configuration requirements are determined as follows: a first inference frame rate and a first quantization precision. The first inference frame rate is a high inference frame rate that meets the real-time obstacle avoidance requirements of the UAV. The first quantization precision is a low quantization precision with low sensitivity to numerical precision, which is adapted to the integer calculation optimization characteristics of the embedded neural network processing unit.

[0026] S12: The decision pipeline is responsible for constructing the temporal state, reasoning the navigation action, and outputting the action selection.

[0027] The decision pipeline is a navigation action decision unit based on deep reinforcement learning.

[0028] Based on the function of the decision pipeline, its configuration requirements are determined as follows: a second inference frame rate and a second quantization precision. The second inference frame rate is lower than the first inference frame rate, and the second quantization precision is higher than the first quantization precision. It is necessary to avoid obstacle avoidance failure and strategy degradation caused by quantization and adapt to the floating-point calculation characteristics of the embedded neural network processing unit.

[0029] In some embodiments, the object detection network may also employ other object detection models such as YOLOv8, YOLOv9, EfficientDet, or CenterNet.

[0030] In some embodiments, the deep reinforcement learning navigation action decision unit may also employ a decision model based on the soft actor-critic algorithm or the dual-delay deep deterministic policy gradient algorithm.

[0031] By decoupling the functional boundaries and execution links of the navigation model, the perception and decision-making tasks are separated into two independent pipelines. Based on this, the corresponding configuration requirements are determined according to the function of each pipeline, laying the foundation for subsequent differentiated lightweight modifications and multi-threaded asynchronous deployment. The configuration requirements of each pipeline match its functional positioning, ensuring precise adaptation of subsequent quantization strategy selection and hardware resource allocation. This avoids excessive retention of computational precision for precision-insensitive perception tasks, resulting in wasted computing power, or the use of low-precision quantization for precision-sensitive decision-making tasks, leading to strategy degradation.

[0032] Step 2: Based on the configuration requirements of the two pipelines, after performing hardware adaptation and lightweighting modifications on the target detection model of the perception pipeline, a perception quantization model is generated using the first precision quantization method and deployed on the neural network processing unit of the embedded platform; a decision quantization model is generated for the deep reinforcement learning navigation decision model of the decision pipeline using the second precision quantization method and deployed on the neural network processing unit of the embedded platform. The second precision quantization method has a higher quantization precision than the first precision quantization method; The pre- and post-processing logic of the two pipelines is deployed on the central processing unit of the embedded platform.

[0033] The specific implementation method of step 2 will be described in detail below.

[0034] S21: Perform operator-level adaptation and computational chain simplification for the target detection model. Replace all native SiLU activation functions of the target detection model, which have poor adaptability to the embedded neural network processing unit, with ReLU activation functions that match the computing power characteristics of the embedded neural network processing unit and have better quantization stability. This eliminates the operator adaptation bottleneck of heterogeneous platforms and ensures that the perception inference frame rate meets the real-time obstacle avoidance requirements of the UAV.

[0035] At the same time, redundant decoding logic in the built-in detection head of the target detection model is removed, and only the original detection head branches output by Backbone and Neck are retained, thereby decoupling the link between detection inference and post-processing decoding, and adapting to the deployment architecture of heterogeneous computing power split between central processing unit and neural network processing unit.

[0036] S22: Export the lightweight modified target detection model into the ONNX universal format, complete the model conversion, first-precision quantization calibration and compilation through the model conversion toolchain corresponding to the embedded platform, generate a special model format adapted to the neural network processing unit of the embedded platform, and deploy the quantized model on the neural network processing unit of the embedded platform.

[0037] The first precision quantization method is INT8 quantization. The first precision quantization calibration is completed using a real image dataset of the UAV target application scenario to ensure scenario-based quantization accuracy.

[0038] S23: The pre- and post-processing logic of the perception pipeline is deployed on the central processing unit (CPU). The pre- and post-processing logic of the perception pipeline includes image preprocessing, detection head anchor box decoding, non-maximum suppression, and semantic mask generation logic. The pre- and post-processing logic of the perception pipeline deployed on the CPU is executed serially within the same perception thread, bound to the perception model inference on the neural network processing unit.

[0039] S24: The core computational branches of the deep reinforcement learning navigation decision network are fully preserved. Second-precision quantization is performed on all weights and activation values ​​of the network's convolutional and fully connected layers. This maximizes numerical accuracy while compressing model computation, preventing navigation policy degradation. The second-precision quantization method is FP16 quantization.

[0040] The quantized model is compiled into a dedicated model format adapted to the neural network processing unit of the embedded platform and deployed on the neural network processing unit of the embedded platform. Further, when the target platform contains multiple independently schedulable neural network processing unit cores, the first quantized model and the second quantized model are respectively bound to different neural network processing unit cores; when the target platform does not support core-level binding, logical isolation execution is achieved through independent inference instances, independent model contexts, or independent inference queues.

[0041] S25: Deploy the pre- and post-processing logic of the decision pipeline on the central processing unit (CPU). The pre- and post-processing logic of the decision pipeline includes timing state stack construction, action selection, and flight control command output logic. The pre- and post-processing logic of the decision pipeline deployed on the CPU is executed serially within the same decision thread, bound to the decision model inference on the neural network processing unit.

[0042] In some embodiments, the first precision quantization method can also be replaced by 8-bit integer-aware quantization or 4-bit integer quantization.

[0043] In some embodiments, the second precision quantization method can also be replaced by 32-bit floating-point quantization.

[0044] In some embodiments, the pre- and post-processing logic can also be rewritten and optimized using other compiled languages ​​such as Rust or Go before deployment.

[0045] Based on the different configuration requirements of the perception pipeline and the decision pipeline, differentiated quantization strategies are adopted. The perception model generates a perception quantization model through first-precision quantization, fully utilizing the integer computational advantages of low-computing-power neural network processing units to significantly improve the inference frame rate and meet the frame rate requirements of real-time obstacle avoidance. The decision model generates a decision quantization model through second-precision quantization, compressing computation while preserving numerical precision, avoiding obstacle avoidance failure and policy degradation caused by quantization loss. The hardware adaptability lightweight modification of the perception model eliminates the heterogeneous operator adaptation bottleneck by replacing activation functions, and decouples detection inference and post-processing by removing redundant decoding logic, improving inference efficiency and engineering flexibility. The pre- and post-processing logic is deployed on the central processing unit, working collaboratively with the neural network processing unit to achieve efficient utilization of heterogeneous computing power.

[0046] Step 3: Configure independent execution threads for the deployed perception quantization model and decision quantization model respectively, so as to realize the two pipelines can be executed in parallel without dependencies; construct a heterogeneous asynchronous execution architecture including the perception thread, the decision thread and a shared circular buffer to realize the parallel execution of the perception quantization model and the decision quantization model without dependencies and the asynchronous transmission of perception results.

[0047] The specific implementation method of step 3 will be described in detail below.

[0048] S31: A dedicated perception thread is allocated to the perception quantization model, bound to a preset core of the central processing unit (CPU). Within this perception thread, the entire perception computation process is executed serially: image preprocessing is completed at the CPU end, perception quantization model inference is completed at the neural network processing unit end, and post-processing and semantic mask generation are completed at the CPU end. After a single frame calculation is completed, the perception result is written to a shared circular buffer. The perception thread has no fixed operating frequency; its execution rate is dynamically determined by the camera input frame rate and the inference computing power of the neural network processing unit.

[0049] S32: An independent decision thread is allocated to the decision quantization model, bound to another preset core of the central processing unit (CPU), and executed asynchronously and in parallel with the perception thread. The decision thread has no fixed running frequency; it only triggers the next decision process after the current navigation action is completed. Upon triggering, the entire decision calculation process is executed serially within the decision thread: the CPU reads the latest frame semantic mask result from the shared circular buffer, inserts it into the perception sequence maintained within the decision thread, constructs the latest temporal state stack, and generates a state description; the neural network processing unit completes the decision quantization model inference; and the CPU completes action selection and instruction output.

[0050] S33: Construct a shared circular buffer of fixed depth as the sole data interaction link between the perception thread and the decision-making thread. The shared circular buffer employs a lock-free read-write mechanism, assigning a unique frame identifier to each frame of data. The buffer depth matches the difference between the UAV navigation control frequency and the perception frame rate. The shared circular buffer retains only the latest multi-frame perception results; new data automatically overwrites the oldest expired data.

[0051] In some embodiments, the depth of the shared circular buffer can be dynamically adjusted based on the ratio of the actual perceived frame rate to the decision frequency.

[0052] In some embodiments, the lock-free read / write mechanism may also be replaced by a synchronization mechanism such as a semaphore or a condition variable.

[0053] By employing a dual-threaded heterogeneous asynchronous execution architecture, the perception quantization model and the decision quantization model are decoupled into two independent parallel threads, breaking through the blocking bottleneck of traditional serial processes. The perception thread independently executes the entire perception computation process, and the decision thread independently executes the entire decision computation process. They achieve asynchronous data interaction through a shared circular buffer, ensuring mutual non-blocking. The lock-free read-write mechanism of the shared circular buffer avoids the thread waiting and data copying overhead caused by traditional mutex lock schemes, further reducing communication latency. The perception thread and the decision thread are bound to different central processing unit cores, fully utilizing the parallel computing capabilities of multi-core processors. Together with the neural network processing unit, they form a heterogeneous parallel computing system, significantly improving overall computing power utilization.

[0054] Step 4: Load the quantization model generated in Step 2 and the asynchronous execution architecture built in Step 3 into the target embedded platform to complete the local storage and operation of all models and execution logic; control the memory usage of the running system by loading model weights on demand and reusing feature map memory; verify whether the running frame rate of the deployed system meets the control frequency requirements of the UAV real-time navigation, and realize the navigation model to run completely offline on a low computing power platform without cloud dependency.

[0055] The specific implementation method of step 4 will be described in detail below.

[0056] S41: The quantized neural network processing unit model, compiled executable file, and configuration parameters are all stored locally on the embedded platform and automatically loaded into memory upon system startup. All model inference and execution logic is completed locally on the embedded platform, without the need for cloud interaction.

[0057] S42: By loading model weights on demand and reusing feature map memory, the memory usage during system operation is optimized, and the peak memory usage of the system is controlled within a preset range.

[0058] S43: After the full-link deployment is completed, verify whether the stable operating frame rate of the system meets the control frequency requirements of the UAV real-time navigation, and confirm that all models and execution logic run completely locally without cloud computing power and real-time communication dependence.

[0059] In some embodiments, the embedded platform may also be other embedded platforms equipped with heterogeneous central processing units and neural network processing units, such as RK3566, RK3576, or Qualcomm Snapdragon.

[0060] By deploying the entire link model and execution logic locally, the navigation model was able to run completely offline on a low-computing-power embedded platform, completely eliminating reliance on cloud computing power and real-time communication, and adapting to extreme scenarios with weak or no communication, such as disaster relief. Through memory optimization techniques such as on-demand loading of model weights and memory reuse of feature maps, the peak system memory usage was kept within the platform's tolerance range, ensuring long-term stable operation on resource-constrained platforms. System operation verification ensured that the deployed navigation system met the control frequency requirements for real-time UAV navigation, guaranteeing navigation performance.

[0061] The low-computing-power embedded platform can be an edge computing platform equipped with at least one coprocessing unit among CPU, NPU, GPU or DSP, and its available memory, power consumption or inference computing power is lower than that of a server-side computing platform.

[0062] This invention provides a lightweight heterogeneous deployment method for UAV visual navigation models on low-computing-power platforms. Through the coordinated operation of four steps—functional decoupling, differentiated quantization, heterogeneous asynchronous execution, and offline deployment—it achieves real-time offline deployment and stable operation of UAV visual navigation systems on low-computing-power embedded platforms.

[0063] At the functional decoupling level, the edge algorithm execution chain is split into two independent tasks: perception and decision-making. Configuration requirements are determined based on the function of each task, providing a basis for subsequent differentiated processing. At the quantization adaptation level, based on the different configuration requirements of the two tasks, low-precision quantization is used for the perception model to fully leverage the integer computation advantages of the neural network processing unit, improving the inference frame rate to meet the frame rate requirements of real-time obstacle avoidance. Higher-precision quantization is used for the decision model to preserve numerical accuracy, avoiding obstacle avoidance failure and strategy degradation due to quantization loss. Simultaneously, operator-level adaptation is performed on the perception model, replacing the poorly adapted activation function of the native embedded neural network processing unit with an activation function that matches the computing power characteristics and has better quantization stability, eliminating the heterogeneous operator adaptation bottleneck. Pre- and post-processing logic is deployed on the central processing unit, achieving heterogeneous computing power collaboration between the central processing unit and the neural network processing unit. At the execution architecture level, independent threads are allocated to the two quantization models, constructing a heterogeneous asynchronous execution architecture including dual threads and a shared circular buffer. The perception thread and decision thread execute independently and in parallel, achieving lock-free asynchronous data interaction through the shared circular buffer, breaking the blocking bottleneck of the traditional serial process and improving the utilization rate of heterogeneous computing power. At the offline deployment level, the quantization model and execution architecture are loaded onto the target embedded platform to complete the local storage and operation of the entire link model and execution logic, enabling the navigation model to run completely offline on a low-computing-power platform without cloud dependency.

[0064] This method solves the core problems of traditional deployment methods, such as difficulty in balancing real-time performance and robustness, low utilization of heterogeneous computing power, poor hardware adaptability, and insufficient offline operation capability, providing reliable technical support for the engineering implementation of UAV visual navigation systems on low-computing-power embedded platforms.

[0065] Example 1 refer to Figure 1 and 2 This embodiment uses a UAV visual navigation model as the deployment target, with the RK3588 embedded development board as the platform. The platform's core configuration includes an 8-core Cortex-A76 and A55 heterogeneous CPU, a neural network processing unit with 6 trillion operations per second (INT8) computing power, and runs an embedded Linux system. The deployment targets include a YOLOv5s-based perception model and a deep reinforcement learning decision-making model incorporating spatiotemporal attention mechanisms. The specific implementation steps are as follows: Step 1: Decouple the perception model and the decision model functionally and determine their respective configuration requirements.

[0066] The perception model is responsible for visual image preprocessing, object detection inference, detection result decoding, and semantic mask generation. The input is a 640×480 pixel image, and the output is a semantic mask of the same size. It is adapted to the real-time environmental perception needs of drone-based disaster relief scenarios in environments without prior maps or where GPS is denied. Based on its functions, the configuration requirements are determined to be high inference frame rate and low latency, with low sensitivity to numerical precision, and adapted to the parallel computing characteristics of the RK3588 platform's neural network processing unit.

[0067] The decision model is responsible for constructing temporal states, reasoning about navigation actions, and outputting action selection. Its input is a semantic mask map of six consecutive time steps, and its output is the action value of discrete navigation actions. Based on its function, the configuration requirements are high numerical accuracy, high sensitivity to quantization loss, and avoidance of obstacle avoidance failure and policy degradation caused by quantization.

[0068] Step 2: Perform differentiated lightweight modification and mixed precision quantization adaptation on the two models.

[0069] First, the YOLOv5s perception model underwent customized structural optimization. To address the low-latency requirements of UAV-based navigation, all SiLU activation functions were replaced with ReLU activation functions, which are computationally efficient and offer better quantization stability on the RK3588 platform's neural network processing unit. This resolved the issues of poor operator adaptability, low inference efficiency, and uncontrollable INT8 quantization accuracy of the native SiLU activation function on the RK3588 platform's neural network processing unit. Simultaneously, redundant decoding logic in the model's built-in detection head was removed, retaining only the three original detection head branches output by the Backbone and Neck, achieving complete decoupling between the core computation of detection inference and the post-processing decoding logic. The lightweight model was exported to ONNX format, and the RKNN-Toolkit2 toolchain was used for model conversion and INT8 quantization. The quantization calibration dataset consisted of real-world images from UAV disaster relief scenarios, compiled into an RKNN model, and deployed on the platform's neural network processing unit.

[0070] Secondly, operator adaptation was performed on the deep reinforcement learning decision network to ensure that all operators were standard operators supported by RKNN-Toolkit2, fully preserving the core computational branches of the network. The model was exported to ONNX format, and FP16 half-precision quantization was performed using the RKNN-Toolkit2 toolchain to maximize the preservation of numerical accuracy while compressing the model's computational load. After being compiled into an RKNN model, it was deployed on the platform's neural network processing unit, physically isolated from the inference channel of the perception model.

[0071] Finally, the image preprocessing, detection head anchor box decoding, non-maximum suppression, and semantic mask generation logic of the perception model, as well as the temporal state stack construction, action selection, and flight control command output logic of the decision model, were all rewritten in C++. Tensor operations were accelerated using the inference library adapted to the RK3588 platform, and after compilation, they were bound to the A76 large core thread of the central processing unit for execution.

[0072] Step 3: Construct a dual-threaded heterogeneous asynchronous execution architecture.

[0073] An independent perception thread is allocated to the perception model, bound to the A76 core 0 of the central processing unit. Within the perception thread, image acquisition, preprocessing at the central processing unit, perception model inference at the neural network processing unit, post-processing at the central processing unit, and semantic mask generation are executed sequentially. The results are then written to a shared circular buffer. The perception thread has no fixed operating frequency; its execution rate is dynamically determined by the camera input frame rate and the computational power of the neural network processing unit's inference.

[0074] An independent decision-making thread is allocated to the decision-making model, bound to the CPU's A76 core 1, and executed asynchronously and in parallel with the perception thread. The decision-making thread has no fixed running frequency; it only triggers the next decision-making process after the current navigation action is completed. After triggering, the decision-making thread executes sequentially: reading the latest frame's semantic mask result from the shared circular buffer, updating the internal 6-frame temporal state space, constructing the temporal state stack, inference of the decision-making model at the neural network processing unit, and action selection and instruction output at the CPU.

[0075] A shared circular buffer with a depth of 4 is constructed to match the difference between the UAV navigation control frequency and the perception frame rate in this embodiment. A lock-free read-write mechanism is adopted to assign a unique frame identifier to each frame of data, retain only the latest 4 frames of perception results, and automatically overwrite the oldest expired data when new data is written.

[0076] Step 4: Complete the offline deployment of the full-link embedded platform.

[0077] The quantized RKNN model, the C++ compiled executable file, and configuration parameters are all stored locally on the RK3588 development board. They are automatically loaded into memory upon system startup, eliminating the need for cloud interaction and enabling completely offline operation. By optimizing model weight loading on demand and feature map memory reuse, peak system memory usage is controlled, ensuring long-term stable platform operation.

[0078] This embodiment achieves millisecond-level end-to-end latency for the UAV visual navigation system on the RK3588 low-computing-power embedded platform through the above steps. While ensuring that the performance of the navigation control strategy is not significantly degraded, it eliminates the dependence on cloud computing power and real-time communication, and adapts to the autonomous navigation operation requirements of UAVs in extreme weak communication scenarios such as disaster relief.

[0079] Example 2 This application also discloses a platform heterogeneous deployment system for a UAV visual navigation model, including: The task decoupling module is used to split the edge algorithm execution link into perception tasks and decision tasks, and determine the corresponding configuration requirements according to the function of each task. The configuration requirements of the perception task include a first inference frame rate and a first quantization accuracy, and the configuration requirements of the decision task include a second inference frame rate and a second quantization accuracy. The first inference frame rate is higher than the second inference frame rate, and the first quantization accuracy is lower than the second quantization accuracy. The quantization deployment module is used to optimize the hardware adaptability of the perception model corresponding to the perception task according to the configuration requirements, quantize it with the first quantization precision to generate a first quantization model, and deploy it on the processing unit of the target platform; quantize the decision model corresponding to the decision task with the second quantization precision to generate a second quantization model, and deploy it on the processing unit of the target platform; and deploy the processing logic of the perception task and the decision task respectively on the central processing unit of the target platform. An asynchronous execution module is used to allocate a first thread to the first quantization model and a second thread to the second quantization model, thereby constructing a heterogeneous asynchronous execution architecture that includes the first thread, the second thread, and a shared circular buffer. The first thread and the second thread execute asynchronously in parallel and interact with each other through the shared circular buffer.

[0080] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0081] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0082] An electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the platform heterogeneous deployment method of the UAV visual navigation model as described in any of the above embodiments.

[0083] Another electronic device provided in this application embodiment may further include: an input port connected to a processor for transmitting multimodal data collected by an external acquisition device to the processor; a display unit connected to the processor for displaying the processor's processing results to the outside world; and a communication module connected to the processor for enabling communication between the electronic device and the outside world. The display unit may be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module includes, but is not limited to, Mobile High Definition Link (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), and wireless connection (including Wi-Fi, Bluetooth, Bluetooth Low Energy, and IEEE 802.11s-based communication technology).

[0084] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the platform heterogeneous deployment method for the UAV visual navigation model as described in any of the above embodiments.

[0085] For descriptions of relevant parts of the heterogeneous deployment system, electronic devices, and computer-readable storage media of the UAV visual navigation model provided in this application, please refer to the detailed descriptions of the corresponding parts in the heterogeneous deployment method of the UAV visual navigation model provided in this application, which will not be repeated here. Furthermore, parts of the technical solutions provided in this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0086] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A platform heterogeneous deployment method for a UAV visual navigation model, characterized in that, Includes the following steps: The edge-side algorithm execution link is split into a perception task and a decision task. The perception task is used to generate navigation semantic perception results based on the real-time visual frames of the UAV, and the decision task is used to generate UAV navigation action commands based on the navigation semantic perception results. Determine the corresponding configuration requirements based on the functions of each task; The configuration requirements for the perception task include a first inference frame rate and a first quantization precision; The configuration requirements for the decision-making task include a second inference frame rate and a second quantization accuracy. The first inference frame rate is higher than the second inference frame rate, and the first quantization precision is lower than the second quantization precision. Based on the configuration requirements, the perception model corresponding to the perception task is optimized for hardware adaptability, then quantized with the first quantization precision to generate a first quantization model, which is then deployed on the processing unit of the target platform. The decision model corresponding to the decision task is quantized with the second quantization precision to generate a second quantization model, which is then deployed on the processing unit of the target platform; the processing logic of the perception task and the decision task are respectively deployed on the central processing unit of the target platform. A first thread is assigned to the first quantization model, and a second thread is assigned to the second quantization model. A heterogeneous asynchronous execution architecture is constructed, which includes the first thread, the second thread, and a shared circular buffer. The first thread and the second thread execute asynchronously in parallel and interact with each other through the shared circular buffer.

2. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, The hardware adaptation optimization of the perception model corresponding to the perception task includes: The activation function in the perception model that does not meet the preset condition for adaptability to the processing unit of the target platform is adjusted to an activation function that matches the computing power characteristics of the processing unit of the target platform. The redundant decoding logic of the built-in detection head in the perception model is removed, and only the original detection head branches output by the backbone network and the feature fusion network are retained.

3. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, The first quantization precision is 8-bit integer quantization, and the second quantization precision is 16-bit half-precision floating-point quantization.

4. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, Within the first thread, the processing logic of the perception task and the inference of the first quantization model are executed serially, and the perception results are written into the shared circular buffer. The second thread serially executes the processing logic of the decision task and the inference of the second quantization model, and reads the perception result from the shared circular buffer.

5. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, The shared circular buffer adopts a lock-free read-write mechanism, assigns a unique frame identifier to each frame of data, and determines the buffer depth based on the difference between the first inference frame rate and the decision frequency. When new data is written, the oldest data is overwritten.

6. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, The perception model is a target detection model, the decision model is a deep reinforcement learning navigation decision model, and the target platform is an embedded platform equipped with a heterogeneous central processing unit and a neural network processing unit.

7. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, The first quantization model and the second quantization model each establish independent inference instances and are respectively bound to different neural network processing unit cores or different inference execution queues of the target platform for execution.

8. The platform heterogeneous deployment method for the UAV visual navigation model according to claim 1, characterized in that, Also includes: The first quantization model, the second quantization model, and the processing logic are stored locally on the target platform and loaded into memory when the system starts, enabling offline operation without cloud dependency. The offline operation includes optimizing the memory usage during system operation by loading model weights on demand and reusing feature map memory, thereby controlling the peak memory usage of the system operation within a preset range.

9. A system for implementing a platform heterogeneous deployment method of the UAV visual navigation model according to any one of claims 1-8, characterized in that, include: The task decoupling module is used to split the edge algorithm execution link into perception tasks and decision tasks, and determine the corresponding configuration requirements according to the function of each task. The configuration requirements of the perception task include a first inference frame rate and a first quantization accuracy, and the configuration requirements of the decision task include a second inference frame rate and a second quantization accuracy. The first inference frame rate is higher than the second inference frame rate, and the first quantization accuracy is lower than the second quantization accuracy. The quantization deployment module is used to optimize the hardware adaptability of the perception model corresponding to the perception task according to the configuration requirements, quantize it with the first quantization accuracy, generate the first quantization model, and deploy it on the processing unit of the target platform. The decision model corresponding to the decision task is quantized with the second quantization precision to generate a second quantization model, which is then deployed on the processing unit of the target platform; the processing logic of the perception task and the decision task are respectively deployed on the central processing unit of the target platform. An asynchronous execution module is used to allocate a first thread to the first quantization model and a second thread to the second quantization model, thereby constructing a heterogeneous asynchronous execution architecture that includes the first thread, the second thread, and a shared circular buffer. The first thread and the second thread execute asynchronously in parallel and interact with each other through the shared circular buffer.

10. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the platform heterogeneous deployment method of the UAV visual navigation model as described in any one of claims 1 to 8.

Images