An unmanned aerial vehicle data processing system based on edge computing

Abstract

The application discloses an unmanned aerial vehicle data processing system based on edge computing and belongs to the technical field of unmanned aerial vehicle data processing.The system comprises an unmanned aerial vehicle end, an edge computing node and a ground control center; wherein the unmanned aerial vehicle end is provided with a multi-modal sensor array and is configured with a data synchronization module and a preprocessing module; a pipeline processing engine is based on a Kafka+Spark framework to realize data sharding parallel processing; a dynamic unloading controller is used to schedule computing tasks according to the computing power state of the unmanned aerial vehicle; and the ground control center is configured with a multi-modal storage system, a metadata management engine and an end-to-end security channel; through the edge node localization processing and the dynamic task unloading mechanism, in combination with the pipeline parallel engine, the end-to-end data processing delay is reduced to below 500 ms; the millisecond-level synchronization of the multi-modal sensor and the adaptive compression strategy improve the data transmission efficiency, effectively solve the massive data transmission bottleneck and meet the high-time efficiency demand of real-time monitoring, emergency response and other scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle data processing, and more particularly to an unmanned aerial vehicle data processing system based on edge computing. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, its application in the fields of surveying and mapping exploration, security monitoring, agricultural plant protection, emergency rescue, etc. is becoming more and more widespread. Modern unmanned aerial vehicles have evolved from single image acquisition devices to multi-modal data sensing platforms, usually equipped with cameras, LiDAR, GPS, IMU, infrared sensors and other sensors, which can simultaneously collect visual images, three-dimensional point clouds, position information, inertial data and thermal infrared data and other types of information, and the amount of data collected grows exponentially.

[0003] The existing unmanned aerial vehicle data processing mode mainly has the following technical bottlenecks:

[0004] Data transmission and real-time contradiction: Traditional systems mostly use "unmanned aerial vehicle collection - ground centralized processing", and a large amount of raw data is transmitted through wireless link, which is easily affected by bandwidth limitation and channel fluctuation, resulting in high transmission delay and high data packet loss rate, which is difficult to meet the timeliness requirements of real-time monitoring (such as forest fire prevention, emergency rescue) and other scenes. Although some systems try to perform simple preprocessing on the unmanned aerial vehicle side, they lack a cooperative mechanism with the backend processing node, and the processing efficiency is limited.

[0005] Computing power allocation and task adaptation imbalance: Unmanned aerial vehicle platforms are limited by size, weight and power consumption, and have limited on-board computing resources (CPU, memory, computing power), making it difficult to run complex algorithms (such as three-dimensional reconstruction, high-precision target recognition); while the ground control center has strong computing power, but remote processing has data round-trip delay, and centralized architecture is prone to performance bottlenecks in multi-machine cooperative scenarios. In existing technologies, the offloading strategy of computing tasks is mostly static configuration, which cannot be dynamically adjusted according to the real-time computing power state of the unmanned aerial vehicle, resulting in low resource utilization.

[0006] High complexity of multi-modal data management: The structured data (such as GPS coordinates) collected by unmanned aerial vehicles, unstructured data (such as images, point clouds) and time series data (such as sensor dynamic measurement values) have large differences in format and complex correlation, and traditional storage systems are difficult to achieve efficient management. At the same time, the metadata such as time stamp, geographic location and sensor parameters (such as focal length, sampling frequency) lack a unified indexing mechanism, resulting in low data retrieval efficiency and difficulty in multi-source data fusion analysis.

[0007] Inadequate security and system scalability: UAV data often involves geographic information, sensitive scenarios, and other private content. The existing encryption mechanism is imperfect, making it easy for data to be leaked. The storage layer lacks distributed redundancy strategies, and the risk of single-point failure is high. In addition, traditional systems are mostly customized architectures that are difficult to adapt to different UAV protocols (such as MAVLink and DJI Onboard SDK). When the types of sensors are expanded or the number of processing nodes is increased or decreased, a large amount of modification is required, and the flexibility is poor.

[0008] The rise of edge computing technology provides a new way to solve the above problems. By deploying computing resources on edge nodes close to the data collection end, data can be processed locally, reducing transmission pressure. However, current UAV systems based on edge computing still have problems such as imperfect coordination mechanisms, extensive task scheduling strategies, and weak multi-modal data fusion capabilities. Therefore, it is necessary to build an integrated processing system that is efficient, dynamically adaptable, secure, and reliable. In view of this, we propose a UAV data processing system based on edge computing. SUMMARY

[0009] The purpose of the present application is to provide a UAV data processing system based on edge computing to solve the problems raised in the background art.

[0010] To achieve the above purpose, the present application provides the following technical solutions:

[0011] A UAV data processing system based on edge computing includes:

[0012] A UAV end, an edge computing node, and a ground control center.

[0013] Among them:

[0014] The UAV end is equipped with a multi-modal sensor array (including a camera, a LiDAR, a GPS, an IMU, and an infrared sensor) and is configured with:

[0015] A data synchronization module: precise cross-sensor time synchronization is achieved through hardware timestamps or PPS signals.

[0016] A preprocessing module: performs data compression (including JPEG / H.265 encoding), noise reduction (including LiDAR point cloud downsampling), outlier filtering, and distortion correction.

[0017] The deployment location of the edge computing node is a ground base station or a low-altitude floating platform (floating platform height range 50-200m, coverage radius matching the UAV operation radius), which is connected to the UAV end through a low-delay communication protocol (including MQTT / WebSocket); the "low delay" in the low-delay communication protocol is defined as: in the scenario where the communication distance between the UAV end and the edge computing node is ≤5km and the wireless channel bit error rate (BER) is ≤1e-5, the end-to-end transmission delay of a single data frame (data volume ≤1MB) from the UAV end to the edge computing node is ≤100ms, and the delay jitter (Jitter) is ≤20ms; the edge computing node is configured:

[0018] Pipeline processing engine: data sharding and parallel processing are realized based on the Kafka+Spark framework;

[0019] Dynamic offloading controller: scheduling computing tasks according to the state of the UAV computing power, running lightweight algorithms (including YOLO Tiny target detection / SLAM positioning);

[0020] The ground control center is configured:

[0021] Multi-modal storage system: hierarchical storage using SQL / NoSQL / Time Series Database, distributed expansion through HDFS / MinIO;

[0022] Metadata management engine: establish a spatio-temporal index to associate data collection time, geographic location and sensor parameters;

[0023] End-to-end secure channel: encryption transmission is realized using AES-256+TLS / SSL, supporting breakpoint resume and dynamic bandwidth optimization.

[0024] Preferably, the preprocessing module integrates an adaptive compression unit, which dynamically switches encoding strategies (H.264 / H.265 / AV1) according to the quality of the wireless channel.

[0025] Preferably, the dynamic offloading controller performs:

[0026] Real-time monitoring of the CPU usage and memory load of the UAV end, the "computing power threshold" of the dynamic offloading controller is defined as the upper limit of the computing power of the UAV on-board processor (such as a quad-core ARM Cortex-A72), and the specific quantitative indicators are: CPU usage ≥85% or memory occupancy ≥90%;

[0027] The "preset value" of the dynamic offloading controller is a trigger threshold set based on the UAV on-board computing power configuration, which is: when the CPU usage is continuously ≥85% for 500ms or the memory occupancy is continuously ≥90% for 500ms, it is determined that the computing power threshold is lower than the preset value, and the SLAM mapping or three-dimensional reconstruction task is migrated to the edge node.

[0028] Preferably, the multi-modal storage system is configured to:

[0029] Distributed redundant backup module: three-copy cross-regional storage strategy is adopted;

[0030] Hot and cold data layering mechanism: automatic migration to SSD / HDD storage pool based on access frequency.

[0031] Preferably, it also includes an intelligent analysis cluster deployed in the ground control center, including:

[0032] Stream computing engine (Flink / Storm): running real-time target detection model (Faster R-CNN / YOLO);

[0033] Offline analysis module: performs point cloud processing (PCL library) and semantic segmentation (U-Net) tasks.

[0034] Preferably, it also includes a visual interaction platform, which provides:

[0035] WebGL / Cesium three-dimensional geographic information rendering;

[0036] AR / VR unmanned aerial vehicle real scene projection;

[0037] Spacetime condition retrieval interface (supports geographic fence / time slice query).

[0038] Preferably, a micro-service architecture is adopted:

[0039] Data collection / processing / storage services are deployed through Docker containers;

[0040] Automatic scaling and failover are realized based on K8s;

[0041] Support for multi-protocol adapters such as MAVLink.

[0042] Preferably, the data synchronization module is configured with a time calibration unit, which periodically corrects local clock deviation through Beidou / GPS dual-mode time signal, ensuring that the cross-sensor time synchronization accuracy is ≤1ms.

[0043] Preferably, the end-to-end secure channel integrates a channel sensing unit, which monitors wireless signal strength (RSSI) and bit error rate (BER) in real time, and automatically triggers a frequency hopping mechanism (supports 2.4GHz / 5.8GHz frequency band switching) when the channel quality is below a threshold.

[0044] Preferably, the intelligent analysis cluster configuration model adaptive scheduler can automatically match the optimal inference model according to the input data type (image / point cloud / infrared) and dynamically allocate GPU computing resources (supporting NVIDIA Jetson / AMD Radeon Pro platform); the quantitative standard of "optimality" is:

[0045] For image data: match real-time target detection model (such as YOLOv8-Tiny / YOLOv9-Lite), model inference accuracy ≥95% (based on COCO dataset verification) and inference speed ≥30fps;

[0046] For LiDAR point cloud data: match point cloud processing model (such as RANSAC segmentation model associated with PCL library), three-dimensional reconstruction accuracy ≤0.5m (relative error) and single-frame point cloud processing speed ≥5 frames / s;

[0047] For infrared sensor data: match lightweight infrared target recognition model (such as MobileNet-SSD infrared adapted version), target recognition rate ≥92% (based on infrared target dataset verification) and inference delay ≤50ms.

[0048] Compared with the prior art, the beneficial effects of the present application are:

[0049] (1) Through the edge node localization processing and dynamic task offloading mechanism, combined with the pipeline parallel engine, the end-to-end data processing delay is reduced to within 500ms, which is faster than the traditional cloud centralized processing mode; millisecond-level synchronization (≤1ms) of multi-modal sensors and adaptive compression strategy improve data transmission efficiency, effectively solve the mass data transmission bottleneck, meet the high-time demand of real-time monitoring, emergency response and other scenes. The coordination of dynamic offloading controller and model adaptive scheduler realizes the intelligent allocation of computing power of unmanned aerial vehicle end, edge node and ground center, prolongs the endurance of unmanned aerial vehicle, improves the utilization rate of GPU computing power; micro-service architecture and multi-protocol adaptation support shorten the system deployment time, compatible with multiple brands of unmanned aerial vehicle equipment, improve the adaptability compared with traditional scheme, greatly reduce the cost of multi-scene expansion.

[0050] (2) The multi-modal storage system combines space-time index and hot and cold layering mechanism to realize efficient management of PB-level data, with retrieval efficiency improved by 5-10 times and storage cost reduced; three-copy redundancy and end-to-end encryption (AES-256+TLS / SSL) combined with dynamic frequency hopping technology make the data reliable, reduce the packet loss rate, and meet the requirements of safe storage and transmission of sensitive data. The intelligent analysis cluster realizes the whole process coverage of "real-time detection-offline deep analysis", with high target detection accuracy and three-dimensional modeling efficiency improved by 3 times; the combination of AR / VR visualization and space-time retrieval interface improves the data analysis efficiency and reduces the operation threshold of non-professional users, providing intuitive decision support for surveying and exploring, security monitoring and other fields. The system breaks through the bottleneck of traditional unmanned aerial vehicle data processing in real-time, resource adaptation, security management and other aspects through edge computing and cloud-edge collaborative architecture, forming an efficient, reliable and flexible integrated solution, with significant technical advancement and industrial application value. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments.

[0052] Embodiment:

[0053] An unmanned aerial vehicle data processing system based on edge computing, comprising:

[0054] An unmanned aerial vehicle end, an edge computing node and a ground control center.

[0055] Among them:

[0056] The unmanned aerial vehicle end carries a multi-modal sensor array (including a camera, a LiDAR, a GPS, an IMU and an infrared sensor) and is configured to:

[0057] A data synchronization module: precise time synchronization across sensors is realized through hardware timestamps or PPS signals; wherein the data synchronization module is configured with a time calibration unit to periodically correct local clock deviation through Beidou / GPS dual-mode time signal, ensuring that the cross-sensor time synchronization accuracy is ≤1ms.

[0058] By periodically correcting the local clock deviation through Beidou / GPS dual-mode time signal, the cross-sensor time synchronization accuracy is controlled within ≤1ms, solving the fusion error problem caused by the inconsistency of the time axis of multi-modal sensors, providing a high-precision space-time reference for subsequent three-dimensional reconstruction, motion trajectory calculation and other applications, and improving the synchronization accuracy by 3-5 times compared with traditional single-mode time synchronization scheme.

[0059] I. Specific implementation scheme of hardware timestamp and PPS signal

[0060] The data synchronization module realizes cross-sensor accurate synchronization through "hardware timestamp chip + standardized interface protocol + PPS signal direct connection circuit". The core hardware selection, interface definition and connection mode are as follows, which can ensure that the skilled person can directly reproduce.

[0061] 1. Carrier and interface protocol of hardware timestamp

[0062] Core chip selection: high-precision timestamp chip DS3231M (Maxim Integrated product) is used as the hardware timestamp carrier. The chip has a built-in temperature-compensated crystal oscillator (TCXO), and the time accuracy can reach ±2ppm (-40℃~85℃ working temperature range). It supports second-level pulse (PPS) output and I²C interface communication, and meets the miniaturization and low power consumption requirements of the unmanned aerial vehicle end (working current ≤100μA).

[0063] Interface protocol and data interaction logic:

[0064] The chip is connected with the unmanned aerial vehicle end host unit (such as a quad-core ARM Cortex-A72 processor) through the I²C interface (SDA pin connects the sensor mainboard I²C_SDA, SCL pin connects I²C_SCL, and the supply voltage is 3.3V);

[0065] Each sensor (camera, LiDAR, GPS, IMU, infrared sensor) is configured with an independent "timestamp collection pin". When the sensor outputs a frame of data, it synchronously sends a "data trigger signal" (high-level pulse, pulse width 10μs) to the DS3231M chip;

[0066] After receiving the trigger signal, the DS3231M immediately records the current time (accuracy ≤100ns), and returns the timestamp (format: year - month - day hour: minute: second. Microsecond) to the host unit through the I²C interface, realizing the one-to-one binding of "data frame - hardware timestamp".

[0067] 2. Access method and connection circuit of PPS signal

[0068] PPS signal source: output by Beidou / GPS dual-mode module (such as UBLOX ZED-F9P). The module supports 1 PPS pulse output per second (pulse width 100ms, high level 3.3V), and the time accuracy of the pulse rising edge is ≤1ns, providing a reference clock for cross-sensor synchronization.

[0069] Connection circuit design:

[0070] Adopt "star topology" circuit to realize the synchronization transmission of PPS signal to all sensors, avoid the synchronization deviation caused by signal attenuation;

[0071] The circuit core components include: 74HC125 buffer (used to enhance the driving ability of PPS signal, avoid signal distortion caused by multi-sensor load), 100Ω matching resistor (series in each PPS signal output terminal, reduce signal reflection), 0.1μF decoupling capacitor (parallel to the power supply end of the buffer, suppress power supply noise);

[0072] Specific wiring logic: PPS pin of UBLOX ZED-F9P → 74HC125 input pin → 74HC125 output pin (5 paths, corresponding to 5 types of sensors) → 100Ω resistor in series in each output terminal → PPS_IN pin of camera, SYNC pin of LiDAR, TIME_REF pin of GPS, CLK_IN pin of IMU, and TRIGGER pin of infrared sensor respectively;

[0073] Circuit PCB layout requirements: PPS signal wiring length difference ≤5cm (avoid transmission delay difference), wiring away from power supply line (distance ≥2mm, reduce electromagnetic interference).

[0074] II. Clock correction mechanism of Beidou / GPS dual-mode time service

[0075] To realize the cross-sensor time synchronization accuracy ≤1ms, the module uses the Beidou / GPS dual-mode time service signal to correct the local clock through the closed-loop logic of "periodic sampling - deviation judgment - incremental correction", and the specific parameters, algorithms and processes are shown in Table 1:

[0076] 1. Definition of core parameters (quantitative indicators) in Table 1

[0077] ;

[0078] 2. Correction algorithm logic (specific steps)

[0079] Adopt "linear interpolation correction algorithm" to avoid data time sequence disorder caused by clock mutation, and the algorithm flow is as follows:

[0080] Time sampling stage:

[0081] "Double-mode time sampling" is triggered once every 100ms, and the main control unit reads the UTC time (including the accurate time corresponding to the rising edge of second-level PPS, format: second.nanosecond) output by UBLOX ZED-F9P through UART interface (baud rate 115200bps).

[0082] At the same time, read the current time of the local DS3231M chip, calculate the difference At of "dual-mode time - local time". If At is positive, it means that the local clock is lagging; if it is negative, it means that the local clock is ahead.

[0083] Deviation judgment stage:

[0084] If |At|≤500ns: determine that the local clock drift is within the allowed range, do not perform correction, only record At for subsequent trend analysis;

[0085] If |At|>500ns: determine that correction needs to be performed, enter the next adjustment logic.

[0086] Incremental correction stage:

[0087] If the local clock lags (At>500ns): send "clock acceleration command" to DS3231M through I2C interface, increase the internal oscillator frequency of the chip by 0.1ppm (corresponding to an increase of 100ns per 100ms) each time, until At≤500ns;

[0088] If the local clock is ahead (At<-500ns): send "clock deceleration command", reduce the oscillator frequency by 0.1ppm (corresponding to a decrease of 100ns per 100ms) each time, until At≤500ns;

[0089] During the correction process, the continuity of the timestamp of the sensor data frame is monitored in real time. If the timestamp jumps (such as the difference between adjacent frame timestamps <0), immediately suspend the correction and revert to the last clock configuration to avoid data timing errors.

[0090] 3. Synchronization accuracy verification data

[0091] In the actual test scenario (drone operation height 50-200m, dual-mode module star search number≥8, signal-to-noise ratio≥45dB), the performance indicators of the data synchronization module are as follows:

[0092] Cross-sensor time synchronization accuracy: average deviation 320ns, maximum deviation 480ns, all≤1ms, meeting the requirement of "≤1ms synchronization accuracy" in this application;

[0093] Clock correction response speed: from detecting deviation>500ns to correcting to≤500ns, the average time consumption is 300ms (3 correction periods), and there is no synchronization failure caused by correction timeout;

[0094] Anti-interference capability: In an electromagnetic interference environment (such as 2.4GHz WiFi signal superposition), the PPS signal transmission delay fluctuation is ≤100ns, and the corrected synchronization accuracy can still be maintained at ≤500ns, with no obvious performance degradation.

[0095] Preprocessing module: performs data compression (including JPEG / H.265 encoding), noise reduction (including LiDAR point cloud downsampling), outlier filtering, and distortion correction; wherein the preprocessing module integrates an adaptive compression unit that dynamically switches encoding strategies (H.264 / H.265 / AV1) according to wireless channel quality.

[0096] The integrated adaptive compression unit can dynamically switch encoding strategies according to wireless channel quality, using H.265 / AV1 encoding to achieve high compression ratio (40% less data than JPEG) when the channel is good, and automatically downgrading to H.264 to ensure transmission stability when the channel is deteriorating, improving overall transmission efficiency, while reducing the proportion of invalid data through LiDAR point cloud downsampling and outlier filtering.

[0097] I. The core logic of the adaptive compression unit is to collect key parameters of the wireless channel in real time, establish a quantitative evaluation system, and use it as the basis for switching encoding strategies to ensure dynamic balance between "transmission efficiency" and "data integrity". The specific channel quality parameters and quantitative threshold values are defined as follows:

[0098] 1. Core evaluation parameters and threshold ranges

[0099] The adaptive compression unit collects 3 types of key channel parameters through the UAV-side communication module (such as an industrial-grade WiFi 6 module or a 4G / 5G data transmission module) in real time, and completes parameter sampling and threshold comparison every 100ms. The specific parameters and switching trigger thresholds are shown in Table 2:

[0100] Table 2 Switching trigger thresholds

[0101] ;

[0102] Note: When there is a conflict among the 3 types of parameters (such as RSSI≥-70dBm but BER≥1e-4), "bit error rate (BER)" is used as the highest priority judgment basis to ensure the accuracy of data transmission is prioritized over transmission efficiency.

[0103] 2. Parameter collection and feedback mechanism

[0104] The UAV-side communication module has a built-in channel monitoring unit that achieves parameter collection and feedback through the following process:

[0105] Step 1: The communication module receives the channel state response frame returned by the edge computing node every 50ms (including the RSSI and BER data received by the edge node side);

[0106] Step 2: The adaptive compression unit aggregates the "drone-end transmitted RSSI" and "edge node received RSSI / BER", calculates the average value (such as the average RSSI of 3 consecutive samples), and avoids false switching caused by single fluctuation;

[0107] Step 3: Compare the calculated parameter value with the threshold table to generate "encoding strategy switching instructions" and send them to the encoding unit of the preprocessing module through the SPI interface.

[0108] II. Specific logic and parameter configuration of encoding strategy switching

[0109] The adaptive compression unit presets differentiated encoding parameters for H.264, H.265, and AV1 encoding strategies, and defines the data connection mechanism in the switching process to avoid frame loss or data disorder caused by switching.

[0110] 1. Core parameter configuration of each encoding strategy

[0111] The parameter configuration of different encoding strategies needs to match the channel quality to ensure the adaptation between "compression ratio" and "decoding complexity". The specific parameters are as follows in Table 3:

[0112] Table 3 Core parameter configuration

[0113] ;

[0114] Note: The code rate adopts "dynamic code rate (VBR)" mode. When the image content complexity is high (such as dense forest vegetation scene), the code rate is automatically increased to the upper limit of the current strategy to avoid picture blur; when the image content is simple (such as open plain), the code rate is reduced to the lower limit to save bandwidth.

[0115] 2. Connection mechanism of encoding switching

[0116] To avoid data loss or decoding anomalies during switching, the adaptive compression unit adopts the "frame synchronization switching" mechanism, and the specific process is as follows:

[0117] Step 1: When it is detected that the channel quality triggers switching (such as switching from H.265 to H.264), the compression unit first completes the encoding and transmission of the current encoding frame (P frame);

[0118] Step 2: Send 1 "encoding switching identification frame" (frame header contains new encoding strategy type and parameter configuration) to the edge computing node to inform the receiving end that the encoding format will be switched;

[0119] Step 3: After receiving the identification frame, the edge computing node sends back a "switch confirmation frame". After receiving the confirmation, the drone-side compression unit starts using the new encoding strategy from the next I-frame.

[0120] Step 4: If no confirmation frame is received from the edge computing node within 100ms, the compression unit automatically reverts to the original encoding strategy and re-triggers channel parameter sampling to avoid data disconnection caused by blind switching.

[0121] III. Hardware Implementation and Performance Verification Data

[0122] Hardware Carrier of Adaptive Compression Unit

[0123] The adaptive compression unit is integrated into the pre-processing chip at the drone end (such as TI TDA4VM or NVIDIA Jetson Nano). It collects RSSI and BER parameters through the built-in "channel monitoring interface" (supporting IEEE 802.11ax protocol) and realizes real-time parameter reading and encoding strategy control through software drivers (based on the netlink protocol of the Linux kernel). The hardware resource occupancy rate is ≤15% (CPU usage ≤10%, memory occupancy ≤5%), which does not affect the normal operation of other modules at the drone end (such as data synchronization and IMU data collection).

[0124] Performance Verification Results

[0125] In the actual test scenario (drone operation height 100m, edge computing node deployed at ground base station, communication distance 3km), the encoding switching effect under different channel qualities is as follows:

[0126] High-quality channel (RSSI=-65dBm, BER=5e-7): using AV1 encoding, the compressed data volume of a single frame image (2000 million pixels, RGB format) is reduced from the original 20MB to 6~8MB, the transmission delay is ≤80ms, and there is no frame loss;

[0127] Medium channel (RSSI=-85dBm, BER=5e-6): automatically switched to H.265 encoding, the compressed data volume is 10~12MB, the transmission delay is ≤95ms, and the packet loss rate is ≤0.1%;

[0128] Poor channel (RSSI=-95dBm, BER=5e-4): automatically switched to H.264 low bit rate mode, the compressed data volume is 14~16MB, the transmission delay is ≤100ms, and the packet loss rate is ≤0.5%, meeting the minimum data integrity requirement (packet loss rate ≤1%) of real-time monitoring scenarios.

[0129] The above verification data shows that the switching logic of the adaptive compression unit can stably adapt to different channel qualities.

[0130] The UAV-side hardware can use a 20-megapixel industrial camera with a frame rate of 30 fps. The LiDAR uses a 16-line laser radar with a ranging range of 5-200 m and a point cloud density of 200 points / m2. The GPS / IMU module selects a product that supports Beidou and GPS dual-mode positioning, with a positioning accuracy of centimeter level and a refresh rate of 10 Hz. A quad-core ARM Cortex-A72 processor is used, coupled with 4GB of LPDDR4 memory, to meet the computing power requirements of data synchronization and preprocessing.

[0131] The edge computing node is deployed at a ground base station or a low-altitude floating platform (floating platform height range 50-200 m, coverage radius matching the UAV operation radius), and is connected to the UAV side through a low-latency communication protocol (including MQTT / WebSocket). In the low-latency communication protocol, "low latency" is defined as: in a scenario where the communication distance between the UAV side and the edge computing node is ≤5 km and the wireless channel bit error rate (BER) is ≤1e-5, the end-to-end transmission delay of a single data frame (data volume ≤1 MB) from the UAV side to the edge computing node is ≤100 ms, and the delay jitter (Jitter) is ≤20 ms. The edge computing node is configured with:

[0132] Pipeline processing engine: based on Kafka+Spark framework, data sharding parallel processing is realized, single node data processing throughput is improved to 2-3 times of traditional architecture, 8-16 UAVs' concurrent data flow can be processed at the same time, processing delay is controlled within 500 ms, meeting the response requirements of real-time monitoring scenarios.

[0133] Dynamic offloading controller: according to the UAV computing power state, the computing task is scheduled, and lightweight algorithms (including YOLO Tiny target detection / SLAM positioning) are run; when the UAV computing power is insufficient, heavy tasks such as SLAM mapping are migrated to the edge node, prolonging the UAV's endurance, and the lightweight algorithms of the edge node can realize 30 frames per second of target detection, reducing the response delay compared to the traditional cloud processing mode.

[0134] Specifically, the dynamic offloading controller performs real-time monitoring of the UAV-side CPU usage and memory load. The "computing power threshold" of the dynamic offloading controller is defined as the upper limit of the computing power of the UAV on-board processor (such as quad-core ARM Cortex-A72), and the specific quantitative indicators are: CPU usage ≥ 85% or memory occupancy rate ≥ 90%.

[0135] The "preset value" of the dynamic offloading controller is a trigger threshold set based on the onboard computing power configuration of the UAV, specifically: when the CPU usage is continuously monitored for 500ms≥85% or the memory occupancy is continuously monitored for 500ms≥90%, it is determined that the computing power threshold is below the preset value, and the SLAM mapping or three-dimensional reconstruction task is migrated to the edge node.

[0136] The edge computing node hardware uses an industrial-grade edge server, configured with an Intel Xeon E-2274G processor, 16GB DDR4 memory, equipped with a 1TB SSD solid state disk, and supports multi-network port design to achieve high-speed communication with multiple UAVs and efficient data processing.

[0137] 1. Quantitative standard of computing power threshold (based on UAV hardware configuration)

[0138] Taking the mainstream onboard processor of the UAV (quad-core ARM Cortex-A72, 1.5GHz, 4GB LPDDR4 memory) as the benchmark, the "computing power threshold" is defined as the maximum carrying capacity of this hardware when running stably, and the specific quantitative indicators and monitoring dimensions are shown in Table 4:

[0139] Table 4 Quantitative standard of computing power threshold

[0140] ;

[0141] 2. Determination basis of preset value (hardware and task dual adaptation)

[0142] The "preset value" is the determination threshold of offloading trigger, which needs to be dynamically adjusted according to the differences in UAV hardware models and the current task priority, and the specific determination logic is as follows:

[0143] Hardware model adaptation: For different onboard processors with different computing power, the preset value is adjusted according to the "hardware computing power ratio". For example:

[0144] If the UAV uses a low-power processor (dual-core ARM Cortex-A53, 1.2GHz), the CPU load threshold preset value is reduced to 75%, and the memory load threshold is reduced to 85% (because the hardware computing power is only 60% of the benchmark configuration);

[0145] If a high-power processor (six-core ARM Cortex-A78, 2.0GHz) is used, the CPU load threshold preset value is increased to 90%, and the memory load threshold is increased to 92%.

[0146] Task priority adaptation: According to the real-time requirements of the current task executed by the UAV, the preset value is fine-tuned by ±5%, and the specific rules are as follows:

[0147] High-priority tasks (such as real-time forest fire monitoring, emergency rescue target identification): preset value down 5% (such as CPU threshold 80%), to avoid insufficient computing power leading to emergency response delay;

[0148] Low-priority tasks (such as offline data collection for farmland mapping): preset value up 5% (such as CPU threshold 90%), to fully utilize on-board computing power, reduce unloading frequency, and reduce edge node communication pressure.

[0149] 3. Configuration and update mechanism of preset values

[0150] The controller has a built-in "hardware computing power detection script". When the drone is powered on and initialized, it automatically reads the processor model, memory capacity, and other hardware information, and generates initial preset values through the pre-stored "hardware-threshold mapping table";

[0151] When the task starts, the user selects the task priority (high / medium / low) through the visual interaction platform of the ground control center. After the controller receives the priority instruction, it automatically completes the ±5% fine-tuning of the preset values, and through the "preset value verification mechanism" (simulating 10% load fluctuation to verify whether the threshold triggers false unloading), it ensures the reasonableness.

[0152] Ground control center configuration:

[0153] Multi-modal storage system: uses SQL / NoSQL / time series database hierarchical storage, and realizes distributed expansion through HDFS / MinIO; Specifically, the multi-modal storage system configuration: distributed redundant backup module: uses three-copy cross-regional storage strategy; hot and cold data layering mechanism: automatically migrates to SSD / HDD storage pool based on access frequency.

[0154] The three-copy cross-regional storage strategy makes the data reliability reach 99.999%, and the hot and cold data layering mechanism realizes storage cost reduction through automatic migration. Combined with the distributed expansion capability of HDFS / MinIO, it can support PB-level data storage and hundreds of GB-level data read-write throughput per second, meeting the long-term data archiving needs of large-scale drone clusters.

[0155] Metadata management engine: establishes spatiotemporal index to associate data collection time, geographic location, and sensor parameters; The established spatiotemporal indexing mechanism improves data retrieval efficiency by 5-10 times, supports millisecond-level geographic fence and time slicing query in million-level data sets, and solves the problem of multi-source data correlation analysis difficulty in traditional storage systems.

[0156] End-to-end secure channel: uses AES-256+TLS / SSL to realize encrypted transmission, supports breakpoint resume and bandwidth dynamic optimization.

[0157] Specifically, the end-to-end secure channel integrates a channel awareness unit to monitor wireless signal strength (RSSI) and bit error rate (BER) in real time. When the channel quality falls below a threshold, it automatically triggers a frequency hopping mechanism (supporting 2.4GHz / 5.8GHz band switching). AES-256+TLS / SSL encrypted transmission combined with the dynamic frequency hopping mechanism ensures data transmission security meets military-grade standards. The channel awareness unit's frequency band switching response time during signal degradation is <100ms, ensuring communication continuity in complex electromagnetic environments and reducing packet loss compared to fixed-band solutions.

[0158] This application also includes an intelligent analysis cluster, deployed at a ground control center, comprising:

[0159] Streaming engine (Flink / Storm): Runs real-time object detection models (Faster R-CNN / YOLO);

[0160] Offline analysis module: performs point cloud processing (PCL library) and semantic segmentation (U-Net) tasks.

[0161] The collaborative architecture of the streaming computing engine and the offline analysis module enables full-process coverage of "real-time detection - offline deep analysis": the Flink / Storm engine supports real-time target detection at 200 frames per second with an accuracy of 98%; the point cloud processing solution of PCL library and U-Net model can complete 3D modeling of a 10 square kilometer area in 2 hours, which is 3 times more efficient than traditional solutions.

[0162] The model adaptive scheduler improves GPU computing power utilization to over 85% by intelligently matching data types with inference models, supports collaborative scheduling of heterogeneous computing resources across multiple platforms, and reduces computing power consumption by 30% while ensuring analysis accuracy.

[0163] This application also includes a visual interactive platform, providing:

[0164] WebGL / Cesium 3D geographic information rendering; AR / VR drone real-scene projection; spatiotemporal condition retrieval interface (supports geofencing / time slice query).

[0165] The combination of WebGL / Cesium 3D rendering and AR / VR real-scene projection technology enables immersive visualization of drone data. The spatiotemporal condition retrieval interface allows users to complete multi-source data filtering for specific areas and time periods within 30 seconds, improving data analysis efficiency by 40% compared to traditional 2D interfaces and significantly reducing the operating threshold for non-professional users.

[0166] This application adopts a microservice architecture:

[0167] Data collection / processing / storage services are deployed through Docker containerization; automatic scaling and failover are implemented based on K8s; and multi-protocol adapters such as MAVLink are supported. The micro-service architecture combined with Docker containerization deployment shortens the system deployment time from the traditional 72 hours to 4 hours, and the automatic scaling capability of K8s can complete resource elasticity adjustment within 5 minutes to meet the business load fluctuation demand. The multi-protocol adapter supports the communication protocols of mainstream unmanned aerial vehicle manufacturers, and the compatibility is improved by 80% compared with the customized solution, significantly reducing the cost of collaborative application of multi-brand unmanned aerial vehicle clusters.

[0168] In this application, the intelligent analysis cluster configuration model adaptive scheduler can automatically match the optimal inference model according to the input data type (image / point cloud / infrared) and dynamically allocate GPU computing resources (supporting NVIDIA Jetson / AMD Radeon Pro platforms); the quantitative standard for "optimal" is:

[0169] For image data: match real-time target detection models (such as YOLOv8-Tiny / YOLOv9-Lite), model inference accuracy ≥ 95% (based on COCO dataset verification) and inference speed ≥ 30fps;

[0170] For LiDAR point cloud data: match point cloud processing models (such as RANSAC segmentation models associated with PCL library), three-dimensional reconstruction accuracy ≤ 0.5m (relative error) and single-frame point cloud processing speed ≥ 5 frames / s;

[0171] For infrared sensor data: match lightweight infrared target recognition models (such as MobileNet-SSD infrared adapted version), target recognition rate ≥ 92% (based on infrared target dataset verification) and inference delay ≤ 50ms.

[0172] The workflow of the system is as follows: the unmanned aerial vehicle end collects data through a multi-modal sensor array, the data synchronization module performs accurate time synchronization on cross-sensor data, the preprocessing module performs compression, noise reduction, outlier filtering and distortion correction and the like on the data. The processed data is transmitted to the edge computing node through a low-delay communication protocol. The pipeline processing engine of the edge computing node performs sharding and parallel processing of data based on the Kafka+Spark framework, the dynamic offloading controller schedules computing tasks according to the unmanned aerial vehicle computing power state, and runs a lightweight algorithm. When the unmanned aerial vehicle computing power is insufficient, part of the tasks are migrated to the edge node. Then, the edge computing node transmits the processed data to the ground control center, the multi-modal storage system of the ground control center performs hierarchical storage of the data, the metadata management engine establishes a spatio-temporal index to associate related information, and the end-to-end secure channel ensures the safety of data transmission. The intelligent analysis cluster performs real-time detection and offline deep analysis of the data, and the visual interaction platform provides a data display and interaction interface for users.

[0173] The pipeline processing engine is based on Kafka (message queue) and Spark Streaming (real-time computing framework) and needs to be configured with a clear topic partition strategy, Spark parallel degree and hardware resource matching to achieve the goals of "data sharding and parallel processing" and "delay ≤ 500ms". The specific configuration parameters and design basis are as follows:

[0174] 1. Kafka topic design and partition strategy

[0175] Kafka, as a "middleware" for data transmission, needs to be divided into topics and partitioned reasonably to achieve parallel reception and ordered distribution of multi-unmanned aerial vehicle data. The core configurations are shown in Tables 5 and 6:

[0176] Table 5 Kafka configuration

[0177] ;

[0178] Table 6, Spark Streaming configuration (benchmark hardware: 8-core 16-thread, 16GB memory)

[0179] ;

[0180] Optimization means for delay ≤ 500ms

[0181] Data preloading: Kafka consumer instance cache (initialization time consumption from 50ms to 5ms); static parameters (camera intrinsic parameters, etc.) are broadcast to the Executor memory to avoid repeated reading.

[0182] Cache mechanism: intermediate results (e.g., point cloud denoising results) are cached in memory (validity period 5 minutes), reducing the time-consuming of repeated calculations from 150ms to 30ms; Kafka offset is checkpointed every 1 second, reducing the time-consuming of restart recovery from 3s to 500ms.

[0183] Computational optimization: data shards ≤20MB per shard, filtering 5% invalid data in advance; algorithm rewritten as SparkUDF to reduce cross-language overhead (single-frame detection time-consuming 80ms→40ms); G1 GC limits pause ≤20ms to avoid delay peaks.

[0184] Distributed storage core strategy (quantitative rules)

[0185] Three-copy cross-regional storage:

[0186] Adopting the "master node + off-site dual backup" strategy, the master copy is stored in the edge node closest to the UAV operation area (e.g., Yulin node in northwest operation), and the backup copy is stored in two regional nodes ≥500km away from the master node (e.g., Xi'an and Lanzhou nodes), ensuring that data is not lost in the event of single-region failure, with a replica synchronization delay ≤100ms.

[0187] Hot and cold data migration:

[0188] Triggering conditions: based on access frequency + storage duration in two dimensions, data has not been accessed for 7 consecutive days, or the number of accesses within 30 days ≤3 times, automatically marked as cold data;

[0189] Migration logic: hot data (high-frequency access within 30 days) is stored in an SSD storage pool (read / write speed ≥500MB / s), and cold data is migrated to an HDD storage pool (cost 60% lower than SSD), with the migration process executed in the background and without interrupting data reading.

[0190] AES-256+TLS / SSL encryption mechanism

[0191] AES-256 key generation using "device unique ID + timestamp" hybrid method: using UAV IMEI code (64 bits) and Beidou time stamp (accurate to milliseconds) as seeds, generating a 256-bit key through HMAC-SHA256 algorithm, and automatically rotating every 24 hours.

[0192] Key distribution uses a combination of offline preloading and dynamic updating mechanism: preloading root key through hardware security module (HSM) at factory; dynamically negotiating session key through TLS 1.3 handshake process during runtime, with key transmission using ECDH elliptic curve encryption (curve secp256r1).

[0193] TLS / SSL configuration forces TLS 1.3, selects TLS_AES_256_GCM_SHA384 for encryption suite, disables TLS 1.2 and below versions; uses ECC elliptic curve certificate (key length 256 bits) with a validity period of 1 year and supports automatic renewal.

[0194] Key parameters of frequency hopping mechanism

[0195] The channel quality judgment standard is RSSI≤-85dBm or BER≥1e-5, with a sampling frequency of 1 time per 50ms, and the frequency hopping is started when it is triggered for 3 consecutive times.

[0196] Channel scanning and switching process: uses a pre-stored channel list for fast matching, with a scanning time of ≤30ms and a switching delay of ≤80ms (including frequency switching + re-synchronization). Preferentially switches to the 5.8GHz frequency band (less interference), and if the quality of this frequency band does not meet the standard, it falls back to the 2.4GHz frequency band.

[0197] Core logic of model adaptive scheduler

[0198] The judgment of the "optimal" inference model uses a scene priority dynamic switching mechanism: real-time response scenes (such as emergency rescue, forest fire prevention) use "speed priority" as the standard (inference delay ≤50ms), and high-precision analysis scenes (such as farmland mapping, cultural relic inspection) use "precision priority" as the standard (accuracy rate ≥95%).

[0199] The mapping relationship between data types and models is clear:

[0200] Image data (visible light) matches YOLOv8-Tiny (real-time scene) or YOLOv9-L (high-precision scene), supporting COCO dataset 1000 class target detection;

[0201] LiDAR point cloud data matches RANSAC segmentation model associated with PCL library (real-time) or PointNet++ (high-precision), realizing three-dimensional target segmentation and reconstruction;

[0202] Infrared sensor data matches MobileNet-SSD infrared adaptation version (real-time) or EfficientNet-B0 infrared special model (high-precision), optimizing the recognition rate for high-temperature targets (such as fire points, equipment thermal faults).

[0203] GPU computing power dynamic allocation strategy

[0204] Based on model calculation (FLOPs) and memory requirements to allocate resources:

[0205] Lightweight models (such as YOLOv8-Tiny, 10 GFLOPs) allocate 30% of single GPU memory (≤2GB) and exclusively use 1 CUDA core;

[0206] Medium models (such as PointNet++, 50 GFLOPs) allocate 50% of memory (2-4GB) and use 2-4 CUDA cores;

[0207] Heavy models (such as EfficientNet-B0, 100 GFLOPs) allocate 60% of memory (4-6GB) and use 4-8 CUDA cores.

[0208] When multiple models are concurrent, time-slice round-robin scheduling is used (switch every 10ms), ensuring no model starvation.

[0209] Supported hardware models and driver configurations:

[0210] NVIDIA Jetson series: Jetson AGX Orin (64GB memory), Jetson Xavier NX (16GB memory), driver JetPack 5.1 (including CUDA 11.4, TensorRT 8.5);

[0211] AMD Radeon Pro series: Radeon Pro W6800 (32GB memory), driver RadeonSoftware 22.40 (supports OpenCL 3.0, ROCm 5.2).

[0212] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A UAV data processing system based on edge computing, characterized in that, include: Unmanned aerial vehicle (UAV) terminals, edge computing nodes, and ground control centers; in: The drone is equipped with a multimodal sensor array, including a camera, LiDAR, GPS, IMU, and infrared sensor, with the following configuration: Data synchronization module: Configured with a time calibration unit, which periodically corrects the local clock deviation through BeiDou / GPS dual-mode timing signal to ensure cross-sensor time synchronization accuracy ≤1ms; Preprocessing module: Performs data compression, noise reduction, outlier filtering, and distortion correction; The edge computing nodes are deployed at ground base stations or low-altitude floating platforms, with the floating platform altitude ranging from 50-200m and a coverage radius matching the UAV's operating radius. They connect to the UAV via a low-latency communication protocol. "Low latency" in this protocol is defined as follows: in scenarios where the communication distance between the UAV and the edge computing node is ≤5km and the wireless channel bit error rate is ≤1e-5, the end-to-end transmission latency of a single data frame from the UAV to the edge computing node is ≤100ms, and the latency jitter is ≤20ms. The edge computing node is configured as follows: Pipeline processing engine: Implements parallel data sharding based on the Kafka+Spark framework; The dynamic offloading controller executes the following: real-time monitoring of the drone's CPU utilization and memory load. The "computing power threshold" of the dynamic offloading controller is defined as the upper limit of the computing power of the drone's onboard processor. The specific quantitative indicators are: CPU utilization ≥ 85% or memory utilization ≥ 90%. The "preset value" of the dynamic offloading controller is a trigger threshold set based on the drone's onboard computing power configuration. Specifically, when the CPU utilization is continuously ≥ 85% for 500ms or the memory utilization is continuously ≥ 90% for 500ms, it is determined that the computing power threshold is lower than the preset value, and the SLAM mapping or 3D reconstruction task is migrated to the edge node. The ground control center is configured as follows: Multimodal storage system: adopts layered storage of SQL / NoSQL / time-series databases, and achieves distributed expansion through HDFS / MinIO; Metadata management engine: Establishes spatiotemporal indexes to associate data collection time, geographical location, and sensor parameters; End-to-end secure channel: AES-256+TLS / SSL is used for encrypted transmission, supporting breakpoint resumption and dynamic bandwidth optimization.

2. The UAV data processing system based on edge computing according to claim 1, characterized in that: The preprocessing module integrates an adaptive compression unit that dynamically switches coding strategies based on the quality of the wireless channel.

3. The UAV data processing system based on edge computing according to claim 1, characterized in that: The multimodal storage system is configured as follows: Distributed redundant backup module: adopts a three-replica cross-regional storage strategy; Hot and cold data tiering mechanism: Automatically migrate data to SSD / HDD storage pools based on access frequency.

4. The UAV data processing system based on edge computing according to claim 1, characterized in that: It also includes an intelligent analysis cluster, deployed at a ground control center, comprising: Streaming engine: Runs real-time object detection models; Offline analysis module: performs point cloud processing and semantic segmentation tasks.

5. The UAV data processing system based on edge computing according to claim 1, characterized in that: It also includes a visual interactive platform, providing: WebGL / Cesium 3D geographic information rendering; AR / VR drone real-scene projection; The spatiotemporal condition retrieval interface supports geofence / time slice queries.

6. The UAV data processing system based on edge computing according to claim 1, characterized in that: Adopting a microservice architecture: Deploy data acquisition / processing / storage services using Docker containerization; Automatic scaling and failover based on Kubernetes; Supports MAVLink multi-protocol adapters.

7. The UAV data processing system based on edge computing according to claim 1, characterized in that: The end-to-end secure channel integrates a channel sensing unit to monitor wireless signal strength and bit error rate in real time, and automatically triggers a frequency hopping mechanism when the channel quality is below a threshold.

8. The UAV data processing system based on edge computing according to claim 4, characterized in that: The intelligent analysis cluster is configured with a model adaptive scheduler, which can automatically match the optimal inference model based on the input data type and dynamically allocate GPU computing resources; the quantification standard for "optimal" is: For image data: Matching real-time object detection models with inference accuracy ≥95%, validated on the COCO dataset and inference speed ≥30fps; For LiDAR point cloud data: Match the point cloud processing model, achieve a 3D reconstruction accuracy of ≤0.5m and a single-frame point cloud processing speed of ≥5 frames / s; For infrared sensor data: Match a lightweight infrared target recognition model with a target recognition rate of ≥92% and an inference latency of ≤50ms.