Network and task dual-driven unmanned aerial vehicle image instant return method in complex environment

Abstract

The invention belongs to the technical field of unmanned aerial vehicle image instant return methods, and particularly relates to a network and task dual-driven unmanned aerial vehicle image instant return method in a complex environment, which comprises the following steps of: acquiring four network index data of bandwidth, delay, jitter and packet loss rate, performing exponential weighted moving average filtering and denoising, and performing weighted calculation to obtain a network comprehensive quality score; five discrete network state levels are mapped, and a lagging threshold value is set for level switching; dividing an original image layer, a standard image layer and a thumbnail layer, setting a compression ratio according to a task type, determining the number of slices of each layer according to a corresponding rule to ensure that tile data volumes are consistent, and establishing association of tiles of different image layers by adopting a numbering rule; the comprehensive priority of image tiles is calculated based on four factors of a current viewport, user interaction, points of interest and a flight route, a priority queue jumping strategy is adopted, a network state level and a task type are combined, a to-be-transmitted image layer, the number of concurrent threads and a preloading strategy are dynamically adjusted, and instant image returning is achieved.

Description

Technical Field

[0001] This invention belongs to the technical field of real-time image transmission methods for unmanned aerial vehicles (UAVs), and particularly relates to a real-time image transmission method for UAVs driven by both network and task in complex environments. Background Technology

[0002] Real-time image transmission technology from drones is widely used in scenarios such as power line inspection, ecological monitoring, and field exploration. However, complex field environments often face problems such as network signal obstruction, drastic bandwidth fluctuations, and unstable latency and packet loss rates, which severely restrict the real-time performance and reliability of image transmission. Traditional transmission methods mostly use fixed compression rates, slice sizes, and transmission strategies, lacking accurate perception of dynamic network conditions. This can easily lead to wasted network resources or transmission interruptions, making it difficult to balance the core requirements of image clarity and smooth loading.

[0003] Existing technologies have many limitations: On the one hand, network quality assessment relies on a single indicator and lacks a multi-dimensional quantitative model, making it difficult to accurately reflect the trend changes of the network in complex environments. Furthermore, the absence of a level switching lag mechanism leads to frequent adjustments in transmission strategies. On the other hand, image preprocessing does not optimize compression and tiling logic in conjunction with task type. Tile association relies on complex spatial coordinate calculations, resulting in low efficiency for cross-layer switching. At the same time, transmission priority only considers some factors and cannot accurately match core needs such as user viewports, points of interest, and flight paths. High image loading latency in key areas affects operational efficiency. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned technical problems by providing a method for real-time image transmission from unmanned aerial vehicles (UAVs) driven by both network and task requirements in complex environments.

[0005] In view of this, the present invention provides a method for real-time image transmission from a UAV driven by both network and task in complex environments, comprising the following steps: Step 1: Collect data on four key network metrics: bandwidth, latency, jitter, and packet loss rate. After filtering and denoising using an exponentially weighted moving average, the network comprehensive quality score is obtained through weighted calculation and mapped to five discrete network status levels. A lag threshold is set for level switching. Step 2: Divide the image into three layers: original image layer, standard image layer, and thumbnail layer. Set the compression rate according to the task type, determine the number of tiles in each layer according to the corresponding rules to ensure consistent tile data, and establish the association between tiles of different image layers using specific numbering rules. Step 3: Calculate the overall priority of image tiles based on four factors: current viewport, user interaction, point of interest, and flight path. Adopt a priority queueing strategy, and dynamically adjust the image layer to be transmitted, the number of concurrent threads, and the preloading strategy in combination with network status level and task type to achieve real-time image transmission.

[0006] Preferably, the key network indicator data collection method in step one is as follows: every 10 seconds during regular patrol and every 20 seconds during fine inspection, a 150KB 684*456 pixel standard image tile test file is uploaded to calculate the bandwidth; Ping packets are sent every second to obtain the average round-trip latency and standard deviation; and the packet loss rate is determined by obtaining the missing proportion of the most recent 100 data packets from the ACK mechanism.

[0007] Preferably, the smoothing factor α of the exponentially weighted moving average filter is 0.5 during normal cruise and 0.2 during fine inspection. The smoothing value is calculated as follows: current smoothing value = α * current measurement value + (1-α) * previous period smoothing value.

[0008] Preferably, the formula for calculating the overall network quality score is: NQS=(0.2B norm +0.3L norm +0.4J norm +0.1P norm )*100; Among them, B norm =min(B / 50Mbps,1.0), L norm =max(0,1-L / 200ms), J norm =max(0,1-J / 50ms), P norm =1-P.

[0009] Preferably, in step two, the image layer compression uses the Bicubic Sharper resampling algorithm, and the pixel resolution of the nth image layer is... m is the compression coefficient; for regular cruise, m=3, and for detailed inspection, m=2.

[0010] Preferably, the number of slices in each image layer is: During regular cruise, the number of slices for the original image layer, standard image layer, and thumbnail layer are 729, 81, and 9, respectively, while during fine inspection, they are 64, 16, and 4, respectively.

[0011] Preferably, the final number of the image tile is the number of the layer to which it belongs, the initial number of the tile above it, and the initial number of the tile in this layer. The initial numbering is carried out in the order from top to bottom, from left to right, and from top to bottom.

[0012] Preferably, the image tile comprehensive priority IRTP = VF + UIF + PIF + FPF, where VF has a weight of 100, UIF has a weight of 40, PIF has a weight of 40, and FPF has a weight of 20. The values ​​of each factor are calculated based on the relationship between the tile and the viewport, user operation prediction, number of points of interest, and flight path distance.

[0013] Preferably, when the network status level is excellent or good, the original image layer is transmitted first, with 8 and 5 concurrent threads respectively; when the level is average, poor, or very poor, the transmission of image layers and key areas is adjusted according to the task type, with 3, 1, and 1 concurrent threads respectively, and the preloading strategy is dynamically started and stopped according to the network status.

[0014] Preferably, when the mission type is routine patrol, the focus is on overall coverage, and the image clarity can be reduced when the network is poor; when fine inspection, the focus is on detailed presentation of key areas, and high-definition image transmission of key areas is still prioritized when the network is poor.

[0015] The beneficial effects of this invention are: through an adaptive design driven by both network and task, the reliability and real-time performance of UAV image transmission in complex environments are significantly improved. Its constructed multi-dimensional network quality assessment model can accurately capture changes in key indicators, and combined with a hysteresis threshold to avoid frequent switching of transmission strategies, along with dynamically adjusted concurrent thread counts and image layer selection, effectively solves the problem of transmission interruptions or stuttering caused by network fluctuations in the field, making image transmission more stable and smooth.

[0016] Meanwhile, the optimized strategies for image preprocessing and transmission scheduling enhance practical value: compression and slicing rules adapted to task types ensure consistent tile data volume, and specific numbering rules enable rapid cross-layer switching of image layers with different resolutions, significantly reducing computational overhead; the multi-factor priority queueing mechanism accurately matches core user needs, prioritizing the transmission of images in key areas, prediction viewports, and around flight routes, demonstrating good adaptability in diverse scenarios such as power line inspection and ecological monitoring, and achieving a dynamic balance between image clarity, transmission speed, and network resource consumption. Attached Figure Description

[0017] Figure 1 Flowchart for real-time image transmission from drones driven by both network and mission requirements; Figure 2 A schematic diagram showing the tile numbering relationship of each image layer; Figure 3 Location map of the Yi County power transmission corridor experimental area; Figure 4 This is an overview map of the power transmission corridor in the study area. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0019] A method for real-time UAV image transmission driven by both network and task in complex environments is proposed to address the difficulties in real-time image transmission caused by complex network environments in the field. The method includes the following steps: Based on four key network status indicators—bandwidth, latency, jitter, and packet loss rate—a multi-dimensional quantitative evaluation model for the real-time network status of UAVs is constructed. This model can quickly reflect the trend changes in the real-time network status of UAVs, avoid frequent switching of network transmission strategies due to instantaneous fluctuations, and minimize the occupation of network resources.

[0020] A multi-threaded, breakpoint-resume network transmission method was adopted. A mathematical relationship between image layer compression rate and the number of tiles was established. The compression and slicing methods for each image layer were determined while taking into account the task type, ensuring consistent image tile data volume. Specific image tile numbering rules were formulated to establish associations between tiles of different image layers, enabling rapid switching between aerial image layers of different resolutions and providing support for preloading strategies.

[0021] By employing a dynamic prioritization approach, this method analyzes four key factors: the current viewport, user interaction, points of interest, and flight path. It calculates the overall priority of image tiles in real time to determine their transmission order and delineates key areas for each image layer. A priority-based queuing strategy is then implemented, dynamically adjusting the queuing threshold based on network conditions and user behavior to promptly respond to user needs and enhance the user experience.

[0022] Based on real-time network quality as the core criterion, and targeting two different mission objectives—routine patrol and detailed inspection—the system automatically adjusts the network transmission strategy for UAV aerial images by dynamically adjusting image clarity, the number of concurrent threads, and preloading strategies. This achieves a balance between smooth user experience and network resource consumption in complex network environments in the field.

[0023] Real-time network status detection: Collect real-time data of key network indicators, smooth and denoise the raw data, evaluate the overall network quality according to task requirements, construct a multi-dimensional quantitative evaluation model of UAV real-time network status, and map the real-time network status level. The implementation steps are as follows.

[0024] Step 1: Collect key network metrics data. To minimize network resource consumption, based on the drone's flight speed, a standard image tile test file (image tile resolution 684*456 pixels, compressed size 150KB) is uploaded to the cloud server every 10 seconds during regular patrols (every 20 seconds during detailed inspections). The standard file upload time is calculated to determine the current available network bandwidth (B). Lightweight Ping packets are sent to the server every second, and the average round-trip time (L) and its standard deviation (J) are calculated to measure the current network latency and jitter. The statistical value of the missing percentage of the most recent 100 data packets is obtained from the network transport layer ACK mechanism to determine the current average packet loss rate (P).

[0025] Step 2: Denoising the raw network indicator data. To accurately reflect the real-time trend changes in network status, noise in the raw detection data needs to be removed. This invention uses an exponentially weighted moving average filtering algorithm to smooth the raw network status detection data. The calculation method is as follows: Current smoothed value = α * Current measured value + (1-α) * Previous period smoothed value, where α is the smoothing factor (0 < α < 1). The larger α is, the more sensitive it is to the latest changes in network status, and the greater the network fluctuation; the smaller α is, the smaller the network fluctuation, but the slower the response to the latest changes in network status. This invention determines the value of the smoothing factor α according to the type of UAV mission: when the UAV is performing routine cruise, its flight speed is relatively fast (10-15 m / s), and the network environment changes more drastically, so α is set to 0.5; when the UAV is performing fine inspection, its flight speed is relatively slow (0-5 m / s), and the network environment changes more gently, so α is set to 0.2.

[0026] Step 3: Network Overall Quality Score. Key network state indicators are normalized, mapping each indicator to the [0,1] interval, where 1 represents the optimal value. The calculation method is as follows: B norm =min(B / ideal bandwidth, 1.0), L norm =max(0, 1-L / maximum tolerable delay), J norm =max(0, 1-J / maximum tolerable jitter), P norm =1-P. Given that the majority of the UAV inspection area in this invention is located in the field, the ideal bandwidth is set at 50Mbps; referring to user internet usage statistics, the maximum tolerable latency is set at 200ms, and the maximum tolerable jitter at 50ms. Based on the application requirements for real-time UAV image transmission, the focus is on user experience, and the weights of key network status indicators are calculated and determined according to their service sensitivity, and then weighted and synthesized. In summary, the Network Quality Score (NQS) calculation method is as follows: NQS=(Wb*B norm +Wl*Lnorm +Wj*J norm +Wp*P norm )*100; Analyzing the service sensitivity of the four key network status indicators, network jitter makes it difficult for users to predict the average loading time of subsequent images, and in extreme cases, may even cause users to abandon the current task; network latency is the main cause of slow response to user commands, which will greatly affect the user's smooth experience; network bandwidth determines the current maximum network load capacity, directly affecting the loading efficiency of aerial images; and packet loss rate determines the loading quality of aerial images. In complex network environments in the field, when users view aerial images of key areas, the basic completion of the task is the minimum requirement, timely response to needs can optimize the task experience, and achieving both high efficiency and high quality is the user's ultimate expectation. Therefore, the weight order of the above four key network status indicators is W. j >W l >W b >W p Building upon the aforementioned foundation, the Analytic Hierarchy Process (AHP) is employed to further determine the relative weights of each key network status indicator. First, a hierarchical model is constructed, comprising a target layer (optimal aerial image viewing experience) and a criterion layer (four network status indicators). Then, based on an expert knowledge base, pairwise comparisons are performed on the business priorities of the key network status indicators to construct a judgment matrix. Finally, through eigenvector calculation and consistency checks, the weights of each indicator are calculated as follows: W j =0.4,W l =0.3,W b =0.2,W p =0.1.

[0027] Step 4: Mapping Network State Levels. Define a set of network quality score intervals, each corresponding to a discrete network state level. Map continuous NQS values ​​to a limited set of 5 network state levels. Set a hysteresis switching interval at the network state level switching threshold to prevent frequent changes in network state levels due to fluctuations near the critical point, thus requiring frequent switching of network transmission strategies. To improve the model's sensitivity to network state improvements, thereby increasing the network transmission efficiency of UAV aerial imagery and enhancing the user experience, this invention appropriately lowers the upgrade and downgrade thresholds for each network state interval. The network state level mapping is shown in Table 1 below.

[0028]

[0029] When the current network status level is good (NQS=65), the NQS suddenly drops to 58 due to network fluctuations. Since the degradation threshold of 50 is not reached, the network status level remains good, thus avoiding frequent switching of transmission strategies.

[0030] The current NQS is 56.15. If the network status level in the previous cycle was good (NQS ≥ 60), the network status level remains good because 56.15 > the downgrade threshold of 50. If the network status level in the previous cycle was average (NQS < 60), the network status level is upgraded to good because 56.15 > the upgrade threshold of 50. This improves the model's sensitivity to network status improvements while maintaining the stability of the transmission strategy.

[0031] Aerial image preprocessing: In order to adapt to the complex network environment in the field and to improve the transmission efficiency and reliability of the multi-threaded breakpoint resume method, it is necessary to perform preprocessing operations such as image compression, slicing, and correlation.

[0032] ① Image Compression. Based on the UAV's cruising speed, the compression ratio for each image layer is set, and the Bicubic Sharper resampling algorithm is used to directly compress the original image. Three image layers are defined: original image (layer 1), standard image (layer 2), and thumbnail (layer 3). The pixel resolution of the original image layer is a×b, and the compression ratio between each image layer is set to 1 / m. After compression, the pixel resolution of the nth layer image is... When the drone is conducting routine patrols, the image layer compression rate is set to 1 / 3. The calculated standard image layer pixel resolution is a×b / 9, and the thumbnail layer pixel resolution is a×b / 81. When the drone is conducting detailed inspections, the image layer compression rate is set to 1 / 2. The calculated standard image layer pixel resolution is a×b / 4, and the thumbnail layer pixel resolution is a×b / 16.

[0033] ② Image tiling. To improve the efficiency and reliability of UAV image network transmission, this invention employs a multi-threaded breakpoint resume network transmission method, which requires image tiling to ensure consistent data volume for each image tile. Assuming a compression rate of 1 / m between image layers, the number of image tiles to be equidistantly tiled in the nth layer is... When the drone is conducting routine patrols, the number of image tiles equidistantly cut from the original image layer is 729, the number of image tiles equidistantly cut from the standard image layer is 81, and the number of image tiles equidistantly cut from the thumbnail layer is 9, with a pixel count of ab / 729 per image tile. When the drone is conducting detailed inspections, the number of image tiles equidistantly cut from the original image layer is 64, the number of image tiles equidistantly cut from the standard image layer is 16, and the number of image tiles equidistantly cut from the thumbnail layer is 4, with a pixel count of ab / 64 per image tile.

[0034] ③ Image Tile Association. To achieve rapid switching between image layers of different resolutions, the nth image tile needs to quickly locate the (n-1)th image tile and the image tiles it comprises, and can also quickly locate the (n+1)th image tile. This invention uses specific image tile numbering rules to establish associations between tiles of different image layers.

[0035] The image tiles are numbered sequentially from the top layer (third layer) to the bottom layer (first layer). Based on the equidistant cutting order, a specific image tile on the top layer is initially numbered from left to right and from top to bottom; since the upper layer image tiles and the corresponding lower layer tiles are m... 2 (m is the compression factor) The geographic areas covered by the image tiles are consistent. Based on the equidistant cutting order, the lower-layer image tiles are initially numbered from left to right and from top to bottom; this process is repeated until the initial numbering of all lower-layer image tiles is complete. The above steps are repeated until the initial numbering of all layer image tiles is completed.

[0036] The final image tile number is the sum of the layer number, the initial number of the upper-layer tile, and the initial number of the tile in this layer. This number retains the image tile number of the upper layer and also determines the m-values ​​it comprises. 2 The image tile numbers of the lower layers. For example, assuming the image layer compression ratio is 1 / 2, the image tile numbered 34 in the third layer corresponds to the tile numbers 241, 242, 243, and 244 in the second layer, and the tile numbers 1411, 1412, 1413, and 1414 in the first layer; 1421, 1422, 1423, and 1424; 1431, 1432, 1433, and 1434; and 1441, 1442, 1443, and 1444. Figure 2 As shown.

[0037] Traditional image switching methods based on spatial coordinates or quadtree indexes require dynamically calculating and querying the row and column numbers of the corresponding tiles at the target layer each time an image is switched, based on the geographic coordinate range of the current viewport. This is particularly problematic during large-scale cross-layer switching, where the number of image tiles requiring one-time location increases exponentially, resulting in significant accumulated computation and query time overhead. The image tile switching method proposed in this invention establishes a directly derived algebraic mapping relationship through tile numbers. It only requires simple numerical parsing of the tile numbers, without involving geographic coordinate calculations or index queries, thus converting traditional spatial coordinate calculations into basic arithmetic operations, greatly reducing computation and query time overhead. Comparative tests with traditional image switching methods show that when switching from layer 3 to layer 2 (involving the location of 4 tiles), the quadtree index-based method takes an average of approximately 3.2ms, while the tile number-based location method of this invention takes an average of approximately 0.15ms. Especially in scenarios involving large-scale cross-layer switching, the performance improvement of this invention is more significant than that of traditional methods, with an average reduction in accumulated time overhead exceeding 80%.

[0038] Tile transmission scheduling optimization: This invention simplifies UAV aerial imagery, which has a wide coverage, high resolution, and large data volume, into image tiles with consistent data volume. It also uses a comprehensive priority to measure the importance of image tiles in the same layer and proposes a network-adaptive optimal image tile transmission strategy.

[0039] ① Determine transmission priority. Based on the concept of dynamic priority, a priority-based queuing strategy is adopted to calculate the overall priority of image tiles in real time to determine the image tile transmission sequence. When the viewport changes, user interaction occurs, or the flight path approaches, the IRTP value of the relevant image tiles will increase sharply. The algorithm dynamically adjusts the queuing threshold according to the real-time network conditions and user behavior. High-priority tiles can interrupt the transmission of low-priority tiles or directly replace low-priority items in the queue.

[0040] The transmission priority of image tiles depends on four main factors: current viewport, user interaction, points of interest, and flight path. Current viewport: Determines whether the image tile is located within the current terminal screen viewport to ensure the smoothness of the area currently seen by the user. Its weight value is the highest (100). User interaction: Based on the user's recent panning and zooming trajectory, predicts the area that the subsequent viewport may switch to, thereby determining the range of image tiles that need to be preloaded. Its weight value is high (40). Points of interest: Whether the image tile contains user-set points of interest or key targets identified by content analysis (such as bird nests, kites, etc.) to ensure high-quality presentation of key information areas. Its weight value is high (40). Flight path: The distance of the image tile from the UAV's planned mission flight path to ensure the preloading of images in the mission area of ​​the flight path. Its weight value is medium (20). The IRTP calculation method of image tiles is as follows: IRTP=VF+UIF+PIF+FPF, where the calculation steps of each factor are as follows: Step 1: Calculate the current viewport factor. Determine the containment relationship between the image tile and the current viewport. If the tile is completely within the viewport area, the VF value is 100; if the tile intersects with the edge of the viewport, the VF value is 80; if the tile is within 3 tiles of the viewport buffer, the VF value is 30; otherwise, the VF value is 0.

[0041] Step 2: Calculate the user interaction factor. Record the user's three most recent mouse translation vectors and zoom center point operations. Use the velocity vector extrapolation algorithm to calculate the range, center point, and radial distance (r) of the area the user might browse next (predicted viewport). Calculate the Euclidean distance (d1) from the center of the image tile to the center of the predicted viewport. If d1 ≤ r, UIF = 40; if d1 > r, UIF = 25r / d1.

[0042] Step 3: Calculate the point of interest factor. PIF=min(20n,40), where n is the number of points of interest or key targets contained in the image tile.

[0043] Step 4: Calculate the flight route factor. Calculate the shortest distance (d2) from the center of the image tile to the center line of the route. If d2 ≤ 15, FPF = 20; if 15 < d2 ≤ 50, FPF = 10; if d2 > 50, FPF = 5.

[0044] ② Network adaptive transmission strategy. The present invention takes the real-time network quality as the core judgment basis, and automatically adjusts the network transmission strategy of the UAV aerial images for two different task objectives of regular cruise and fine inspection. By dynamically adjusting the image clarity, the number of concurrent threads, and the preloading strategy, it achieves a balance between the smooth user experience and the consumption of network resources in the complex field network environment.

[0045] Focus on the primary task objective. The regular cruise task focuses on achieving the global coverage of the cruise area, and low-clarity images can be accepted when the network is poor; the fine inspection focuses on achieving the fine presentation of key areas, and the image clarity of key areas is prioritized even when the network quality is poor.

[0046] Dynamically optimize network resources. The better the network quality, the more fully the resources are called. By increasing the resolution of the image layer and the maximum number of concurrent threads, the loading speed is全力提升加载速度. The worse the network quality, the more conservative the transmission strategy. By reducing the image clarity and the number of concurrent threads until the multi-threading is disabled, priority is given to ensuring the continuous transmission and the uninterruption of basic services. When the network quality is average, preloading is triggered, and high-definition images of key areas are preloaded in advance to balance the immediate experience and the bandwidth pressure. The breakpoint resumption transmission method is used to prevent repeated transmission due to network interruption and improve the network transmission efficiency. The network adaptive transmission strategy aims to maximize the user experience while completing basic services, as shown in the following table (Table 2).

[0047]

[0048] For example, when performing a regular cruise task, the network status at the current position of the UAV is average. At this time, the number of concurrent threads for breakpoint resumption transmission is 3, with 2 threads used to transmit the second-layer standard images and the other 1 thread used to preload the first-layer original images (spare) of key areas. If the task type is switched to fine inspection and the network status is average, 2 threads are used to transmit the first-layer original images of key areas, and the other 1 thread is used to preload the first-layer original images (spare) of other areas; when the network status is poor, if the clarity of the second-layer standard images can meet the user's needs, there is no need to load the first-layer original images of key areas, otherwise continue to load; when the network status is extremely poor, if the clarity of the second-layer standard images of key areas can meet the user's needs, there is no need to load the first-layer original images of key areas, otherwise continue to load.

[0049] To verify the actual effect of the present invention in a complex network environment, the central and southern regions of Yi County, Baoding City, Hebei Province ( Figure 3This area serves as the experimental zone. It is a typical low-mountain and hilly area, with an elevation ranging from 80 to 450 meters and significant topographic relief. The experimental transmission line is approximately 10 km long, traversing a mountain buffer zone with over 70% vegetation coverage. Common vegetation types within the area include trees, shrubs, grasslands, and some farmland. Based on historical data, this region experiences hot and windy summers and dry winters, making it prone to forest fires. Therefore, strengthened patrols and monitoring are necessary. Figure 4 As shown.

[0050] To accurately reflect the actual working conditions of the power inspection department, the weather was sunny with a wind speed of level 3 and a temperature of 18-25℃ on the day of the test. All tests were conducted between 10:00 and 14:00. The network environment of the transmission corridor in the experimental section was primarily 4G, with 5G network coverage in the residential areas (Dalonghua Township, Yaoshunkou Village, and Yanggezhuang Village), while the network signal was weak and fluctuated significantly in the mountainous areas (Tuanshan and Dazhaishan). A total of 5 test points were set up (1 near residential areas, 2 in mountainous areas, and 2 in other areas), corresponding to the aforementioned 5 network status levels. The experiment used a DJI Matrice 300 RTK drone equipped with a wireless network detection module (built-in 4G / 5G multimodal), and the ground station was free from interference from other high-load processes.

[0051] The drone's flight and hovering altitude were uniformly set to 100m (relative altitude). It flew along a preset flight path, hovering for 30 seconds at each test point to collect real-time data on network bandwidth, latency, jitter, and packet loss rate. High-definition imagery covering a 100m radius on both sides of the power transmission line's centerline was captured. Ground operators performed a standard operating sequence (panning the viewport, zooming to maximum resolution, and marking points of interest), panning the viewport at a constant speed along the flight path, moving approximately 1 / 3 of the screen width per second (panning test); a "zoom-pan-zoom" loop was performed at fixed points to simulate typical browsing behavior (zooming test); and a combination of panning, zooming, and marking points of interest simulated real-world inspection work (mixed operation). The interaction response time was defined as the time interval from the completion of the user's operation to the complete presentation of the image in the corresponding new viewport. At each data collection point, each type of interaction was repeated 20 times. The system automatically recorded the duration of each interaction response, and the average value was used as the final test data for that test point. All network quality scores acquired at that test point within the test duration were recorded, and the average value was used as the final network quality score for that test point. To test the overall effectiveness of the strategy of this invention, control group 1 adopted a fixed strategy for transmission, without using network state awareness and adaptive mechanisms, without classifying task types, selecting a fixed slice size (512×512 pixels) for the original aerial image, and prioritizing scheduling was only associated with the current user's viewport. The above test process was repeated. The experimental results are shown in Table 3 below.

[0052]

[0053] To test the image loading optimization of key areas in this invention, control group 2 selected a fixed image slice size (512×512 pixels) for the original aerial image, without distinguishing the transmission priority of the image slices. The remaining strategy selection and parameter settings were the same as those in experimental group 1. The average loading time of aerial images within a 15-meter radius around the center flight path, the current viewport switched with the user, and the area of ​​interest was recorded and calculated, and compared with the corresponding data in experimental group 1. The experimental results are shown in Table 4 below.

[0054]

[0055] Analysis of the experimental data shows that, compared with the fixed strategy transmission method, the present invention reduces the overall loading latency of UAV aerial imagery by about 30% in complex network environments; especially in weak network conditions such as mountainous and forest areas, it reduces the loading latency of UAV aerial imagery in key task areas by more than 50% and improves the user demand response efficiency by about 60%, demonstrating significant engineering practical value.

[0056] To simulate routine resource monitoring tasks in the forest areas of Northeast my country, this invention selected the high-altitude Yanshan Mountains in the border region between Beijing and Hebei in northern China. This area has an average altitude of approximately 800 meters, a relative elevation difference exceeding 400 meters, and a forest coverage rate greater than 85%. Due to the complex terrain, network signals in the test area are easily obstructed, resulting in unstable network connections, large network fluctuations, and high packet loss rates, significantly impacting the acquisition of high-definition, complete canopy images and pest and disease monitoring data during forestry patrols. This invention identifies channel quality through real-time network status sensing and employs a task-oriented priority transmission scheduling mechanism to prioritize the transmission of limited and unreliable bandwidth resources for key forest area images. Test results show that this invention can improve the image loading success rate in key forest areas from 42% to 61% and reduce the average latency by 64%, indicating that this invention also has good adaptability in ecological monitoring scenarios.

[0057] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for real-time image transmission from a UAV driven by both network and task in complex environments, characterized in that: Includes the following steps: Step 1: Collect data on four network metrics: bandwidth, latency, jitter, and packet loss rate. After filtering and denoising using an exponentially weighted moving average, the network comprehensive quality score is obtained through weighted calculation and mapped to five discrete network status levels. A lag threshold is set for level switching. Step 2: Divide the image into three layers: original image layer, standard image layer, and thumbnail layer. Set the compression rate according to the task type, determine the number of tiles in each layer according to the corresponding rules to ensure consistent tile data, and establish the association between tiles of different image layers using specific numbering rules. Step 3: Calculate the overall priority of image tiles based on four factors: current viewport, user interaction, point of interest, and flight path. Adopt a priority queueing strategy, and dynamically adjust the image layer to be transmitted, the number of concurrent threads, and the preloading strategy in combination with network status level and task type to achieve real-time image transmission.

2. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 1, is characterized in that: The key network indicator data collection method in step one is as follows: every 10 seconds during regular patrol and every 20 seconds during fine inspection, a 150KB 684*456 pixel standard image tile test file is uploaded to calculate the bandwidth. Ping packets are sent every second to obtain the average round-trip latency and standard deviation. The packet loss rate is determined by obtaining the missing proportion of the most recent 100 data packets from the ACK mechanism.

3. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 2, is characterized in that: The smoothing factor α of the exponentially weighted moving average filter is 0.5 during normal cruise and 0.2 during fine inspection. The smoothing value is calculated as follows: Current smoothing value = α * current measurement value + (1-α) * previous period smoothing value.

4. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 2, is characterized in that: The formula for calculating the overall network quality score is as follows: NQS=(0.2B norm +0.3L norm +0.4J norm +0.1P norm )*100; Among them, B norm =min(B / 50Mbps,1.0), L norm =max(0,1-L / 200ms), J norm =max(0,1-J / 50ms), Pnorm=1-P.

5. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 1, is characterized in that: In step two, the image layer compression uses the Bicubic Sharper resampling algorithm, and the pixel resolution of the nth image layer is... m is the compression coefficient; for regular cruise, m=3, and for detailed inspection, m=2.

6. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 5, is characterized in that: The number of slices in each image layer is During regular cruise, the number of slices for the original image layer, standard image layer, and thumbnail layer are 729, 81, and 9, respectively, while during fine inspection, they are 64, 16, and 4, respectively.

7. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 5, is characterized in that: The final number of the image tile is the number of the layer to which it belongs, the initial number of the tile above it, and the initial number of the tile in this layer. The initial numbering is carried out in the order from top to bottom, from left to right, and from top to bottom.

8. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 1, is characterized in that: Image tile comprehensive priority IRTP = VF + UIF + PIF + FPF, where VF has a weight of 100, UIF has a weight of 40, PIF has a weight of 40, and FPF has a weight of 20. The values ​​of each factor are calculated based on the relationship between the tile and the viewport, user operation prediction, number of points of interest, and flight path distance.

9. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 1, is characterized in that: When the network status is excellent or good, the original image layer is transmitted first, with 8 and 5 concurrent threads respectively; when the status is average, poor, or very poor, the transmission of image layers and key areas is adjusted according to the task type, with 3, 1, and 1 concurrent threads respectively, and the preloading strategy is dynamically started and stopped according to the network status.

10. The method for real-time image transmission from a UAV driven by both network and task in a complex environment, as described in claim 1, is characterized in that: When the mission type is routine patrol, the focus is on overall coverage, and the image clarity may be reduced when the network is poor; when the mission type is detailed inspection, the focus is on detailed presentation of key areas, and high-definition image transmission of key areas is still prioritized when the network is poor.

Images