A perception-driven airborne lidar data adaptive transmission method and cooperative control system

Abstract

This application provides a perception-driven adaptive transmission method and cooperative control system for airborne LiDAR data. The method includes: each airborne terminal performing target detection on point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency; calculating the scene urgency of the current frame based on the target detection results and uploading it to a ground terminal; multiple airborne terminals sharing the same wireless link via a star network; the ground terminal allocating bandwidth to each airborne terminal based on the scene urgency reported by each airborne terminal and notifying each airborne terminal; each airborne terminal transmitting the target detection results and point cloud data on the wireless link through different ports, with the target detection results transmitted through a high-priority port and the point cloud data transmitted through a low-priority port. The transmission rate of the point cloud data is constrained by the bandwidth allocated to the current frame, and under this bandwidth constraint, the transmission accuracy is adaptively selected based on the scene urgency and link quality for transmission.

Description

Technical Field

[0001] This invention relates to the field of remote sensing control technology, specifically to a perception-driven adaptive data transmission method and cooperative control system for airborne lidar. Background Technology

[0002] Unmanned aerial vehicle (UAV)-borne lidar systems, with their high-precision, all-weather 3D environmental perception capabilities, have been widely used in surveying, security, and inspection. In a typical operational mode, the UAV transmits the raw point cloud data collected by the lidar back to the ground station via a wireless link, where a high-performance computing platform performs target detection and scene analysis. However, with the increasing demands for real-time performance, existing technologies are gradually revealing the following bottlenecks:

[0003] Full point cloud transmission leads to severe transmission delays and data loss. Modern LiDAR can generate millions of points per second, with data volumes reaching tens to hundreds of megabits. In long-distance or complex electromagnetic environments, wireless link bandwidth often cannot support such high-speed data streams. This results in two consequences: either frame dropping is initiated due to insufficient bandwidth, or excessive transmission delays cause the perception information acquired by ground operators to lag significantly behind the actual location of the UAV. Ground personnel often have to wait for the entire frame or several seconds of point cloud data to be fully received before they can start the analysis process. This "collect first, process later" approach is completely unable to meet the needs of real-time perception.

[0004] Migrating the detection task to an airborne terminal is a direct approach to solving transmission latency. However, relevant 3D point cloud detection models, such as algorithms based on sparse convolution, heavily rely on sparse convolution operators. However, limited by power consumption, size, and cost, edge computing platforms on UAVs typically only support standard dense convolution operators. Their dedicated inference chips lack efficient hardware acceleration support for sparse convolution, preventing the direct deployment of mainstream high-precision detection models. Airborne terminals often can only run simple rule-based filtering or traditional machine learning methods, resulting in detection accuracy and robustness far lower than ground-based terminals.

[0005] Furthermore, when multiple drones collaborate by sharing the same wireless channel, they compete for limited link bandwidth. Existing systems typically employ fixed allocation or passive adjustment strategies based on channel quality, completely ignoring the differences in the urgency of each drone's current sensing task. For example, one drone may have detected a high-value moving target and needs to transmit critical data back at high frequency and high resolution, while other drones are only performing broad-area inspections. When bandwidth is congested, information about urgent targets may be delayed or even lost due to fair competition with other non-urgent traffic, directly impacting the operational effectiveness of multi-drone collaboration.

[0006] In summary, the relevant technologies face technical challenges in collaborative sensing tasks involving multiple UAV-borne LiDAR systems, including the inability of the airborne end to detect in real time, low utilization of wireless transmission bandwidth, and a lack of intelligent scheduling for multi-UAV collaboration. Summary of the Invention

[0007] In view of this, this application proposes a perception-driven adaptive data transmission method and cooperative control system for airborne lidar.

[0008] Specifically, this application is implemented through the following technical solution:

[0009] According to a first aspect of the embodiments of this specification, a perception-driven adaptive data transmission method for airborne lidar is provided, comprising the following steps:

[0010] Step S1: Each airborne terminal performs target detection on the point cloud data collected by its airborne LiDAR based on the point cloud detection model without sparse convolution dependency, calculates the scene urgency of the current frame based on the target detection results, and uploads it to the ground terminal; wherein, multiple airborne terminals share the same wireless link through a star-shaped network; in the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling on the convolution output according to the ratio of the effective number of pixels in the receptive field to the area of ​​the convolution kernel;

[0011] Step S2: The ground terminal allocates bandwidth to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and notifies each airborne terminal.

[0012] In step S3, each airborne terminal sends target detection results and point cloud data on the wireless link through different ports. The target detection results are sent through high-priority ports, and the point cloud data is sent through low-priority ports. The transmission rate of the point cloud data is constrained by the bandwidth allocated to the current frame. Under this bandwidth constraint, the transmission accuracy is adaptively selected for transmission based on the urgency of the scenario and the link quality.

[0013] According to a second aspect of the embodiments of this specification, a cooperative control system is provided, including a ground terminal and multiple airborne terminals, wherein the multiple airborne terminals share the same wireless link through a star network and communicate with the ground terminal through the wireless link; wherein:

[0014] On the airborne end, a target detection method is used to perform target detection on point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency. The method calculates the scene urgency of the current frame based on the target detection results and uploads it to the ground end. In the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling on the convolution output according to the ratio of the effective number of pixels in the receptive field to the area of ​​the convolution kernel. The method also transmits the target detection results and point cloud data on the wireless link through different ports. The target detection results are transmitted through high-priority ports, and the point cloud data is transmitted through low-priority ports. The transmission rate of the point cloud data is constrained by the bandwidth allocated to the current frame, and under this bandwidth constraint, the transmission accuracy is adaptively selected based on the scene urgency and link quality.

[0015] The ground-based system allocates bandwidth to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and notifies each airborne terminal.

[0016] The embodiments of this application have at least the following technical effects:

[0017] First, the embodiments of this application realize the forward shift of point cloud target detection from the ground end to the airborne end by deploying a point cloud detection model without sparse convolution dependency on the airborne end. The point cloud detection model uses only standard two-dimensional convolution operators throughout the entire link, does not depend on sparse convolution operator library, can be deployed on general edge inference chips that only support standard operators, is not limited by specific GPU platforms or specific manufacturers' NPUs, and has the widest edge chip compatibility.

[0018] Second, in the adaptive transmission architecture of this application embodiment, the transmission priority and bandwidth allocation are not manually preset or passively adjusted according to the link quality, but are driven in real time by the inference results of the point cloud detection model. The target detection results output by the point cloud detection model are used to adjust the sending rate after the scene urgency is calculated, forming a closed loop from perception to transmission, so that the transmission resources are dynamically allocated according to the scene semantics; and the target detection results are always sent with the highest priority, so that real-time delivery can still be guaranteed even when the link bandwidth is severely insufficient.

[0019] Third, in the multi-machine bandwidth scheduling mechanism of this application embodiment, the bandwidth allocation weight of each airborne terminal comes from the perception results of its own detection model rather than the link-side indicators, so that the airborne terminal that discovers high-value targets automatically obtains more bandwidth and prioritizes the transmission of key information. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 This is a schematic diagram of the system architecture of a collaborative control system shown in an exemplary embodiment of this application;

[0022] Figure 2 This is a schematic flowchart illustrating a perception-driven adaptive data transmission method for airborne lidar, as shown in an exemplary embodiment of this application.

[0023] Figure 3 This is a schematic diagram illustrating the transmission process of lidar data based on a priority strategy, as shown in an exemplary embodiment of this application.

[0024] Figure 4 This is a schematic diagram of a multi-GPU fusion processing pipeline architecture shown in an exemplary embodiment of this application;

[0025] Figure 5 This is a structural block diagram of a cooperative control system illustrated in an exemplary embodiment of this application. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] Figure 1 This is a schematic diagram of the system architecture of a cooperative control system shown in an exemplary embodiment of this application, such as... Figure 1As shown, the collaborative control system of this application embodiment includes three parts: an airborne terminal, a wireless link, and a ground terminal. The airborne terminal in the system shares the wireless link through a star network to communicate with the ground terminal. The edge computing module of the airborne terminal runs a lightweight point cloud detection model without sparse convolution dependency, performs real-time target detection on the point cloud data collected by LiDAR, calculates the scene urgency based on the target detection results, and adaptively transmits the point cloud data. The ground terminal allocates transmission rates proportionally according to the scene urgency reported by each airborne terminal, and configures independent GPU fusion processing pipelines for the point cloud data streams of each airborne terminal. Raw byte parsing and coordinate transformation are completed within the same thread of the GPU computing shader. When motion detection is enabled, background modeling is also performed within the same thread immediately after coordinate transformation. Multiple processing results are overlaid and rendered in a unified 3D scene.

[0029] The embodiments described in this specification will now be described in detail.

[0030] This application provides a perception-driven adaptive data transmission method for airborne lidar. Figure 2 This is a schematic flowchart illustrating a perception-driven adaptive data transmission method for airborne lidar, as shown in an exemplary embodiment of this application. Figure 2 As shown, the airborne lidar data adaptive transmission method includes at least the following steps:

[0031] Step S1: Each airborne terminal performs target detection on the point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency, calculates the scene urgency of the current frame based on the target detection results, and uploads it to the ground terminal; wherein, multiple airborne terminals share the same wireless link through a star-shaped network; in the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling of the convolution output according to the ratio of the number of effective pixels in the receptive field to the area of ​​the convolution kernel.

[0032] Each airborne edge computing module runs a lightweight point cloud detection model without sparse convolution dependency to perform real-time target detection on the raw point cloud data collected by the airborne LiDAR. The detection process includes the following steps:

[0033] First, the original point cloud is encoded into columns by grouping the point cloud according to vertical columns. A dynamic feature extraction network is then used to obtain the feature vector for each column. Next, the column features are mapped to a bird's-eye view using a scattering operation. Figure 2A bird's-eye view feature map is generated on a 3D grid. Next, a multi-scale 2D convolutional backbone network is used to extract the bird's-eye view features. The first two stages of the backbone network use density-aware convolution to process the sparse bird's-eye view feature map, scaling the convolution output according to the ratio of the number of effective pixels in the receptive field to the area of ​​the convolution kernel, with the scaling factor always less than or equal to 1 to eliminate the contamination of effective features by zero-value regions; subsequent stages use standard 2D convolution. The entire link uses only standard 2D convolution operators, does not rely on a sparse convolution operator library, and is compatible with the standard operator set of edge inference chips. Finally, multi-scale features are fused through a feature pyramid, and the detection head outputs the category, 3D bounding box, orientation, and confidence score for each target.

[0034] The point cloud detection model is exported using the standard model exchange format and then quantized into a mixed-precision model via the edge chip toolchain for deployment.

[0035] It is worth noting that the standard two-dimensional convolution described in this application embodiment refers to the convolution kernel performing sliding calculations at all spatial locations of the input feature map without skipping or selectively calculating any location. Its calculation process does not rely on sparse index management, custom kernel scheduling, or dedicated sparse computation libraries. It is a regular convolution operation natively supported by deep learning frameworks (such as PyTorch and TensorFlow) and is also a standard operator natively supported by the instruction set of edge inference chips.

[0036] Step S2: The ground terminal allocates a transmission rate to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and notifies each airborne terminal.

[0037] In step S3, each airborne terminal transmits target detection results and point cloud data on the wireless link through different ports. The target detection results are transmitted through high-priority ports, and the point cloud data is transmitted through low-priority ports. Under the constraint of the transmission rate, the transmission accuracy of the point cloud data is adaptively selected for transmission based on the urgency of the scenario and the link quality.

[0038] In some embodiments, step S1 calculates the scene urgency index of the current frame through the following steps:

[0039] The original urgency is obtained by weighting and summing the values ​​of each detected target according to its category weight, detection confidence, and distance decay factor.

[0040] Based on a preset smoothing coefficient and the scene urgency index of the previous frame, the original urgency is smoothed inter-frame to obtain the scene urgency index of the current frame.

[0041] In some embodiments, step S3, which adaptively selects the transmission precision for transmission based on the urgency of the scenario and the link quality, includes:

[0042] Determine the expected content to be transmitted for point cloud data based on the scenario urgency index and link quality.

[0043] Based on the transmission rate corresponding to the allocated bandwidth of the current frame, a transmission decision is made for the expected transmission content;

[0044] Adaptive transmission of point cloud data based on decision results.

[0045] In some possible implementations of this embodiment, the expected transmission content of the point cloud data is determined based on the scenario urgency index and link quality, including:

[0046] When the urgency index of the scene is higher than the preset threshold and the link quality is good, the expected content to be transmitted is a point cloud of the region of interest centered on the target.

[0047] When the urgency index of the scenario is higher than the preset threshold and the link quality is poor, the expected content to be transmitted is not to send point clouds;

[0048] When the urgency index of the scenario is lower than the preset threshold and the link quality is good, the expected content to be transmitted is the downsampled full point cloud.

[0049] When the urgency index of the scenario is lower than the preset threshold and the link quality is poor, the expected content to be transmitted is not to send point clouds;

[0050] Link quality is obtained by sending sequenced heartbeat packets from each airborne terminal to the ground terminal at regular intervals, and the ground terminal calculates the packet loss rate and sends it back.

[0051] In some possible implementations of this embodiment, making a transmission decision on the desired content includes:

[0052] If the desired transmission content is not to send point cloud data, then the decision is not to send point cloud data.

[0053] If the desired content to be transmitted is a region of interest (ROI) point cloud, the decision is as follows: if the transmission rate of the current frame is sufficient to transmit the ROI point cloud, then upload the ROI point cloud; if the transmission rate of the current frame is only sufficient to transmit the downsampled full point cloud, then upload the downsampled full point cloud; if the transmission rate of the current frame is insufficient to transmit the downsampled full point cloud, then do not upload the point cloud data.

[0054] If the expected content to be transmitted is a downsampled full point cloud, the decision is as follows: if the transmission rate of the current frame is sufficient to transmit the downsampled full point cloud, then upload the downsampled full point cloud; otherwise, do not upload the point cloud data.

[0055] Research has found that ground-based visualization systems cannot support real-time, unified rendering of multi-channel point clouds in relevant technical solutions. Most existing ground-based point cloud visualization software is designed for single-channel data streams. When receiving data from multiple drones, operators must frequently switch between multiple independent windows, failing to obtain a global situational awareness that integrates multiple drone perspectives within the same 3D scene. Furthermore, traditional software uses the CPU for point-by-point coordinate transformation, shading, and rendering pipeline data transmission, which can introduce latency of tens of milliseconds when processing a single frame of millions of point clouds. In multi-channel concurrency scenarios, each frame of data undergoes multiple memory copies and intermediate object creations during its reception, parsing, coordinate transformation, and final delivery to the graphics processor. These additional data handling overheads are amplified exponentially, leading to a sharp drop in rendering frame rate, screen stuttering, and an inability to meet the demands of real-time monitoring.

[0056] Based on this Figure 2 The method shown also includes the following steps:

[0057] Step S4: The ground end configures independent GPU fusion processing pipelines for the point cloud data streams of each airborne end, and overlays and renders the processing results of each GPU fusion processing pipeline in a unified 3D scene. Among them, each GPU fusion processing pipeline writes the pixel bytes carried by the point cloud data packets uploaded by the corresponding airborne end into the GPU shared buffer. Each GPU fusion processing pipeline starts the GPU computing shader. Each thread of the GPU computing shader reads the corresponding pixel bytes from the GPU shared buffer and performs the following operations: extracts the distance value and reflection intensity through displacement mask, converts the distance value into 3D coordinates according to the optical model, and outputs 3D vertices containing 3D coordinates and reflection intensity.

[0058] The following is combined with Figure 3 and Figure 4 This application details the adaptive data transmission process of the airborne lidar.

[0059] like Figure 3 As shown, each airborne terminal performs real-time detection of the point cloud data collected by the LiDAR on the airborne terminal based on the locally deployed point cloud detection model. The specific detection process can be referred to the relevant embodiments in the previous text, and will not be repeated here.

[0060] Each airborne terminal calculates the scene urgency of the current frame based on the target detection results and smooths out sudden fluctuations through a time decay mechanism.

[0061] Specifically, the original urgency of the current frame is calculated using the following formula:

[0062] (1)

[0063] In formula (1), Indicates the original urgency of the current frame. This indicates the total number of targets detected in the current frame. Indicates the first The class weights of each detected target Indicates the first The detection confidence of each target. This represents the distance decay factor; the category weights can be preset according to the importance of the target, for example, 5.0 for people, 3.0 for vehicles, 4.0 for two-wheeled vehicles, and 1.0 for other categories.

[0064] The formula for calculating the distance attenuation factor is as follows:

[0065] (2)

[0066] In formula (2), This represents the horizontal distance from the k-th detected target to the airborne lidar sensor. This indicates the preset reference distance, which can be optionally set to 50 meters.

[0067] To prevent drastic changes in urgency due to false detections in a single frame, this embodiment also smooths the original urgency using an exponential moving average:

[0068] (3)

[0069] In formula (3), This indicates the urgency level of the scene after smoothing. This represents the smoothing coefficient, which can optionally be set to 0.3. This indicates the urgency of smoothing in the previous frame.

[0070] After calculating the scene urgency of the current frame, each airborne terminal uploads the scene urgency of the current frame to the ground terminal.

[0071] The ground control calculates bandwidth allocation weights based on the urgency level reported by each machine:

[0072] (4)

[0073] In formula (4), Indicates the first Bandwidth allocation weights at the airborne end. Indicates the first The urgency level of the scenario reported by the airborne terminal. This indicates the number of currently connected airborne terminals. Indicates the basic guaranteed weight, which is optional. This ensures that even drones with zero urgency can obtain the minimum bandwidth for heartbeat and status reporting.

[0074] The ground unit notifies each device of its allocated transmission rate via the downlink:

[0075] (5)

[0076] In formula (5), Indicates assignment to the first The transmission rate of the airborne terminal, This indicates the total available bandwidth, which refers to the remaining bandwidth after deducting the overhead of heartbeats and detection results.

[0077] After receiving the transmission rate allocated to them by the ground terminal, each airborne terminal performs adaptive data transmission.

[0078] This embodiment adopts a logical priority separation architecture on the same data transmission link, distinguishes data types through different UDP ports, and uses sender rate control to ensure low latency for high-priority data.

[0079] In this embodiment, target detection results are sent through a high-priority port, ensuring continuous transmission. Each frame contains one data packet, including the frame number, urgency level, number of targets, and detection information for all targets. The data size of a single frame is approximately 4 kilobytes. Target detection result packets are marked as the highest priority in the transmission queue, taking precedence over point cloud packets and are not subject to point cloud data flow control restrictions.

[0080] Point cloud data is transmitted through low-priority ports. Under the constraint of transmission rate, the content to be transmitted is dynamically selected based on the urgency of the scenario and link quality. Link quality is obtained by the airborne end periodically sending sequenced heartbeat packets to the ground end, and the ground end calculates the packet loss rate and sends it back.

[0081] The decision logic for the expected transmitted content is shown in Table 1:

[0082]

[0083] Table 1

[0084] In practical applications, the threshold for scenario urgency can be set as needed. The link quality can be compared with the set threshold, and the link quality can be determined as good or bad based on the comparison result. Those skilled in the art can refer to relevant technical solutions regarding link quality assessment; this embodiment does not limit this approach.

[0085] In this embodiment, each airborne terminal maintains a token bucket queue. Target detection result packets are sent directly without token restrictions, while point cloud data packets are sent at the token rate. When link bandwidth is tight, point cloud packets are automatically rate-limited, but detection result packets are always guaranteed to be sent.

[0086] Specifically, each machine updates the token bucket parameters according to the allocated rate and dynamically adjusts the amount of point cloud data sent: when the allocated rate is sufficient to transmit the point cloud of the region of interest, the point cloud of that region is transmitted; when it is only sufficient to transmit the downsampled point cloud, the downsampled point cloud is transmitted; when the rate is insufficient, the point cloud is not transmitted, and only the detection results are guaranteed.

[0087] It is worth noting that the raw point cloud data collected by each airborne terminal is always cached in the local storage of that airborne terminal so that it can be exported offline after the airborne terminal lands.

[0088] The ground terminal in this embodiment also has a multi-GPU real-time visualization function.

[0089] like Figure 4 As shown, a visualization system is deployed on the ground. This visualization system is responsible for simultaneously receiving, processing, and displaying point cloud data and detection results from multiple UAVs. It uses a GPU fusion processing architecture to achieve real-time 3D visualization of multi-path point clouds.

[0090] The ground-based system allocates independent data monitoring port groups to each airborne terminal, allowing each airborne terminal to process its point cloud data stream independently without interference. Each point cloud data stream maintains an independent data receiving controller, detection result parser, and 3D scene coordinate transformation module.

[0091] For example, each point cloud data stream is configured with an independent GPU fusion processing pipeline instance. This pipeline completes raw byte parsing and 3D coordinate transformation within the same thread of the GPU compute shader, and also performs background modeling within the same thread when motion detection is enabled. The specific process is as follows:

[0092] After the original point cloud data packet is verified by packet header and footer markings, the pixel byte data is directly written to the GPU shared buffer in a zero-copy manner without CPU-side pixel-by-pixel parsing. At the same time, a metadata index containing column angle, byte offset and timestamp is constructed.

[0093] The GPU compute shader operates on a thread mesh size based on the total number of pixels, with each thread processing one pixel. Within the same thread, the following operations are performed consecutively: reading raw byte data from a shared buffer and extracting fields such as distance and reflection intensity using a displacement mask; converting the distance value into 3D coordinates based on the optical model; and outputting the 3D vertex. When motion detection is enabled, background modeling is also performed immediately after the coordinate transformation within the same thread. This involves reading the background state of the pixel location, which is persisted across frames in GPU memory, comparing it with the current distance value to determine motion, updating the background state, and outputting a motion marker.

[0094] When motion detection is enabled, a spatial morphological filtering compute shader is also encoded in the same GPU command buffer to perform 8-neighborhood voting filtering on motion markers, suppressing isolated noise points and filling holes in motion regions.

[0095] Each pipeline maintains two sets of GPU buffers, using semaphore protection to ensure pipelined parallelism between CPU data filling and GPU computation. When motion detection is enabled, the background model buffer is persisted across frames and does not participate in the rotation.

[0096] The GPU processing results for all routes are overlaid and rendered in a unified 3D scene: Each airborne terminal obtains its own position and attitude information through the onboard GPS / IMU integrated navigation module, and attaches the pose data of the current frame to the detection result packet. The point clouds of each route on the ground are transformed to a unified world coordinate system according to the position and attitude information reported by the corresponding UAV through rotation and translation matrices; different color channels or hue offsets are used for the point clouds of different routes to facilitate the differentiation of data sources from different UAVs; the detection results reported by each UAV are overlaid and rendered in the form of 3D bounding boxes onto the unified scene. The vertex buffer output by the GPU processing is directly used as the rendering input, without the need for intermediate copying by the CPU.

[0097] This embodiment also underwent experimental verification.

[0098] The technical parameters for this experiment are set as follows:

[0099] LiDAR parameters: resolution is 600 columns by 150 rows, totaling 90,000 pixels per frame; 600 UDP packets per frame; 628 bytes per packet; hardware frame rate is approximately 3 frames per second; raw data rate is approximately 9 Mbps per channel.

[0100] Onboard computing platform: An embedded edge computing board with a neural network processing unit. In one embodiment, the Horizon Robotics RDK S100 (BPU Nash architecture, 80 TOPS, approximately 45W power consumption) is used. The point cloud detection model adopts a pure 2D convolutional architecture with no sparse convolution dependency. The model computation is approximately 2 GFLOPS. After INT8 quantization, it is deployed to the BPU. All operators are successfully mapped to standard operators supported by the BPU, requiring no custom operator adaptation. The inference speed was measured at 19.2 frames per second (pipeline mode) on the RDK S100, meeting the real-time processing requirements.

[0101] Wireless link: adopts long-distance data transmission radio, network port pass-through mode, dual-band operation, communication distance greater than 15 kilometers, network port pass-through bandwidth up to 20Mbps, star network topology.

[0102] Data transmission volume: Detection results are approximately 4 kilobytes per frame, region of interest point clouds are approximately 50 to 200 kilobytes per frame, and downsampled full point clouds are approximately 100 to 300 kilobytes per frame.

[0103] Adaptive transmission parameters: urgency threshold is set to 10.0, link quality threshold is set to 0.8 (corresponding to a packet loss rate of less than 20%), and urgency smoothing coefficient is set to 0.3.

[0104] Under the above technical parameters, the performance test data of the single-path GPU fusion processing pipeline on the ground end are shown in Table 2:

[0105]

[0106] Table 2

[0107] The performance scaling estimation results for multi-path parallelism are shown in Table 3:

[0108]

[0109] Table 3

[0110] Based on Tables 2 and 3, the pure GPU execution time accounts for only 5.5% of the total single-path processing time, with the GPU remaining idle most of the time. In multi-path parallel processing, GPU computation can be time-division multiplexed on the same GPU, and the total GPU execution time increases linearly with the number of paths, but the base is extremely small. The theoretical frame rate for a single path far exceeds the hardware input frame rate (approximately 3 frames per second), providing ample margin for multi-path parallel processing.

[0111] Based on the above embodiments of this application, the airborne lidar data adaptive transmission scheme of this application has at least the following advantages:

[0112] (1) By deploying a detection model based on density-aware convolution at the airborne end, point cloud target detection is moved from the ground end to the airborne end. The detection model uses only standard two-dimensional convolution operators throughout the entire link, does not depend on sparse convolution operator libraries, can be deployed on general edge inference chips that only support standard operators, is not limited by specific GPU platforms or specific vendor NPUs, and has the widest edge chip compatibility;

[0113] (2) Perception-driven adaptive transmission architecture: Transmission priority and bandwidth allocation are not manually preset or passively adjusted according to link quality, but are driven in real time by the inference results of the detection model. The target category, confidence level and distance information output by the model are directly controlled by the token bucket sending rate after being calculated by the urgency formula, forming a closed loop of perception → urgency → transmission strategy, so that transmission resources are dynamically allocated according to the semantics of the scene; the detection results are always sent with the highest priority, and can still be delivered in real time even when the link bandwidth is severely insufficient;

[0114] (3) In the multi-machine bandwidth scheduling mechanism, the bandwidth allocation weight of each machine comes from the perception results of its own detection model rather than the link-side indicators, so that the UAV that discovers high-value targets automatically obtains more bandwidth and prioritizes the transmission of key information. At the same time, the minimum weight mechanism ensures that UAVs with low urgency still maintain basic communication capabilities.

[0115] (4) The ground-based multi-GPU real-time visualization system enables the simultaneous reception and rendering of point clouds from multiple UAVs through the GPU fusion processing pipeline. The pure execution time of a single GPU accounts for only about 5.5% of the total processing time. The GPU has ample spare capacity and supports multi-path parallel expansion. The multi-path point clouds are superimposed and displayed in a unified three-dimensional scene, allowing operators to obtain global situational awareness of multiple UAVs from a single perspective.

[0116] (5) The full original point cloud is always cached locally on the airborne end, and the data is not lost due to bandwidth limitations, thus ensuring the need for detailed offline analysis.

[0117] This application also provides a collaborative control system. Figure 5 This is a structural block diagram of a cooperative control system illustrated in an exemplary embodiment of this application. The cooperative control system includes a ground terminal 510 and multiple airborne terminals 520. The multiple airborne terminals share the same wireless link through a star network and communicate with the ground terminal through the wireless link; wherein:

[0118] The airborne terminal 520 is used to perform target detection on point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency, calculate the scene urgency of the current frame based on the target detection results, and upload it to the ground terminal; in the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling on the convolution output according to the ratio of the effective number of pixels in the receptive field to the area of ​​the convolution kernel; and transmit the target detection results and point cloud data on the wireless link through different ports. The target detection results are transmitted through the high-priority port, and the point cloud data is transmitted through the low-priority port. The transmission rate of the point cloud data is constrained by the bandwidth allocated to the current frame, and under this bandwidth constraint, the transmission accuracy is adaptively selected for transmission based on the scene urgency and link quality.

[0119] Ground terminal 510 is used to allocate bandwidth to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and to notify each airborne terminal.

[0120] In some embodiments, the ground terminal 510 is further configured to configure independent GPU fusion processing pipelines for the point cloud data streams of each airborne terminal, and to overlay and render the processing results of each GPU fusion processing pipeline in a unified 3D scene; wherein, each GPU fusion processing pipeline writes the pixel bytes carried by the point cloud data packets uploaded by the corresponding airborne terminal into the GPU shared buffer, each GPU fusion processing pipeline starts the GPU computing shader, and each thread of the GPU computing shader reads the corresponding pixel bytes from the GPU shared buffer and performs the following operations: extracting distance values ​​and reflection intensity through displacement masks, converting distance values ​​into 3D coordinates according to the optical model, and outputting 3D vertices containing 3D coordinates and reflection intensity.

[0121] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0122] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A perception-driven adaptive data transmission method for airborne lidar, characterized in that, Includes the following steps: Step S1: Each airborne terminal performs target detection on the point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency, calculates the scene urgency of the current frame based on the target detection results, and uploads it to the ground terminal; wherein, multiple airborne terminals share the same wireless link through a star-shaped network; in the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling of the convolution output according to the ratio of the number of effective pixels in the receptive field to the area of ​​the convolution kernel; Step S2: The ground terminal allocates a transmission rate to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and notifies each airborne terminal. In step S3, each airborne terminal transmits target detection results and point cloud data on the wireless link through different ports. The target detection results are transmitted through high-priority ports, and the point cloud data is transmitted through low-priority ports. Under the constraint of the transmission rate, the transmission accuracy of the point cloud data is adaptively selected for transmission based on the urgency of the scenario and the link quality.

2. The method according to claim 1, characterized in that, Step S1 calculates the scene urgency index of the current frame through the following steps: The original urgency is obtained by weighting and summing the values ​​of each detected target according to its category weight, detection confidence, and distance decay factor. Based on a preset smoothing coefficient and the scene urgency index of the previous frame, the original urgency is smoothed inter-frame to obtain the scene urgency index of the current frame.

3. The method according to claim 1, characterized in that, The expression for the distance attenuation factor for each detected target is: ； in, This represents the horizontal distance from the k-th detected target to the airborne lidar sensor. This indicates the preset reference distance. This represents the distance attenuation factor for the k-th detected target.

4. The method according to claim 1, characterized in that, Step S3, which adaptively selects transmission precision based on the urgency of the scenario and link quality for transmission, includes: Determine the expected content to be transmitted for point cloud data based on the scenario urgency index and link quality. Based on the transmission rate corresponding to the allocated bandwidth of the current frame, a transmission decision is made for the expected transmission content; Adaptive transmission of point cloud data based on decision results.

5. The method according to claim 4, characterized in that, The process of determining the expected transmission content of point cloud data based on scenario urgency indicators and link quality includes: When the urgency index of the scene is higher than the preset threshold and the link quality is good, the expected content to be transmitted is a point cloud of the region of interest centered on the target. When the urgency index of the scenario is higher than the preset threshold and the link quality is poor, the expected content to be transmitted is not to send point clouds; When the urgency index of the scenario is lower than the preset threshold and the link quality is good, the expected content to be transmitted is the downsampled full point cloud. When the urgency index of the scenario is lower than the preset threshold and the link quality is poor, the expected content to be transmitted is not to send point clouds; Link quality is obtained by sending sequenced heartbeat packets from each airborne terminal to the ground terminal at regular intervals, and the ground terminal calculates the packet loss rate and sends it back.

6. The method according to claim 4, characterized in that, The decision to send the desired content includes: If the desired transmission content is not to send point cloud data, then the decision is not to send point cloud data. If the desired content to be transmitted is a region of interest (ROI) point cloud, the decision is as follows: if the transmission rate of the current frame is sufficient to transmit the ROI point cloud, then upload the ROI point cloud; if the transmission rate of the current frame is only sufficient to transmit the downsampled full point cloud, then upload the downsampled full point cloud; if the transmission rate of the current frame is insufficient to transmit the downsampled full point cloud, then do not upload the point cloud data. If the expected content to be transmitted is a downsampled full point cloud, the decision is as follows: if the transmission rate of the current frame is sufficient to transmit the downsampled full point cloud, then upload the downsampled full point cloud; otherwise, do not upload the point cloud data.

7. The method according to claim 1, characterized in that, The transmission rate allocated to each airborne terminal in step S2 is expressed by the following formula: ； in, Indicates assignment to the first The transmission rate of the airborne terminal, This indicates the total available bandwidth. Indicates the first Bandwidth allocation weights at the airborne end. Indicates the first The urgency level of the scenario reported by the airborne terminal. This indicates the number of currently connected airborne terminals. This indicates the preset minimum guarantee weight.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes the following steps: Step S4: The ground end configures independent GPU fusion processing pipelines for the point cloud data streams of each airborne end, and overlays and renders the processing results of each GPU fusion processing pipeline in a unified 3D scene. Among them, each GPU fusion processing pipeline writes the pixel bytes carried by the point cloud data packets uploaded by the corresponding airborne end into the GPU shared buffer. Each GPU fusion processing pipeline starts the GPU computing shader. Each thread of the GPU computing shader reads the corresponding pixel bytes from the GPU shared buffer and performs the following operations: extracts the distance value and reflection intensity through displacement mask, converts the distance value into 3D coordinates according to the optical model, and outputs 3D vertices containing 3D coordinates and reflection intensity.

9. A cooperative control system, characterized in that, It includes a ground terminal and multiple airborne terminals. The multiple airborne terminals share the same wireless link through a star network and communicate with the ground terminal through the wireless link; wherein: On the airborne end, a target detection method is used to perform target detection on point cloud data collected by its airborne LiDAR based on a point cloud detection model without sparse convolution dependency. The method calculates the scene urgency of the current frame based on the target detection results and uploads it to the ground end. In the backbone network of the point cloud detection model, the first N levels of each stage perform density scaling on the convolution output according to the ratio of the effective number of pixels in the receptive field to the area of ​​the convolution kernel. The method also transmits the target detection results and point cloud data on the wireless link through different ports. The target detection results are transmitted through high-priority ports, and the point cloud data is transmitted through low-priority ports. The transmission rate of the point cloud data is constrained by the bandwidth allocated to the current frame, and under this bandwidth constraint, the transmission accuracy is adaptively selected based on the scene urgency and link quality. The ground-based system allocates bandwidth to each airborne terminal based on the urgency of the scenario reported by each airborne terminal and notifies each airborne terminal.

10. The system according to claim 9, characterized in that, On the ground, it is also used to configure independent GPU fusion processing pipelines for the point cloud data streams of each airborne terminal, and to overlay and render the processing results of each GPU fusion processing pipeline in a unified 3D scene. Among them, each GPU fusion processing pipeline writes the pixel bytes carried by the point cloud data packets uploaded by the corresponding airborne terminal into the GPU shared buffer. Each GPU fusion processing pipeline starts the GPU computing shader. Each thread of the GPU computing shader reads the corresponding pixel bytes from the GPU shared buffer and performs the following operations: extracting distance values ​​and reflection intensity through displacement masks, converting distance values ​​into 3D coordinates according to the optical model, and outputting 3D vertices containing 3D coordinates and reflection intensity.

Images