Linkage display control method and system based on unmanned aerial vehicle

Abstract

The invention discloses a linkage display control method and system based on unmanned aerial vehicles, and relates to the technical field of unmanned aerial vehicle cluster control, and the method comprises the steps: receiving a display content source through a ground control terminal, and dividing the content into sub-image blocks corresponding to each unmanned aerial vehicle in an unmanned aerial vehicle array; acquiring the spatial position, attitude angle and ambient light intensity data of each unmanned aerial vehicle in real time, and generating a dynamic compensation parameter set; packing the sub-picture blocks and the corresponding dynamic compensation parameter sets into data packets, and distributing the data packets to each unmanned aerial vehicle through a time division multiplexing channel; after the unmanned aerial vehicle side analyzes the data packet, an FPGA hardware accelerator is adopted to execute the following real-time compensation; and triggering the frame refreshing operation of all the unmanned aerial vehicle display units through the high-precision synchronous signal generator. Through FPGA, multi-sensor fusion compensation and layered synchronous control, the problems of high delay, non-uniform color, poor dynamic response and low reliability in an unmanned aerial vehicle display scheme are solved, and the method is suitable for large-scale, high-dynamic and strong-interference complex application scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) cluster control, and in particular to a linkage display control method and system based on UAVs. Background Art

[0002] With the maturity of drone technology and the reduction in costs, drone performances have become an essential component of large-scale events, commercial advertising, and urban landscapes. Market demand for more creative, interactive, and flexible drone display systems is growing. Technological evolution is manifesting itself in many areas: display technology is evolving from single-drone light displays to multiple drones working together to form a three-dimensional display, pursuing higher resolution, richer colors, and smoother dynamic effects; control technology is evolving from centralized control to distributed intelligent control, improving system reliability and adaptability; and application scenarios are expanding from simple entertainment performances to multiple fields such as emergency communications, advertising and marketing, and educational displays.

[0003] However, there are still some problems that need to be solved: there is an obvious synchronization delay problem when multiple drones display collaboratively, resulting in image tearing or discontinuous dynamic effects; there are color and brightness differences between the display units of different drones, affecting the overall display quality; when the drone position is dynamically adjusted, existing technology makes it difficult to compensate for the deformation and distortion of the display content in real time. Summary of the Invention

[0004] In order to solve the above technical problems, a linkage display control method and system based on drones are provided. This technical solution solves the above problems of display synchronization delay, inconsistent color brightness, and insufficient dynamic compensation.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] A linkage display control method and system based on a drone, comprising:

[0007] receiving a display content source through a ground control terminal and dividing the content into sub-screen blocks corresponding to each drone in the drone array;

[0008] Collect the spatial position, attitude angle and ambient light intensity data of each drone in real time to generate a dynamic compensation parameter set;

[0009] The sub-image blocks and the corresponding motion compensation parameter sets are packaged into data packets and distributed to each UAV through a time division multiplexing channel;

[0010] After parsing the data packet on the drone side, an FPGA hardware accelerator is used to perform the following real-time compensation;

[0011] The frame refresh operation of all drone display units is triggered by a high-precision synchronization signal generator.

[0012] Preferably, receiving the display content source through the ground control terminal and dividing the content into sub-screen blocks corresponding to each drone in the drone array specifically includes:

[0013] Establish a virtual display plane coordinate system for the drone array and calculate the boundary coordinates of the projection area of each drone in the virtual plane based on the real-time positioning data of each drone and the preset formation parameters;

[0014] Based on the boundary coordinates of the projection area, the display content source is non-uniformly divided to generate sub-screen blocks that match the physical display size of the drone;

[0015] For the overlapping areas of adjacent drones, a bilateral filtering algorithm is used to gradually fuse the overlapping pixels;

[0016] Monitor the real-time rendering load of each drone's display unit. If a drone's load exceeds a threshold, reduce the resolution of that drone's sub-screen block to a predetermined threshold and simultaneously increase the image detail compensation of adjacent drones.

[0017] Spatial position metadata is added to each sub-picture block, including normalized coordinates in the virtual plane coordinate system, identifiers of overlapping areas with adjacent sub-pictures, and a color consistency check code.

[0018] Preferably, the real-time collection of spatial position, attitude angle and ambient light intensity data of each UAV to generate a dynamic compensation parameter set specifically includes:

[0019] The drone's dual-frequency RTK-GNSS and UWB indoor positioning system are integrated to output 3D coordinates. The IMU inertial measurement unit's acceleration data is combined to update the position information at a predetermined frequency.

[0020] Use MEMS nine-axis sensor to measure pitch, roll and yaw angles in real time; use Kalman filter algorithm to eliminate sensor drift error;

[0021] The multispectral ambient light sensor is used to collect the ambient light intensity, main light source color temperature, and incident angle.

[0022] Preferably, the real-time collection of spatial position, attitude angle and ambient light intensity data of each UAV to generate a dynamic compensation parameter set includes:

[0023] Calculate the perspective transformation matrix of the overlapping area based on the relative distance and attitude angle differences of adjacent drones;

[0024] When the distance between drones changes beyond a threshold, the sub-pixel edge interpolation algorithm is triggered to reconstruct the image connection;

[0025] Generate a color correction matrix based on the ambient light color temperature and the color gamut mapping table of the drone display unit; if the ambient light color temperature changes beyond a predetermined threshold, enable real-time gamma curve adjustment based on the CIE1931 color space; calculate the regional backlight compensation coefficient based on the difference between the ambient light intensity and the preset brightness value;

[0026] Predict the displacement of the next frame based on the drone's speed; if the predicted displacement causes the pixel offset to exceed the predetermined pixel, the motion blur compensation engine is activated;

[0027] When it is detected that the positioning data packet loss rate is greater than the established threshold and the ambient light sampling is abnormal, it switches to the SLAM-assisted positioning mode based on visual feature points and uses the average ambient light data of the nearest neighboring drone for compensation.

[0028] Preferably, the step of packaging the sub-picture blocks and the corresponding motion compensation parameter sets into a data packet and distributing the data packet to each drone via a time division multiplexing channel specifically includes:

[0029] Sub-image block encoding uses a custom FPGA-based compression algorithm for each sub-image block, dynamically adjusting the compression ratio to prioritize high-frequency details. It also adds spatial location metadata, including the drone ID and normalized boundary coordinates in the virtual coordinate system.

[0030] Dynamic compensation parameter binding, encapsulating the compensation parameter set into a binary structure, including perspective transformation matrix, color correction matrix, backlight compensation coefficient, and timestamp;

[0031] The channel allocation strategy is to divide the transmission cycle into 10ms time slots. Each time slot includes a control time slot for transmitting the drone ID, synchronization signal, and retransmission instructions; and a data time slot for dynamically allocating sub-screen data according to priority.

[0032] The transmission mode is switched based on the channel quality. The high-bandwidth mode uses 64QAM modulation to transmit uncompressed reference frames. The anti-interference mode uses QPSK modulation + forward error correction to compensate for the parameter set.

[0033] If a UAV does not receive data in consecutive time slots, it will start the relay forwarding by the neighboring UAVs and use the motion vector-based picture prediction algorithm to temporarily fill in the frames.

[0034] Preferably, the drone-side parsing of the data packet specifically includes:

[0035] The FPGA hard core parses the custom protocol header of the data packet to extract the sub-picture block, motion compensation parameters, and timestamp data. A CRC-32 hardware checker is used to verify data integrity, triggering a retransmission request if the check fails.

[0036] Import the sub-image data stream and compensation parameters into the dual-port block RAM of the FPGA respectively; align the processing timing according to the timestamp synchronization signal.

[0037] Preferably, after parsing the data packet on the drone side, using the FPGA hardware accelerator to perform the following real-time compensation specifically includes:

[0038] Based on the perspective transformation matrix, the edges of the sub-image are reconstructed through a bilinear interpolation hardware circuit. When the drone's displacement speed is detected to be greater than a predetermined speed, the motion prediction engine is activated to predict the displacement of the next frame based on the IMU data. The pixel offset compensation value is calculated using a phase correlation algorithm.

[0039] The FPGA's DSP48E1 array performs color matrix operations. If the ambient light color temperature changes by more than a predetermined threshold, the 3D LUT hardware is triggered to perform nonlinear mapping.

[0040] Dynamically adjust the PWM duty cycle based on the backlight compensation coefficient to increase the LED drive current to the rated value in strong light environments; enable the local dimming algorithm in low light environments and turn off the backlight in dark areas;

[0041] Monitor the utilization of FPGA logic units in real time. If the utilization exceeds a certain percentage, downgrade processing to non-critical areas and send a resource shortage warning to the ground control terminal.

[0042] The FPGA junction temperature is monitored by a temperature sensor. If the temperature exceeds a certain level, dynamic clock frequency adjustment is enabled and non-real-time tasks are shut down.

[0043] The compensated sub-image is written into the DDR3 video memory controller and output in strict timing according to the V-Sync signal; the adaptive overdrive circuit is enabled for high dynamic images to reduce LCD response delay;

[0044] Receives PTPv2 synchronization pulses from the ground terminal and refreshes the display unit within the specified time window.

[0045] Preferably, the frame refresh operation of triggering all drone display units by a high-precision synchronization signal generator specifically includes:

[0046] The ground control terminal is equipped with a rubidium atomic clock module to generate a 10MHz reference clock signal; the clock signal is distributed to all drones via the IEEE 1588v2 protocol.

[0047] Hierarchical synchronization triggering: the first-level synchronization is the global frame start pulse broadcast by the ground end, and the period accurately matches the display frame rate; the second-level synchronization is that each drone generates a local V-Sync signal according to the FSP, and the digital phase-locked loop inside the FPGA eliminates transmission jitter;

[0048] The FPGA integrates a programmable delay line module to dynamically adjust the signal delay according to the distance between the drone and the ground terminal; the V-Blank signal of the display unit is strictly aligned with the FSP;

[0049] Real-time monitoring of the drone's motion status. If the acceleration exceeds a predetermined speed, predictive frame advance triggering is enabled to adjust the display unit's overdrive voltage to shorten the LCD response time.

[0050] Online deviation detection compares the deviation between the drone's local clock and the ground reference within each frame period. If a deviation occurs, the DPLL relock is triggered; the screen refresh time is measured through the photoelectric sensor array and feedback is used to calibrate the synchronization parameters; the FPGA temperature changes are monitored, and if the temperature rises to a predetermined threshold, the clock delay compensation value is increased.

[0051] Furthermore, a drone-based linkage display control system is provided for implementing the drone-based linkage display control method described above, comprising:

[0052] The ground control module includes a video segmentation unit that divides the input display content into sub-screen blocks corresponding to the drone array; a synchronous encoder that generates high-precision synchronization signals and distributes them to the drone cluster via a wireless communication network; and a dynamic compensation calculation unit that generates color correction, brightness compensation, and image stitching parameters based on the real-time drone posture and ambient light data.

[0053] Each drone in the drone swarm module includes: an RTK / UWB high-precision positioning unit that provides centimeter-level spatial position data; a nine-axis IMU sensor that measures pitch, roll, and yaw angles in real time; a multispectral ambient light sensor that collects ambient light intensity and color temperature; an FPGA hardware accelerator for performing real-time sub-image compensation, including edge interpolation and color backlight adjustment; and an HDR display unit that supports local dimming and dynamic refresh rates.

[0054] The wireless communication module adopts a layered transmission architecture. Control commands are broadcast via the LoRa wide area network. Video data and compensation parameters are transmitted point-to-point via 60GHz millimeter waves. Synchronization signals are distributed via the IEEE 1588v2 protocol.

[0055] The exception handling module is used to switch to the auxiliary positioning mode based on visual SLAM when positioning packet loss or communication interruption occurs, enable relay communication with neighboring drones, and predict image compensation parameters based on Kalman filtering.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention proposes to use an FPGA hardware accelerator to perform real-time compensation, and reduce frame refresh synchronization errors through a high-precision synchronization signal generator, making it suitable for high-speed dynamic scenes. Through a multispectral ambient light sensor + CIE 1931 color space conversion, a color correction matrix is generated in real time. Combined with a local dimming algorithm and PWM dynamic backlight adjustment, brightness consistency errors are reduced to achieve visual seamless splicing across UAV display units. Based on RTK / IMU data, the perspective transformation matrix is calculated in real time, and the geometric deformation of the sub-screen is dynamically adjusted. Rendering load monitoring and adaptive resolution reduction are introduced to ensure a stable frame rate, and the image integrity and smoothness can still be maintained when the UAV speed is >2m / s. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is an internal flow chart of a linkage display control method based on a UAV;

[0059] Figure 2 A flow chart of a method for segmenting sub-image blocks corresponding to a drone;

[0060] Figure 3 A flow chart of a method for generating a dynamic compensation parameter set;

[0061] Figure 4 A flow chart of a method for time division multiplexing channel distribution;

[0062] Figure 5 This is an internal framework diagram of a UAV-based linkage display control system. DETAILED DESCRIPTION

[0063] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0064] Reference Figure 1 As shown, a linkage display control method based on a drone includes:

[0065] receiving a display content source through a ground control terminal and dividing the content into sub-screen blocks corresponding to each drone in the drone array;

[0066] Collect the spatial position, attitude angle and ambient light intensity data of each drone in real time to generate a dynamic compensation parameter set;

[0067] The sub-image blocks and the corresponding motion compensation parameter sets are packaged into data packets and distributed to each UAV through a time division multiplexing channel;

[0068] After parsing the data packet on the drone side, an FPGA hardware accelerator is used to perform the following real-time compensation;

[0069] The frame refresh operation of all drone display units is triggered by a high-precision synchronization signal generator.

[0070] It should be noted that this method has a real-time data closed-loop feedback mechanism. After the FPGA on the drone performs compensation, the actual display effect, such as color deviation and brightness error, is transmitted back to the ground control terminal through a wireless channel to optimize the compensation parameter generation for the next frame, including updating the color mapping matrix or the overlapping area interpolation algorithm.

[0071] When a UAV fails to receive the synchronization signal due to communication delay, it automatically uses the clock extrapolation of the previous cycle to maintain short-term synchronization until the signal is restored.

[0072] When the drone array is first deployed, all display modules must be colorimetrically calibrated by the ground station (using an X-Rite colorimeter); the communication link latency of each drone must be measured and written into the FPGA's timing compensation registers; and the entire fleet must be synchronized and stress-tested (simulating an extreme scenario with a packet loss rate of <1%).

[0073] Reference Figure 2 As shown, the receiving of the display content source through the ground control terminal and the division of the content into sub-screen blocks corresponding to each drone in the drone array include:

[0074] Establish a virtual display plane coordinate system for the drone array and calculate the boundary coordinates of the projection area of each drone in the virtual plane based on the real-time positioning data of each drone and the preset formation parameters;

[0075] Based on the boundary coordinates of the projection area, the display content source is non-uniformly divided to generate sub-screen blocks that match the physical display size of the drone;

[0076] For the overlapping areas of adjacent drones, a bilateral filtering algorithm is used to gradually fuse the overlapping pixels;

[0077] Monitor the real-time rendering load of each drone's display unit. If a drone's load exceeds a threshold, reduce the resolution of that drone's sub-screen block to a predetermined threshold and simultaneously increase the image detail compensation of adjacent drones.

[0078] Spatial position metadata is added to each sub-picture block, including normalized coordinates in the virtual plane coordinate system, identifiers of overlapping areas with adjacent sub-pictures, and a color consistency check code.

[0079] It should be noted that the virtual display plane coordinate system includes:

[0080] The coordinate system initialization requires a clear reference for establishing a virtual plane, with the formation's geometric center as the origin and the Z axis perpendicular to the ground. When the number of drones changes, the coordinate system's dynamic adjustment algorithm refits the plane equation based on the least squares method; the sudden acceleration of the drone's posture is >3m / s 2 The coordinate system update frequency is increased from the default 100Hz to 500Hz, and the abnormal data filtering method is used. When the RTK signal is lost, the α-β filter is used to smooth the trajectory.

[0081] Non-uniform segmentation includes using the minimum segmentation unit 4×4 pixel block for high-detail areas (faces, text); merging low-complexity areas (solid background) into macroblocks of 32×32 pixel blocks;

[0082] For anti-aliasing of the segmentation boundary, a transition zone of 1-2 pixels is reserved on both sides of the projection boundary, and the Lanczos interpolation algorithm is applied to avoid the aliasing effect.

[0083] The fusion of overlapping areas uses bilateral filtering algorithm with spatial standard deviation σ s and color standard deviation σ r The calculation formula is:

[0084]

[0085] Where d is the width of the overlap region, ΔE adj This is the measured color difference between adjacent drone display modules.

[0086] When a pixel belongs to the overlapping area of three or more drones at the same time, three-level weighted fusion is enabled: priority weight is given to the drone with the closest geometric center; secondary weight is allocated according to the brightness consistency of the display modules; the final blending result must pass the CRC-8 check.

[0087] The load threshold is set based on the mapping between the GPU utilization threshold and the resolution reduction ratio:

[0088] When the load rate is between 80% and 90%, the resolution is reduced to 90%, and the adjacent compensation is +5% detail; when the load rate is between 90% and 100%, the resolution is reduced to 70%, and the adjacent compensation is +15% detail;

[0089] Among them, detail compensation is to extract high-frequency components through wavelet transform and embed detail data into the DCT coefficients of adjacent drones; the amount of compensation data must not exceed 10% of the original picture block to prevent channel overload.

[0090] The color check code is obtained by sampling the RGB mean value of the central 5% area of the sub-image block; converting it to CIE-Lab value and calculating the ΔE00 color difference; rounding it and storing it in 8-bit binary code (0 to 255 corresponds to ΔE0 to 10).

[0091] Abnormal scenarios include rapid formation changes. When the formation change speed is detected to be >2m / s: the non-uniform segmentation is suspended and switched to fixed grid mode; the overlap area width is automatically increased by 20%; and the color parameters of the entire team are forced to synchronize.

[0092] Reference Figure 3 As shown, the real-time collection of spatial position, attitude angle and ambient light intensity data of each UAV to generate a dynamic compensation parameter set includes:

[0093] The drone's dual-frequency RTK-GNSS and UWB indoor positioning system are integrated to output 3D coordinates. The IMU inertial measurement unit's acceleration data is combined to update the position information at a predetermined frequency.

[0094] Use MEMS nine-axis sensor to measure pitch, roll and yaw angles in real time; use Kalman filter algorithm to eliminate sensor drift error;

[0095] The multispectral ambient light sensor is used to collect the ambient light intensity, main light source color temperature, and incident angle.

[0096] Calculate the perspective transformation matrix of the overlapping area based on the relative distance and attitude angle differences of adjacent drones;

[0097] When the distance between drones changes beyond a threshold, the sub-pixel edge interpolation algorithm is triggered to reconstruct the image connection;

[0098] Generate a color correction matrix based on the ambient light color temperature and the color gamut mapping table of the drone display unit; if the ambient light color temperature changes beyond a predetermined threshold, enable real-time gamma curve adjustment based on the CIE1931 color space; calculate the regional backlight compensation coefficient based on the difference between the ambient light intensity and the preset brightness value;

[0099] Predict the displacement of the next frame based on the drone's speed; if the predicted displacement causes the pixel offset to exceed the predetermined pixel, the motion blur compensation engine is activated;

[0100] When it is detected that the positioning data packet loss rate is greater than the established threshold and the ambient light sampling is abnormal, it switches to the SLAM-assisted positioning mode based on visual feature points and uses the average ambient light data of the nearest neighboring drone for compensation.

[0101] It should be noted that the RTK-UWB-IMU data weight distribution and sensor confidence weights in different scenarios are:

[0102] Outdoor scene (RTK signal is good): RTK weight 70% + IMU 30%

[0103] Indoor scene (UWB-dominated): UWB weight 60% + IMU 40%

[0104] When the signal is lost: pure IMU calculation, using the cumulative error compensation algorithm.

[0105] Time alignment method for multi-sensor data: IEEE 1588v2 protocol is used to align the clocks of each sensor; kinematic model forward prediction is applied to the IMU data (prediction time = communication delay + processing delay).

[0106] Kalman filter parameter Q k Dynamic adjustment, online update rule of the noise covariance matrix:

[0107]

[0108] Where Q0 is the factory calibration value, and the static mode prioritizes trusting the gyroscope data.

[0109] Magnetic interference compensation: When the magnetometer data changes suddenly by more than 30μT, the magnetometer is temporarily used to calculate the yaw angle, and the IMU angular velocity integration + visual assistance correction is used instead.

[0110] Drone spacing change threshold, basic threshold = display module pixel spacing × 2; when the drone speed is > 2m / s, the threshold is reduced by 30%.

[0111] Trigger conditions for SLAM-assisted positioning: positioning packet loss rate > 5% (no data for three consecutive cycles); abnormal ambient light sampling = color temperature sudden change > 1000K or illumination difference > 50%.

[0112] The ambient light data of neighboring drones was screened by selecting nodes with a spatial distance of <3m and a historical data similarity of >90%. Median filtering was used instead of mean filtering for abnormal data.

[0113] Reference Figure 4 As shown, the sub-picture block and the corresponding motion compensation parameter set are packaged into a data packet and distributed to each user via a time division multiplexing channel, including:

[0114] Sub-image block encoding uses a custom FPGA-based compression algorithm for each sub-image block, dynamically adjusting the compression ratio to prioritize high-frequency details. It also adds spatial location metadata, including the drone ID and normalized boundary coordinates in the virtual coordinate system.

[0115] Dynamic compensation parameter binding, encapsulating the compensation parameter set into a binary structure, including perspective transformation matrix, color correction matrix, backlight compensation coefficient, and timestamp;

[0116] The channel allocation strategy is to divide the transmission cycle into 10ms time slots. Each time slot includes a control time slot for transmitting the drone ID, synchronization signal, and retransmission instructions; and a data time slot for dynamically allocating sub-screen data according to priority.

[0117] The transmission mode is switched based on the channel quality. The high-bandwidth mode uses 64QAM modulation to transmit uncompressed reference frames. The anti-interference mode uses QPSK modulation + forward error correction to compensate for the parameter set.

[0118] If a UAV does not receive data in consecutive time slots, it will start the relay forwarding by the neighboring UAVs and use the motion vector-based picture prediction algorithm to temporarily fill in the frames.

[0119] It should be noted that the high-frequency detail preservation mechanism uses a 5×5DCT kernel to separate frequency domain components, retaining the original accuracy of high-frequency coefficients (>50% energy), and using 4:1 lossy quantization for low-frequency coefficients;

[0120] The dynamic compression ratio is adjusted based on the complexity of the image through Sobel edge detection statistics and the quantization step size;

[0121] Additional spatial location metadata, spatial coordinate encoding format: normalized coordinates (x, y, z) are stored in FP16 floating point; boundary coordinates are marked with a 1-pixel guard band.

[0122] Channel quality assessment standards and hard thresholds for mode switching: High-bandwidth mode conditions are signal-to-noise ratio ≥ 20dB, bit error rate ≤ 1e-6, and delay jitter ≤ 100μs; anti-interference mode conditions are signal-to-noise ratio < 15dB, bit error rate ≥ 1e-4, and delay jitter ≥ 500μs.

[0123] The reference frame selection strategy is to forcibly send one uncompressed reference frame (I frame) every 10 frames; when the cumulative packet loss rate is greater than 5%, an additional I frame is temporarily inserted.

[0124] A neighboring drone can only serve as a relay node if its signal strength RSSI>-70dBm, remaining battery>30%, and current load rate<60%.

[0125] Reference Figure 5 As shown, a linkage display control system based on a UAV includes:

[0126] The ground control module includes a video segmentation unit that divides the input display content into sub-screen blocks corresponding to the drone array; a synchronous encoder that generates high-precision synchronization signals and distributes them to the drone cluster via a wireless communication network; and a dynamic compensation calculation unit that generates color correction, brightness compensation, and image stitching parameters based on the real-time drone posture and ambient light data.

[0127] Each drone in the drone swarm module includes: an RTK / UWB high-precision positioning unit that provides centimeter-level spatial position data; a nine-axis IMU sensor that measures pitch, roll, and yaw angles in real time; a multispectral ambient light sensor that collects ambient light intensity and color temperature; an FPGA hardware accelerator for performing real-time sub-image compensation, including edge interpolation and color backlight adjustment; and an HDR display unit that supports local dimming and dynamic refresh rates.

[0128] The wireless communication module adopts a layered transmission architecture. Control commands are broadcast via the LoRa wide area network. Video data and compensation parameters are transmitted point-to-point via 60GHz millimeter waves. Synchronization signals are distributed via the IEEE 1588v2 protocol.

[0129] The exception handling module is used to switch to the auxiliary positioning mode based on visual SLAM when positioning packet loss or communication interruption occurs, enable relay communication with neighboring drones, and predict image compensation parameters based on Kalman filtering.

[0130] It should be noted that the ground control module includes:

[0131] The video segmentation unit's dynamic load balancing automatically adjusts the segmentation granularity when it detects a sudden change in the complexity of the image content, such as switching from static text to dynamic video:

[0132]

[0133] Real-time edge density analysis is achieved through FPGA hardware, using Sobel operator parallel computing.

[0134] The dynamic compensation calculation unit includes:

[0135] The perspective transformation matrix is used to correct the image deformation caused by the difference in the drone's posture:

[0136]

[0137] Where (x, y) is the pixel coordinate of the original sub-image, which comes from the output of the video segmentation module; (x′, y′) is the pixel coordinate of the transformed target image, which needs to be seamlessly spliced with the adjacent drone images; H is the 3×3 perspective transformation matrix, which is generated in real time by the motion compensation calculation unit; h 11 -h 33 It is a matrix parameter calculated using the following data: the UAV RTK positioning coordinates (X, Y, Z), the pitch angle θ and roll angle φ measured by the IMU, and the preset virtual display plane equation Ax+By+Cz+D=0.

[0138] The drone swarm module includes:

[0139] Thermal management of the FPGA accelerator, dynamic voltage and frequency adjustment, temperature range <75°C, core voltage 1.0V, operating frequency 200MHz; temperature range ≥85°C, core voltage 0.9V, operating frequency 150MHz.

[0140] The exception handling module includes:

[0141] The trigger conditions for SLAM-assisted positioning are: continuous loss of positioning data ≥3 frames (each frame period 10ms); RTK positioning variance >5cm (static scene) or >15cm (dynamic scene); visual feature point matching success rate <60%.

[0142] In summary, the advantages of the present invention are: through FPGA hardware acceleration + rubidium atomic clock time reference, random delays caused by software processing are eliminated; based on the dynamic compensation algorithm of multi-spectral ambient light perception, color consistency of ΔE<3 is achieved, and brightness differences are controlled through local dimming algorithm and PWM dynamic adjustment; when positioning is abnormal, visual SLAM assistance is automatically switched, and the neighboring drone self-organizing network relay is enabled when communication is interrupted, and motion prediction compensation is used for data loss, with good fault tolerance and autonomous capabilities; the application scenarios are broad, including the aerial stereoscopic display of tens of thousands of drones in super-large performances; dynamic optical camouflage systems in the military field; communication relay platforms that can be quickly deployed after disasters for emergency communications; and low-latency air traffic guidance systems for intelligent transportation.

[0143] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the invention as claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A linkage display control method based on drone, characterized in that: include: receiving a display content source through a ground control terminal and dividing the content into sub-screen blocks corresponding to each drone in the drone array; Collect the spatial position, attitude angle and ambient light intensity data of each drone in real time to generate a dynamic compensation parameter set; The sub-image blocks and the corresponding motion compensation parameter sets are packaged into data packets and distributed to each UAV through a time division multiplexing channel; After parsing the data packet on the drone side, an FPGA hardware accelerator is used to perform the following real-time compensation; The frame refresh operation of all drone display units is triggered by a high-precision synchronization signal generator.

2. The method for controlling linkage display based on a drone according to claim 1, characterized in that: The receiving of the display content source through the ground control terminal and dividing the content into sub-screen blocks corresponding to each drone in the drone array specifically includes: Establish a virtual display plane coordinate system for the drone array and calculate the boundary coordinates of the projection area of each drone in the virtual plane based on the real-time positioning data of each drone and the preset formation parameters; Based on the boundary coordinates of the projection area, the display content source is non-uniformly divided to generate sub-screen blocks that match the physical display size of the drone; For the overlapping areas of adjacent drones, a bilateral filtering algorithm is used to gradually fuse the overlapping pixels; Monitor the real-time rendering load of each drone's display unit. If a drone's load exceeds a threshold, reduce the resolution of that drone's sub-screen block to a predetermined threshold and simultaneously increase the image detail compensation of adjacent drones. Spatial position metadata is added to each sub-picture block, including normalized coordinates in the virtual plane coordinate system, identifiers of overlapping areas with adjacent sub-pictures, and a color consistency check code.

3. The method for controlling linkage display based on a drone according to claim 2, characterized in that: The real-time collection of spatial position, attitude angle and ambient light intensity data of each UAV to generate a dynamic compensation parameter set specifically includes: The drone's dual-frequency RTK-GNSS and UWB indoor positioning system are integrated to output 3D coordinates. The IMU inertial measurement unit's acceleration data is combined to update the position information at a predetermined frequency. Use MEMS nine-axis sensor to measure pitch, roll and yaw angles in real time; use Kalman filter algorithm to eliminate sensor drift error; The multispectral ambient light sensor is used to collect the ambient light intensity, main light source color temperature, and incident angle.

4. The method according to claim 3, characterized in that Further including: Calculate the perspective transformation matrix of the overlapping area based on the relative distance and attitude angle differences of adjacent drones; When the distance between drones changes beyond a threshold, the sub-pixel edge interpolation algorithm is triggered to reconstruct the image connection; Generate a color correction matrix based on the ambient light color temperature and the color gamut mapping table of the drone display unit; if the ambient light color temperature changes beyond a predetermined threshold, enable real-time gamma curve adjustment based on the CIE1931 color space; calculate the regional backlight compensation coefficient based on the difference between the ambient light intensity and the preset brightness value; Predict the displacement of the next frame based on the drone's speed; if the predicted displacement causes the pixel offset to exceed the predetermined pixel, the motion blur compensation engine is activated; When it is detected that the positioning data packet loss rate is greater than the established threshold and the ambient light sampling is abnormal, it switches to the SLAM-assisted positioning mode based on visual feature points and uses the average ambient light data of the nearest neighboring drone for compensation.

5. The method for controlling linkage display based on a drone according to claim 4, characterized in that: The step of packaging the sub-picture blocks and the corresponding motion compensation parameter sets into a data packet and distributing the data packet to each drone via a time division multiplexing channel specifically includes: Sub-image block encoding uses a custom FPGA-based compression algorithm for each sub-image block, dynamically adjusting the compression ratio to prioritize high-frequency details. It also adds spatial location metadata, including the drone ID and normalized boundary coordinates in the virtual coordinate system. Dynamic compensation parameter binding, encapsulating the compensation parameter set into a binary structure, including perspective transformation matrix, color correction matrix, backlight compensation coefficient, and timestamp; The channel allocation strategy is to divide the transmission cycle into 10ms time slots. Each time slot includes a control time slot for transmitting the drone ID, synchronization signal, and retransmission instructions; and a data time slot for dynamically allocating sub-screen data according to priority. The transmission mode is switched based on the channel quality. The high-bandwidth mode uses 64QAM modulation to transmit uncompressed reference frames. The anti-interference mode uses QPSK modulation + forward error correction to compensate for the parameter set. If a UAV does not receive data in consecutive time slots, it will start the relay forwarding by the neighboring UAVs and use the motion vector-based picture prediction algorithm to temporarily fill in the frames.

6. The method for controlling linkage display based on a drone according to claim 5, characterized in that: The drone-side parsing data packet specifically includes: The FPGA hard core parses the custom protocol header of the data packet to extract the sub-picture block, motion compensation parameters, and timestamp data. A CRC-32 hardware checker is used to verify data integrity, triggering a retransmission request if the check fails. Import the sub-image data stream and compensation parameters into the dual-port block RAM of the FPGA respectively; align the processing timing according to the timestamp synchronization signal.

7. The method for controlling linkage display based on a drone according to claim 6, characterized in that: After the drone parses the data packet, the FPGA hardware accelerator is used to perform the following real-time compensation, specifically including: Based on the perspective transformation matrix, the edges of the sub-image are reconstructed through a bilinear interpolation hardware circuit. When the drone's displacement speed is detected to be greater than a predetermined speed, the motion prediction engine is activated to predict the displacement of the next frame based on the IMU data. The pixel offset compensation value is calculated using a phase correlation algorithm. The FPGA's DSP48E1 array performs color matrix operations. If the ambient light color temperature changes by more than a predetermined threshold, the 3D LUT hardware is triggered to perform nonlinear mapping. Dynamically adjust the PWM duty cycle based on the backlight compensation coefficient to increase the LED drive current to the rated value in strong light environments; enable the local dimming algorithm in low light environments and turn off the backlight in dark areas; Monitor the utilization of FPGA logic units in real time. If the utilization exceeds a certain percentage, downgrade processing to non-critical areas and send a resource shortage warning to the ground control terminal. The FPGA junction temperature is monitored by a temperature sensor. If the temperature exceeds a certain level, dynamic clock frequency adjustment is enabled and non-real-time tasks are shut down. The compensated sub-image is written into the DDR3 video memory controller and output in strict timing according to the V-Sync signal; the adaptive overdrive circuit is enabled for high dynamic images to reduce LCD response delay; Receives PTPv2 synchronization pulses from the ground terminal and refreshes the display unit within the specified time window.

8. The method for controlling linkage display based on a drone according to claim 7, characterized in that: The frame refresh operation of triggering all drone display units by the high-precision synchronization signal generator specifically includes: The ground control terminal is equipped with a rubidium atomic clock module to generate a 10MHz reference clock signal; the clock signal is distributed to all drones via the IEEE 1588v2 protocol. Hierarchical synchronization triggering: the first-level synchronization is the global frame start pulse broadcast by the ground end, and the period accurately matches the display frame rate; the second-level synchronization is that each drone generates a local V-Sync signal according to the FSP, and the digital phase-locked loop inside the FPGA eliminates transmission jitter; The FPGA integrates a programmable delay line module to dynamically adjust the signal delay according to the distance between the drone and the ground terminal; the V-Blank signal of the display unit is strictly aligned with the FSP; Real-time monitoring of the drone's motion status. If the acceleration exceeds a predetermined speed, predictive frame advance triggering is enabled to adjust the display unit's overdrive voltage to shorten the LCD response time. Online deviation detection compares the deviation between the drone's local clock and the ground reference within each frame period. If a deviation occurs, the DPLL relock is triggered; the screen refresh time is measured through the photoelectric sensor array and feedback is used to calibrate the synchronization parameters; the FPGA temperature changes are monitored, and if the temperature rises to a predetermined threshold, the clock delay compensation value is increased.

9. A linkage display control system based on drones, characterized in that: A method for realizing a linkage display control method based on a drone as described in claims 1 to 8, comprising: The ground control module includes a video segmentation unit that divides the input display content into sub-screen blocks corresponding to the drone array; a synchronous encoder that generates high-precision synchronization signals and distributes them to the drone cluster via a wireless communication network; and a dynamic compensation calculation unit that generates color correction, brightness compensation, and image stitching parameters based on the real-time drone posture and ambient light data. Each drone in the drone swarm module includes: an RTK / UWB high-precision positioning unit that provides centimeter-level spatial position data; a nine-axis IMU sensor that measures pitch, roll, and yaw angles in real time; a multispectral ambient light sensor that collects ambient light intensity and color temperature; an FPGA hardware accelerator for performing real-time sub-image compensation, including edge interpolation and color backlight adjustment; and an HDR display unit that supports local dimming and dynamic refresh rates. The wireless communication module adopts a layered transmission architecture. Control commands are broadcast via the LoRa wide area network. Video data and compensation parameters are transmitted point-to-point via 60GHz millimeter waves. Synchronization signals are distributed via the IEEE 1588v2 protocol. The exception handling module is used to switch to the auxiliary positioning mode based on visual SLAM when positioning packet loss or communication interruption occurs, enable relay communication with neighboring drones, and predict image compensation parameters based on Kalman filtering.

Images