Unmanned aerial vehicle low-delay high-definition image transmission method

Abstract

The invention provides a low-delay high-definition image transmission method for an unmanned aerial vehicle, and the method comprises the steps: enabling an image collection module to collect and process a video frame, enabling a coding module to carry out the H.265 hardware coding through an intra-frame refreshing mechanism, and storing a coding frame into an annular buffer region; the code rate control module monitors the frame number N of a buffer area in real time, and gradually adjusts the output code rate through a state machine; if N exceeds the high water level threshold value, the buffer area is emptied, and an emergency signal is triggered; the transmission module reads the data, dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiving end, and sends the data after encoding; receiving and trying FEC decoding by a receiving end; if the FEC decoding is successful, decoding a display picture and feeding back a packet loss rate; if the ARQ retransmission request fails, judging whether to initiate the ARQ retransmission request in combination with the timestamp, and if yes, requesting retransmission; and the sending end adjusts the FEC according to the feedback, responds to the emergency signal and continuously monitors the buffer area. According to the invention, the problems of coding delay, transmission reliability and congestion control lag are solved, and low-delay high-definition image transmission in a long-distance complex environment is realized.

Description

Technical Field

[0001] This invention relates to the field of wireless video transmission technology, and in particular to a method for low-latency high-definition image transmission from unmanned aerial vehicles (UAVs). Background Technology

[0002] High-definition digital image transmission systems for drones are the core components for enabling remote control and real-time video transmission of drones. As drone applications continue to expand, the industry is placing increasingly stringent demands on the transmission distance, image quality, and end-to-end latency of image transmission systems.

[0003] However, existing drone image transmission technology still has several shortcomings in addressing the above requirements, mainly in the following three aspects: 1. The widely adopted H.264 / H.265 encoding standard relies on periodic I-frames as the decoding reference. Because I-frames contain a massive amount of data, typically 5 to 10 times that of P-frames, their generation and transmission cause a momentary expansion of the encoder buffer, introducing an additional delay of 2-3 frames. Simultaneously, the resulting bitrate spikes can easily trigger network congestion, further degrading transmission performance.

[0004] 2. Existing solutions have limitations in terms of reliable transmission mechanisms. If relying solely on ARQ, a full RTT must be waited for retransmission after packet loss, significantly increasing end-to-end latency. If only FEC is used, a fixed redundant bandwidth must be preset, leading to wasted bandwidth resources under good channel conditions, while insufficient redundancy may prevent effective error correction under poor channel conditions.

[0005] 3. Traditional bitrate control strategies are mostly adjusted based on lagging indicators such as packet loss rate or RTT fed back from the receiver, which cannot accurately and in real time detect the backlog status of the sender's buffer. This feedback lag makes the system slow to react when network conditions fluctuate suddenly, which can easily lead to buffer overflow, video frame drops, and a sharp increase in latency.

[0006] In summary, existing technologies address the problem in a relatively isolated manner at each level, making collaboration difficult. Therefore, there is an urgent need for a drone image transmission system capable of multi-dimensional, in-depth collaborative optimization from encoding and transmission to congestion control, in order to ultimately achieve stable video transmission over long distances with low latency and high image quality. Summary of the Invention

[0007] To address the above issues, this invention employs intra-frame refresh coding, dynamic rate control based on a circular buffer level and state machine, and a collaborative design of a hybrid FEC and ARQ transmission protocol to resolve problems related to encoding latency, transmission reliability, and congestion control lag, thereby achieving low-latency high-definition image transmission in complex long-distance environments.

[0008] According to an embodiment of the present invention, a method for low-latency high-definition video transmission of an unmanned aerial vehicle is provided.

[0009] In a first aspect of the present invention, a method for low-latency high-definition video transmission of an unmanned aerial vehicle is provided. The method includes: Step S01: Start the sender and the receiver. The image acquisition module acquires and processes video frames. The encoding module performs H.265 hardware encoding using an intra-frame refresh mechanism and stores the encoded video frames in a circular buffer; Step S02: The bitrate control module monitors the number of frames in the buffer in real time, dynamically adjusts the encoding bitrate through a state machine model, and performs emergency processing when the number of frames exceeds the high watermark threshold; The transmission module reads the data, dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, and sends the data after encoding; Step S03: The receiver receives the data and attempts FEC decoding. If successful, it performs H.265 decoding and displays the result and feeds back the packet loss rate; If failed, it triggers the ARQ retransmission mechanism and judges the retransmission validity in combination with the timestamp; Step S04: The sender dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, and responds to the emergency event signal, and then returns to Step S02 to continue execution.

[0010] Further, the image acquisition module in Step S01 acquires video frame data and processes it through VI, ISP, and VPSS.

[0011] Further, the state machine model in Step S02 includes a hold state, a bitrate increase state, and a bitrate decrease state, and performs state transition according to the comparison result of the buffer frame number N with the target frame number threshold T_target and the high watermark threshold T_high.

[0012] Further, the state transition specifically includes: When in the hold state and N < T_target, start timing. If N continues to be less than T_target within the preset hold period, switch to the bitrate increase state; When in the hold state and N = T_target, immediately switch to the bitrate decrease state and record the current bitrate as the peak bitrate; When in the bitrate increase state and N < T_target, increase the bitrate by one step every preset bitrate increase interval period until the preset upper limit bitrate is reached; When in the bitrate increase state and N = T_target, immediately switch to the bitrate decrease state; When in the bitrate decrease state and N = T_target, decrease the bitrate by one step every preset bitrate decrease interval period until the preset lower limit bitrate is reached; When in the state of code rate reduction and N < T_target, switch to the hold state; When N ≥ T_high, the code rate control module triggers emergency processing, and the emergency processing includes: clearing the circular buffer, recording the number of emergencies, and sending an emergency event signal to the transmission module through the message queue.

[0013] Further, after receiving the emergency event signal, the transmission module performs at least one of the following operations: increasing the FEC redundancy, forcibly switching the working frequency point, and requesting the encoding module to send a key frame.

[0014] Further, the specific steps of step S03 are as follows: Step S031: The receiving end receives data through the wireless transceiver unit and attempts FEC decoding. If the decoding is successful, enter step S032; if the decoding fails, enter step S033; Step S032: Decode and display the video picture by H.265, and at the same time count the packet loss rate and feedback it to the sending end; Step S033: Trigger an ARQ retransmission request, and judge the retransmission validity in combination with the decoding timestamp of the video frame. If it is valid, send a retransmission request to the sending end, and the sending end retransmits the corresponding data packet and returns to step S031; if it is invalid, abandon the retransmission.

[0015] Further, the method for judging the retransmission validity in step S033 is: the receiving end calculates the remaining available time of the data packet to be retransmitted according to the decoding timestamp of the video frame. If the expected arrival time of the retransmitted packet exceeds the decoding time, abandon the retransmission request; When the transmission module triggers selective retransmission, it judges whether the data packet to be retransmitted is still valid in combination with the decoding timestamp of the video frame. If the expected arrival time has exceeded the decoding time, abandon the retransmission request.

[0016] Further, the method further includes a frequency hopping anti-jamming step: real-time monitoring of the RSSI and packet loss rate of each candidate frequency point. When the channel quality of the current working frequency point is lower than the preset threshold, dynamically switch to the optimal frequency point and achieve synchronous switching between the transceiver parties.

[0017] In the second aspect of the present invention, a device for low-latency high-definition image transmission of an unmanned aerial vehicle is provided. The device includes: An image acquisition module: configured at the sending end, including an image sensor, a VI capture unit with a MIPI interface, an ISP image processing unit, and a VPSS video processing unit, for collecting and processing video frame data; Encoding module: Connected to the image acquisition module, it is used to perform H.265 hardware encoding on video frame data. The encoding module adopts an intra-frame refresh mechanism to replace the traditional I-frame, evenly distributing the key frame refresh load to multiple P-frames, eliminating bit rate spikes and compressing the encoding buffer depth to 1 frame. Circular buffer: Connected to the encoding module, used to buffer encoded video frame data, with a preset capacity of 10 frames; Rate control module: connected to the circular buffer, used to monitor the number of frames N buffered in the circular buffer in real time, and gradually adjust the output rate of the encoding module through a state machine model based on the comparison result of the number of frames N with the preset threshold. Transmission module: Configured at the sending and receiving ends, it is used to transmit video data through a wireless channel. The transmission module adopts a hybrid transmission protocol of FEC and ARQ. It dynamically adjusts the forward error correction redundancy according to the real-time packet loss rate fed back by the receiving end, and triggers selective retransmission when FEC error correction fails. At the same time, it combines the video frame decoding timestamp to judge the validity of retransmission and avoids invalid retransmission from increasing latency. Wireless transceiver unit: Configured at the transmitting and receiving ends, responsible for the wireless transmission and reception of video data; Receiver processing module: Configured at the receiver, including FEC decoding unit, H.265 decoding unit, display unit and packet loss rate statistics feedback unit, used to receive video data, attempt FEC decoding, display H.265 decoding, and provide real-time statistics of packet loss rate feedback to the sender. Debugging and Feedback Interface: Configured between the sender and receiver, used to transmit packet loss rate feedback signals and ARQ retransmission requests.

[0018] The above-mentioned English abbreviations are explained as follows: MIPI: Mobile Industry Processor Interface VI: Video Input ISP: Image Signal Processor VPSS: Video Processing Sub-System H.264: H.264 / MPEG-4 Part 10 / Advanced Video Coding (AVC), Advanced Video Coding H.265: H.265 / High Efficiency Video Coding (HEVC), a high-efficiency video coding standard. I-frame: Intra-coded frame P-frame: Predictive-coded frame, forward-predicted frame FEC: Forward Error Correction ARQ: Automatic Repeat reQuest. RSSI: Received Signal Strength Indicator RTT: Round-Trip Time This invention solves the problems of encoding delay, transmission reliability, and congestion control lag by using intra-frame refresh coding, dynamic rate control based on ring buffer level and state machine, and collaborative design of FEC and ARQ hybrid transmission protocols, thereby achieving low-latency high-definition image transmission in long-distance complex environments.

[0019] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description.

[0020] The beneficial effects of this invention are: 1. Significantly reduced encoding latency: The intra-frame refresh mechanism replaces the traditional I-frame, eliminating bit rate spikes and encoding buffer bloat, compressing the encoding latency from 3 frames to less than 1 frame (16.7ms@60fps). 2. Early Congestion Detection and Controllable Delay: State machine rate control based on a circular buffer level detects congestion one RTT earlier than lag adjustments based on packet loss rate feedback, effectively preventing buffer overflow. In a 15km test, end-to-end latency remained stable at <100ms, with latency jitter reduced by 60%. 3. Enhanced transmission reliability: Under a 5% random packet loss environment, the FEC and ARQ hybrid transmission protocol can recover 95% of lost packets instantly by FEC without retransmission; the remaining lost packets are selectively retransmitted by ARQ, and the effective delay increment caused by retransmission is <10ms. 4. The three elements work together to form a closed-loop adaptive mechanism: the stable bitstream output by intra-frame refresh is beneficial to the efficiency of FEC; the circular buffer level feedback connects coding and transmission, enabling the bit rate control to be dynamically adjusted according to channel quality and buffer status; changes in FEC redundancy can also serve as a reference for bit rate adjustment. The three elements work together to achieve optimal overall system performance. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. Wherein: Figure 1 A flowchart of a method for low-latency high-definition image transmission by a drone according to an embodiment of the present invention is shown; Figure 2 A rate control state machine transition diagram according to an embodiment of the present invention is shown; Figure 3 A flowchart of a hybrid FEC and ARQ transport protocol according to an embodiment of the present invention is shown; Figure 4 A schematic diagram of the intra-frame refresh coding bitrate versus time curve according to an embodiment of the present invention is shown; Figure 5 This diagram illustrates the change in bitrate over time for traditional I-frame coding. Figure 6 A diagram of a device for low-latency high-definition image transmission for drones according to an embodiment of the present invention is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] According to an embodiment of the present invention, a method for low-latency high-definition image transmission of unmanned aerial vehicles (UAVs) is proposed. By using intra-frame refresh coding, dynamic rate control based on ring buffer level and state machine, and collaborative design of FEC and ARQ hybrid transmission protocols, the problems of coding delay, transmission reliability and congestion control lag are solved, thereby realizing low-latency high-definition image transmission in long-distance complex environments.

[0024] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.

[0025] Figure 1 This is a schematic flowchart of a method for low-latency high-definition image transmission from a drone according to an embodiment of the present invention. The method includes: Step S01: Start the sending and receiving ends. The image acquisition module acquires and processes video frames. The encoding module uses an intra-frame refresh mechanism to perform H.265 hardware encoding and stores the encoded video frames into a circular buffer. Step S02: The rate control module monitors the number of frames in the buffer in real time, dynamically adjusts the encoding rate through a state machine model, and performs emergency processing when the number of frames exceeds the high watermark threshold; the transmission module reads the data, dynamically adjusts the FEC redundancy based on the packet loss rate fed back by the receiver, and sends the encoded data. Step S03: The receiving end receives data and attempts FEC decoding. If successful, it performs H.265 decoding, displays the result, and reports the packet loss rate. If it fails, it triggers the ARQ retransmission mechanism and uses the timestamp to determine the validity of the retransmission. Step S04: The sending end dynamically adjusts the FEC redundancy based on the packet loss rate fed back by the receiving end, responds to the emergency event signal, and then returns to step S02 to continue execution.

[0026] It should be noted that although the operation of the method of the present invention has been described in a specific order in the above embodiments and figures, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0027] To provide a clearer explanation of the aforementioned method for low-latency high-definition image transmission from drones, a specific embodiment will be used for illustration below. However, it is worth noting that this embodiment is only for better illustrating the present invention and does not constitute an improper limitation of the present invention.

[0028] The following specific example will further illustrate the method of low-latency high-definition image transmission for drones: Step S01: Start the transmitting and receiving ends. The image acquisition module acquires and processes video frames, and the encoding module performs H.265 hardware encoding using an intra-frame refresh mechanism and stores the encoded video frames in a circular buffer. Specifically, the image acquisition module acquires video frame data and processes it through VI, ISP, and VPSS.

[0029] The intra-frame refresh mechanism includes: setting the refresh period to 30 frames, refreshing 1 / 30 of the screen area in each P-frame, and setting the encoding buffer depth to 1 frame.

[0030] Step S02: The rate control module monitors the number of frames in the buffer in real time, dynamically adjusts the encoding rate through a state machine model, and performs emergency processing when the number of frames exceeds the high watermark threshold; the transmission module reads the data, dynamically adjusts the FEC redundancy based on the packet loss rate fed back by the receiver, and sends the encoded data.

[0031] Specifically, the rate control module monitors the number of frames N buffered in the circular buffer in real time and progressively adjusts the output rate of the encoding module using a state machine model based on the comparison between N and a preset threshold. When N exceeds the high-water mark threshold, emergency processing is triggered, the circular buffer is cleared, and an emergency event signal is sent. The transmission module reads video data from the circular buffer, dynamically adjusts the FEC redundancy based on the real-time packet loss rate fed back from the receiver, performs FEC encoding, and then transmits the data through the wireless transceiver unit.

[0032] The state machine model includes a holding state, a code rate increasing state, and a code rate decreasing state, and performs state transitions according to the comparison results of the cached frame number N with the target frame number threshold T_target and the high water level threshold T_high.

[0033] When in the holding state and N < T_target, start timing. If N remains less than T_target within the preset holding period, switch to the code rate increasing state; When in the holding state and N = T_target, immediately switch to the code rate decreasing state, and record the current code rate as the peak code rate; When in the code rate increasing state and N < T_target, increase the code rate by one step every preset code rate increasing interval period until the preset upper limit code rate is reached; When in the code rate increasing state and N = T_target, immediately switch to the code rate decreasing state; When in the code rate decreasing state and N = T_target, decrease the code rate by one step every preset code rate decreasing interval period until the preset lower limit code rate is reached; When in the code rate decreasing state and N < T_target, switch to the holding state; When N ≥ T_high, the code rate control module triggers emergency handling, and the emergency handling includes: clearing the circular buffer, recording the number of emergencies, and sending an emergency event signal to the transmission module through the message queue.

[0034] The code rate increasing interval period and the code rate decreasing interval period are both 100 ms, the code rate step is 500 kbps, the upper limit code rate is 20 Mbps, and the lower limit code rate is 2 Mbps.

[0035] After receiving the emergency event signal, the transmission module performs at least one of the following operations: increasing the FEC redundancy, forcibly switching the working frequency point, and requesting the encoding module to send a key frame.

[0036] The receiving end statistically calculates the packet loss rate in real time and feeds it back to the sending end. The sending end adjusts the redundancy according to the packet loss rate. The higher the packet loss rate, the higher the redundancy, and the redundancy adjustment range is 5% - 20%.

[0037] Step S03: The receiving end receives data and attempts FEC decoding. If successful, perform H.265 decoding and display and feed back the packet loss rate; if failed, trigger the ARQ retransmission mechanism and judge the retransmission validity in combination with the time stamp.

[0038] The specific steps are as follows: Step S031: The receiving end receives data through the wireless transceiver unit and attempts FEC decoding. If the decoding is successful, enter step S032; if the decoding fails, enter step S033.

[0039] Step S032: H.265 decoding and display of video footage, while simultaneously calculating packet loss rate and feeding it back to the sending end.

[0040] Step S033: Trigger an ARQ retransmission request, and determine the validity of the retransmission by combining the video frame decoding timestamp. If valid, send a retransmission request to the sender, and the sender retransmits the corresponding data packet and returns to step S031; if invalid, abandon the retransmission.

[0041] The method for determining the validity of retransmission is as follows: the receiving end calculates the remaining available time of the data packet to be retransmitted based on the decoding timestamp of the video frame. If the expected arrival time of the retransmission packet exceeds the decoding time, the retransmission request is abandoned.

[0042] When the transmission module triggers selective retransmission, it determines whether the data packet to be retransmitted is still valid by combining the decoding timestamp of the video frame. If the expected arrival time has exceeded the decoding time, the retransmission request is abandoned to avoid invalid retransmissions that increase latency.

[0043] Step S04: The sending end dynamically adjusts the FEC redundancy based on the packet loss rate fed back by the receiving end, responds to the emergency event signal, and then returns to step S02 to continue execution.

[0044] Specifically, it also includes frequency hopping anti-interference steps: real-time monitoring of RSSI and packet loss rate of each candidate frequency point; when the channel quality of the current working frequency point is lower than the preset threshold, dynamic switching to the optimal frequency point is achieved, and synchronous switching is realized between the transmitting and receiving parties.

[0045] This embodiment uses a Hisilicon hi3519dv500 processor, an OS04A10 image sensor, and a wireless transmission module that operates in a dual-band 2.4GHz / 5.8GHz. The ground receiving platform is an RK3588 development board.

[0046] 1. Image Acquisition Module Configuration: The OS04A10 image sensor is connected to the VI unit of the hi3519dv500 via a MIPI interface. After capturing the raw image, the VI sends it to the ISP for image optimization (including automatic exposure, automatic white balance, dehazing, etc.), and then it is scaled and format-converted by VPSS to output 1080p@60fps YUV image data. The VPSS output is directly bound to the input of the VENC module.

[0047] 2. Encoding Module Configuration: In the VENC module of the hi3519dv500, intra-frame refresh mode was enabled, with a refresh period set to 30 frames per second (fps). This means a full frame refresh is completed every 30 fps, and 1 / 30 of the screen area is refreshed per P-frame. The encoding buffer depth was set to 1 frame, resulting in a stable encoding output stream with no significant bitrate spikes. In actual testing at 1080p@60fps, the encoding latency remained stable at approximately 16ms.

[0048] 3. Circular Buffer and Rate Control Module Configuration: A circular buffer with a capacity of 10 frames is allocated in the sender's memory to buffer encoded video frames. The rate control module runs in an independent thread, performing an adjustment cycle every 100ms. The target frame rate threshold T_target is set to 1 frame, the high watermark threshold T_high to 5 frames, and the maximum watermark threshold T_max to 8 frames. The state machine logic of the rate control module is as follows: Figure 2 As shown, the specific parameters are as follows: Hold period: corresponds to 10 seconds (i.e. 100 adjustment periods). When N<1 frames last for 10 seconds, it enters the upcoding state. Bitrate increment interval: 2 seconds (i.e., 20 adjustment cycles), the bitrate is increased by 500kbps every 2 seconds until the upper limit of 20Mbps is reached; Bitrate reduction interval: The bitrate is reduced by 500kbps every 100ms until the lower limit of 2Mbps is reached; Emergency handling: When N≥5 frames, immediately clear the circular buffer, record the number of emergency events, and send an emergency event signal through the message queue; when N≥8 frames, in addition to clearing the buffer, forcibly reduce the encoding frame rate to 30fps.

[0049] 4. Transmission module configuration: such as Figure 3 As shown, the transmission module uses RS(255,223) encoding as the basis for FEC, meaning that 32 bytes of redundancy are generated for every 223 bytes of original data, with a redundancy of approximately 14%. The receiving end calculates the packet loss rate every 100ms and feeds it back to the sending end. The sending end dynamically adjusts the redundancy based on the packet loss rate. Packet loss rate <3%: Redundancy reduced to 5% (using RS(255,243)); Packet loss rate 3%~10%: Redundancy maintained at 14%; Packet loss rate > 10%: Redundancy increased to 20% (using RS(255,212)).

[0050] When FEC decoding fails, the receiver sends a NACK request for retransmission. The retransmission control unit determines the retransmission time based on the PTS (Display Timestamp) of the video frame. If the expected arrival time of the retransmission packet has exceeded the decoding time, the request is discarded to avoid invalid retransmission.

[0051] 5. Receiver Configuration: After receiving data, the receiver's wireless module sends it to the FEC decoding unit to attempt recovery. If successful, it sends it to the H.265 decoder (RK platform hardware decoding) for display; if it fails, it triggers an ARQ retransmission request. Simultaneously, the packet loss rate statistics unit calculates the packet loss rate every 100ms and sends it to the sender via the feedback link.

[0052] 6. Test Results A 15km range test was conducted in an open area, with the drone flying at an altitude of 200 meters and a speed of 15m / s. The test results are as follows: End-to-end latency (from sensor exposure to ground-side decoding and display): average 92ms, maximum 118ms, minimum 78ms; In the 5% random packet loss injection test, the screen was smooth and there was no lag or stuttering. The subjective image quality score was 4.8 out of 5. The bitrate adjustment is smooth, with no noticeable image quality jitter.

[0053] Table 1 shows the performance comparison data between the present invention and the prior art. It can be seen that the present invention has better improvements in end-to-end latency, encoding buffer depth, number of retransmissions under 5% packet loss, and bitrate fluctuation.

[0054] Table 1 Table 2 shows the packet loss resistance performance of this invention under different FEC redundancy levels. It can be seen that as the FEC redundancy increases, the data recovery rate at each packet loss rate significantly improves: in a 5% packet loss environment, when the redundancy increases from 5% to 20%, the recovery rate rises from 98.5% to 99.9%, achieving almost lossless transmission; in a 10% packet loss environment, the recovery rate jumps from 76.2% to 99.2%; even in an extreme packet loss environment of 15%, 20% redundancy still maintains a high recovery rate of 95.4%. More importantly, this invention adopts a strategy of dynamically adjusting the FEC redundancy based on the real-time packet loss rate. When the channel is good, a low redundancy of 5%~10% is used to save bandwidth; when the channel deteriorates, it automatically increases to a high redundancy of 15%~20%, ensuring transmission reliability in high packet loss environments. Compared to a fixed redundancy scheme, this adaptive mechanism achieves the best balance between bandwidth utilization and packet loss resistance while ensuring low latency.

[0055] Table 2 like Figure 4 and Figure 5 As shown, the curves of the intra-frame refresh coding bitrate of the present invention and the traditional I-frame coding bitrate over time are respectively. It can be seen that the intra-frame refresh mechanism of the present invention has the following beneficial effects: (1) The bitrate stability is significantly improved, and the fluctuation amplitude is reduced from 2.28 times that of the traditional scheme to 1.40 times, a reduction of about 38.6%; (2) Periodic bitrate spikes are eliminated, and the coding buffer depth is compressed from 3 frames to 1 frame, reducing the coding delay by about 66%; (3) Stable bitstream output is conducive to the dynamic adjustment of FEC redundancy of subsequent transmission modules, avoiding network congestion and retransmission requests caused by bitrate mutations, and indirectly improving transmission reliability.

[0056] Based on the same inventive concept, this invention also proposes a device for low-latency high-definition image transmission for unmanned aerial vehicles (UAVs). The implementation of this device can be found in the implementation of the method described above; repeated details will not be repeated. Figure 6 As shown, the device includes: Image acquisition module: Configured on the transmitting end, it includes an image sensor, a MIPI interface VI capture unit, an ISP image processing unit, and a VPSS video processing unit, used to acquire and process video frame data; Encoding module: Connected to the image acquisition module, it is used to perform H.265 hardware encoding on video frame data. The encoding module adopts an intra-frame refresh mechanism to replace the traditional I-frame, evenly distributing the key frame refresh load to multiple P-frames, eliminating bit rate spikes and compressing the encoding buffer depth to 1 frame. Circular buffer: Connected to the encoding module, used to buffer encoded video frame data, with a preset capacity of 10 frames; Rate control module: connected to the circular buffer, used to monitor the number of frames N buffered in the circular buffer in real time, and gradually adjust the output rate of the encoding module through a state machine model based on the comparison result of the number of frames N with the preset threshold. Transmission module: Configured at the sending and receiving ends, it is used to transmit video data through a wireless channel. The transmission module adopts a hybrid transmission protocol of FEC and ARQ. It dynamically adjusts the forward error correction redundancy according to the real-time packet loss rate fed back by the receiving end, and triggers selective retransmission when FEC error correction fails. At the same time, it combines the video frame decoding timestamp to judge the validity of retransmission and avoids invalid retransmission from increasing latency. Wireless transceiver unit: Configured at the transmitting and receiving ends, responsible for the wireless transmission and reception of video data; Receiver processing module: Configured at the receiver, including FEC decoding unit, H.265 decoding unit, display unit and packet loss rate statistics feedback unit, used to receive video data, attempt FEC decoding, display H.265 decoding, and provide real-time statistics of packet loss rate feedback to the sender. Debugging and Feedback Interface: Configured between the sender and receiver, used to transmit packet loss rate feedback signals and ARQ retransmission requests.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0058] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0059] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for low-latency high-definition image transmission from a drone, characterized in that, The method includes: Step S01: Start the sender and the receiver. The image acquisition module acquires and processes video frames. The encoding module performs H.265 hardware encoding using the intra-frame refresh mechanism and stores the encoded video frames into the circular buffer. Step S02: The bitrate control module monitors the number of frames in the buffer in real time, dynamically adjusts the encoding bitrate through the state machine model, and performs emergency processing when the number of frames exceeds the high watermark threshold. The transmission module reads the data, dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, and sends the data after encoding. Step S03: The receiver receives the data and attempts FEC decoding. If successful, it performs H.265 decoding and displays the video, and feeds back the packet loss rate. If failed, it triggers the ARQ retransmission mechanism and judges the retransmission validity in combination with the timestamp. Step S04: The sender dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, responds to the emergency event signal, and then returns to Step S02 to continue execution.

2. The method for low-latency high-definition image transmission from a UAV according to claim 1, characterized in that, In Step S01, the image acquisition module acquires video frame data and processes it through VI, ISP, and VPSS.

3. The method for low-latency high-definition image transmission from a UAV according to claim 1, characterized in that, The state machine model in Step S02 includes a holding state, a bitrate increasing state, and a bitrate decreasing state, and performs state transition according to the comparison results of the number of cached frames N with the target frame number threshold T_target and the high watermark threshold T_high.

4. The method for low-latency high-definition image transmission from a UAV according to claim 3, characterized in that, The specific state transition includes: When in the holding state and N < T_target, start timing. If N continues to be less than T_target within the preset holding period, switch to the bitrate increasing state. When in the holding state and N = T_target, immediately switch to the bitrate decreasing state and record the current bitrate as the peak bitrate. When in the bitrate increasing state and N < T_target, increase the bitrate by one step every preset bitrate increasing interval period until the preset upper limit bitrate is reached. When in the bitrate increasing state and N = T_target, immediately switch to the bitrate decreasing state. When in the bitrate decreasing state and N = T_target, decrease the bitrate by one step every preset bitrate decreasing interval period until the preset lower limit bitrate is reached. When in the bitrate decreasing state and N < T_target, switch to the holding state. When N ≥ T_high, the bitrate control module triggers emergency processing, which includes: clearing the circular buffer, recording the emergency count, and sending an emergency event signal to the transmission module through the message queue.

5. The method for low-latency high-definition image transmission from a UAV according to claim 4, characterized in that, After receiving the emergency event signal, the transmission module performs at least one of the following operations: increasing the FEC redundancy, forcibly switching the working frequency point, and requesting the encoding module to send a key frame.

6. The method for low-latency high-definition image transmission from a UAV according to claim 1, characterized in that, The specific steps of Step S03 are: Step S031: The receiver receives the data through the wireless transceiver unit and attempts FEC decoding. If the decoding is successful, it enters Step S032; if the decoding fails, it enters Step S033. Step S032: Perform H.265 decoding to display the video picture, and at the same time count the packet loss rate and feed it back to the sender. Step S033: Trigger an ARQ retransmission request, and determine the validity of the retransmission by combining the video frame decoding timestamp. If valid, send a retransmission request to the sender, and the sender retransmits the corresponding data packet and returns to step S031; if invalid, abandon the retransmission.

7. The method for low-latency high-definition image transmission from a UAV according to claim 6, characterized in that, The method for determining the validity of retransmission in step S033 is as follows: the receiving end calculates the remaining available time of the data packet to be retransmitted based on the decoding timestamp of the video frame. If the expected arrival time of the retransmission packet exceeds the decoding time, the retransmission request is abandoned. When the transmission module triggers selective retransmission, it determines whether the data packet to be retransmitted is still valid by combining the decoding timestamp of the video frame. If the expected arrival time has exceeded the decoding time, the retransmission request is abandoned.

8. The method for low-latency high-definition image transmission from a UAV according to claim 1, characterized in that, The method also includes a frequency hopping anti-interference step: real-time monitoring of RSSI and packet loss rate of each candidate frequency point; when the channel quality of the current working frequency point is lower than a preset threshold, dynamic switching to the optimal frequency point is performed, and synchronous switching is achieved between the transmitting and receiving parties.

9. A device for low-latency high-definition image transmission for unmanned aerial vehicles (UAVs), characterized in that, The device includes: Image acquisition module: Configured on the transmitting end, it includes an image sensor, a MIPI interface VI capture unit, an ISP image processing unit, and a VPSS video processing unit, used to acquire and process video frame data; Encoding module: Connected to the image acquisition module, it is used to perform H.265 hardware encoding on video frame data. The encoding module adopts an intra-frame refresh mechanism to replace the traditional I-frame, evenly distributing the key frame refresh load to multiple P-frames, eliminating bit rate spikes and compressing the encoding buffer depth to 1 frame. Circular buffer: Connected to the encoding module, used to buffer encoded video frame data, with a preset capacity of 10 frames; Rate control module: connected to the circular buffer, used to monitor the number of frames N buffered in the circular buffer in real time, and gradually adjust the output rate of the encoding module through a state machine model based on the comparison result of the number of frames N with the preset threshold. Transmission module: Configured at the sending and receiving ends, it is used to transmit video data through a wireless channel. The transmission module adopts a hybrid transmission protocol of FEC and ARQ. It dynamically adjusts the forward error correction redundancy according to the real-time packet loss rate fed back by the receiving end, and triggers selective retransmission when FEC error correction fails. At the same time, it combines the video frame decoding timestamp to judge the validity of retransmission and avoids invalid retransmission from increasing latency. Wireless transceiver unit: Configured at the transmitting and receiving ends, responsible for the wireless transmission and reception of video data; Receiver processing module: Configured at the receiver, including FEC decoding unit, H.265 decoding unit, display unit and packet loss rate statistics feedback unit, used to receive video data, attempt FEC decoding, display H.265 decoding, and provide real-time statistics of packet loss rate feedback to the sender. Debugging and Feedback Interface: Configured between the sender and receiver, used to transmit packet loss rate feedback signals and ARQ retransmission requests.

Images