A video optimization method and system based on wireless transmission

Abstract

The application discloses a video optimization method and system based on wireless transmission and belongs to the technical field of image communication. The application scheme achieves the effect of adaptive bandwidth allocation through signal strength monitoring, adopts multiple optimization modes, can effectively alleviate the image distortion and operation delay problems in wireless transmission, and can significantly improve the user experience, especially in the fields of cloud games, remote medical treatment, unmanned plane control and the like which have extremely high real-time requirements. The application combines a prediction algorithm, can obtain data for decoding more quickly through comparison and correction of actual frames and predicted frames, and adopts interframe interpolation, frame freezing, background filling and the like to slow down the perceptual loss.

Description

Technical Field

[0001] This invention belongs to the field of image communication technology, specifically relating to a video optimization method and system based on wireless transmission. Background Technology

[0002] During wireless transmission, it takes time for signals to travel between devices, and this transmission delay can lead to a series of visual and operational problems:

[0003] 1. Image Distortion: During wireless transmission, data packets may be lost or delayed due to interference, bandwidth limitations, or network congestion. Video data is typically transmitted in the form of frame sequences. When some frames are lost or arrive late, the receiving end may not be able to reconstruct the complete image in time, resulting in screen tearing, stuttering, or pixelation. Furthermore, to compensate for lost data, the system may employ error concealment techniques, but this can lead to a decrease in image quality, such as color shifts or image blurring.

[0004] 2. Delayed Operation: For remote control and real-time interactive applications, such as telemedicine, drone operation, or cloud gaming, transmission latency directly affects the real-time nature of operations. User-inputted commands need to be transmitted to the device via a wireless network before the feedback is sent back to the user. Excessive transmission latency can lead to a disconnect between operation and visual feedback. For example, a delay of several hundred milliseconds during drone flight could cause path misjudgment, or in cloud gaming, the character might not jump after the player presses the jump button, significantly reducing the user experience and even posing safety risks.

[0005] In summary, transmission latency not only affects visual quality but also reduces the accuracy and sensitivity of operations, especially in demanding real-time application scenarios where these problems are more pronounced. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention discloses a video optimization method and system based on wireless transmission, which optimizes from multiple levels and proposes intelligent scheduling and adaptive strategies to optimize wireless network protocols, improve bandwidth utilization, and reduce packet loss and latency (ACK).

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] This invention provides a video optimization method based on wireless transmission, comprising the following steps:

[0009] Step 1: The wireless device collects video data;

[0010] Step 2: The wireless device determines whether the frame rate used for transmission needs to be adjusted based on dynamic parameters. If adjustment is required, the frame group structure is simplified in the encoder. If no adjustment is required, the current encoder configuration is maintained.

[0011] Step 3: The Schottky detector circuit on the wireless device side obtains the current image transmission signal strength and determines whether the signal can be used for image transmission. If the current signal strength does not meet the requirements, the adaptive allocation method based on the Schottky diode detector circuit is used to adjust the gain or switch the channel. This step is repeated until it is determined to be successful.

[0012] Step 4: The Schottky detector circuit on the wireless device side obtains the current channel modulation state and determines whether it can maintain the high-order modulation state for transmission. If it determines that it cannot maintain the high-order modulation state, it will switch to low-order modulation and continue to send the image transmission signal. At the same time, it requests the encoder to readjust the dynamic frame rate and re-executes step 3. When the channel resources are sufficient, it maintains the high-order modulation state to transmit the image transmission signal to the ground.

[0013] Step 5: The Schottky detector circuit on the wireless device side obtains the current data transmission signal strength and determines whether it meets the requirements. If it does not meet the requirements, it performs gain adjustment or channel switching based on the adaptive allocation method implemented by the Schottky diode detector circuit. This step is repeated until it is determined that the signal is acceptable, and then the data transmission signal is sent to the ground. The ground returns data to the wireless device.

[0014] Step 6: Compare and correct the current actual frame data of the image transmission signal sent by the wireless device with the frame data predicted based on the previous frame. After the corrected data is decoded, it is rendered and output to the screen. The corrected image data is input into the prediction algorithm, and the prediction algorithm outputs the next frame prediction data.

[0015] Furthermore, in step 2, the frame rate used for transmission is adjusted when the dynamic parameters change to a preset condition.

[0016] Furthermore, in step 3, the adaptive allocation method based on the Schottky diode detection circuit includes: determining that when the interference signal is too large, the FPGA modifies the transmission frequency point to avoid the interference segment; determining that when the RF signal is too small, the gain of the VGA or LNA is modified; and when the signal after the gain adjustment is still too small, the FPGA adjusts the encoding parameters.

[0017] Furthermore, the situation in step 4 where the high-order modulation state cannot be maintained is when the signal is distorted.

[0018] Furthermore, in step 6, the image transmission data sent from the wireless device to the ground is demodulated and stored in the buffer. The next frame prediction data output by the prediction algorithm is transmitted to the buffer. The current actual frame data and the frame data predicted based on the previous frame are both extracted from the buffer.

[0019] Furthermore, the data transmission signals returned by the wireless device are input to the flight control system, which then provides variables to the prediction algorithm.

[0020] A video optimization system based on wireless transmission includes a wireless device and a flight control system. The wireless device includes a wireless transmission module, which is used to buffer video data collected from the network and sensors. After evaluating the data transmission and image transmission channels using an adaptive allocation method, it selects an appropriate channel or gain, adopts an appropriate channel modulation state, and selects an appropriate encoding and transmission method. The flight control system receives the data transmitted by the wireless transmission module from the remote controller receiver, and performs data comparison and calibration, image processing, prediction, image rendering, and final output.

[0021] The beneficial effects of this invention are as follows:

[0022] This invention achieves adaptive bandwidth allocation through signal strength monitoring and employs multiple optimization methods to effectively alleviate image distortion and delayed operation issues in wireless transmission. It significantly improves user experience, particularly in fields with extremely high real-time requirements such as cloud gaming, telemedicine, and drone control. Furthermore, this invention combines a prediction algorithm with comparison and correction between actual and predicted frames to obtain data for decoding more quickly. It also mitigates perceptual loss through inter-frame interpolation, frame freezing, and background filling. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the channel transmission between the drone and the remote controller.

[0024] Figure 2 A schematic diagram of some electronic modules of a drone for hardware improvements.

[0025] Figure 3 This is a schematic diagram of the channel adaptive allocation logic implemented based on a Schottky diode detection circuit.

[0026] Figure 4 This is a schematic diagram of the video optimization method based on wireless transmission provided by the present invention. Detailed Implementation

[0027] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0028] The solution of this invention is implemented based on the following optimization strategies:

[0029] 1. Network optimization steps, including:

[0030] Edge computing processes data at edge nodes closer to the user, reducing latency to and from the core network. For example, cloud gaming or remote control applications can quickly render visuals on a local server and transmit them back to the terminal.

[0031] Network slicing is employed. Based on 5G technology, network slicing can be used to create independent virtual networks according to application needs, providing dedicated resources for low-latency, high-bandwidth applications and reducing the risk of network congestion.

[0032] An adaptive bandwidth management method is adopted to dynamically adjust the transmission rate and adjust the video encoding quality and frame rate in real time according to the network status, so as to avoid delays or data loss caused by sudden traffic.

[0033] Taking a common drone operation model as an example, during flight, the drone and remote controller typically establish two sets of wireless communications: one is a data transmission signal used to control and monitor the drone's status, using a 2.4GHz Wi-Fi channel as an example. The other is an image transmission signal used to transmit real-time video, using a 5GHz Wi-Fi channel as an example. However, Wi-Fi signals are prone to signal fluctuations during transmission, such as... Figure 1 The diagram shows two types of channel fluctuations. This invention achieves adaptive bandwidth allocation through signal strength monitoring.

[0034] To achieve adaptive bandwidth allocation, optimization is needed at the wireless device level, requiring the addition of [specific components] to the device-side circuitry. Figure 2 The section within the red box represents the addition of a coupler, a Schottky diode detector circuit, and a control circuit, forming an automatic gain control system. When a decrease in received power is detected, the VGA gain is first increased to increase the transmitted power. Once the maximum transmitted power is reached, the received gain can be further increased. The Schottky diode detector circuit mainly consists of a matching circuit, a Schottky diode, and a filter. Due to the internal structure and low junction capacitance of the Schottky diode, its switching speed is much faster than that of a PN diode, and its reverse recovery time is extremely short. The filter's response time is related to the cutoff frequency and is also in the nanosecond range. Therefore, this invention can respond much faster and in real-time, avoiding situations where the received signal weakens when the drone encounters interference, resulting in delays ranging from tens to hundreds of milliseconds and video stuttering.

[0035] The logic of adaptive channel allocation based on Schottky diode detection circuit is as follows: Figure 3 As shown:

[0036] The RF signal is input to the detection circuit, and the output voltage is judged by a threshold. If the interference signal is too strong, the FPGA modifies the transmission frequency to avoid the interference band. If the signal is too weak, the gain of the VGA or LNA is modified. If the signal is still too weak after the gain adjustment, the FPGA will adjust the encoding parameters to strengthen the signal.

[0037] The detection circuit updates its status in real time. The received signal is amplified by the LNA and then passes through the coupler. The power output by the coupler enters the Schottky diode detection circuit. The detection circuit outputs a voltage, which enters the control circuit. The control circuit judges the corresponding signal magnitude and adjusts the VGA and LNA gain accordingly. After configuration, the signal loops back to the FPGA until the threshold is met.

[0038] Based on the above control logic, the present invention can analyze the wireless channel quality in real time based on the signal, avoid interference frequency bands in advance, or briefly switch to a low bit rate when interference suddenly increases to ensure smooth transmission.

[0039] The signal acquisition process on the drone side is as follows:

[0040] (1) Video acquisition and encoding: The drone camera acquires images and the video data is compressed by an encoder.

[0041] (2) Signal strength monitoring: Real-time detection of wireless signal strength, such as RSSI (Received Signal Strength Indicator) or signal-to-noise ratio.

[0042] (3) Adaptive controller decision: Based on the signal strength threshold, determine the bit rate and resolution adjustment strategy: when the signal is strong, maintain a high bit rate, high resolution and high frame rate (e.g. 1080p / 60fps), and when the signal is weak, use a low bit rate, reduce the resolution and frame rate, and reduce bandwidth usage (e.g. 720p / 30fps).

[0043] (4) Encoding parameter adjustment: Based on the decision results, dynamically modify the encoder parameters and reset the bit rate, resolution and frame rate.

[0044] (5) Video stream transmission: Real-time video stream is transmitted via wireless module to ensure smooth picture as much as possible in areas with weak signal.

[0045] 2. Video encoding optimization

[0046] Low-latency coding technology: Using high-efficiency encoders such as H.265 and AV1 can significantly reduce the bitrate while maintaining the same image quality. It also enables low-latency mode (ACK time) and communication protocol latency, reducing encoding and decoding processing time.

[0047] Forward Error Correction (FEC): Redundant data is added during transmission so that even if some data packets are lost, the receiving end can recover the complete image using error correction algorithms, reducing distortion caused by packet loss. Common methods include Raptor codes and LDPC.

[0048] Layered coding: The video stream is divided into a base layer and an enhancement layer. When the network conditions are poor, only the base layer is transmitted to ensure the minimum image quality and smoothness. Once the network recovers, image details are gradually added.

[0049] 3. Transmission protocol optimization

[0050] QUIC Protocol: The UDP-based QUIC protocol supports faster connection establishment and data recovery, making it more suitable for real-time interactive scenarios than traditional TCP.

[0051] RTSP / RTMP optimization: Optimize buffering strategies for real-time audio and video protocols to reduce unnecessary buffering time and avoid high latency caused by excessive accumulation.

[0052] Traffic priority scheduling: When transmitting multiple data streams, prioritize the processing of critical control commands and core video frames to ensure that operation commands and key images are delivered quickly.

[0053] 4. Device-side optimization

[0054] Hardware acceleration: Utilize dedicated decoding chips (such as GPUs and FPGAs) to speed up video processing and reduce processing latency on the device side.

[0055] Preloading and caching strategy: In a stable network environment, some data is pre-buried to appropriately balance latency and image continuity, avoiding severe stuttering caused by network fluctuations. The buffer size on the remote control is adjusted according to flight speed and network status. The buffer depth is reduced during high-speed flight to lower latency, while the buffer is increased during low-speed hovering to improve image quality stability. A preset speed threshold is used to distinguish between high and low-speed flight.

[0056] Intelligent latency compensation: In remote control scenarios, image algorithms are used to pre-render potential images. On the remote controller, sensor data and flight attitude are combined to predict the next frame and pre-render potential images, avoiding image stuttering due to network latency and reducing operator perception delay.

[0057] 5. Intelligent scheduling and adaptive strategies

[0058] By monitoring network status using sensor data returned by the drone, such as increased flight speed coupled with poor network speed, variables are provided to the image prediction algorithm, and it is determined whether image compensation is needed. The prediction algorithm can be trained using publicly available AI models.

[0059] Dynamic frame rate adjustment: In low-latency scenarios, the transmission frequency of non-critical frames is automatically reduced, and only critical action frames are retained to reduce the transmission burden.

[0060] These multi-layered optimization methods can effectively alleviate the problems of image distortion and untimely operation in wireless transmission, especially in fields with extremely high real-time requirements such as cloud gaming, telemedicine, and drone control, which can significantly improve the user experience.

[0061] Based on the above optimization strategies, this invention provides a video optimization system based on wireless transmission, including a wireless device (UAV) and a ground flight control system. The wireless device includes a wireless transmission module, which buffers video data collected from the network and sensors. After evaluating the channel using the aforementioned adaptive channel allocation method, it selects an appropriate channel or gain. An adaptive allocation method for the image transmission channel is used to monitor and adjust the signal, selecting suitable encoding and transmission methods. The flight control system receives data transmitted from the UAV transmission module via a remote controller receiver, and performs data comparison and calibration, image processing, prediction, image rendering, and final output.

[0062] This invention provides a video optimization method based on wireless transmission, the process of which is as follows: Figure 4 As shown, it includes the following steps:

[0063] Step 1: The wireless device (in this example, a drone) receives the video stream;

[0064] Step 2: The wireless device, combining dynamic parameters such as speed sensor, vertical sensor, and network status, determines whether the current frame rate used for transmission needs adjustment. If certain dynamic parameters change significantly, such as during acceleration, large-scale aircraft rotation, or network fluctuations, adjustment is deemed necessary. The frame group structure is simplified within the encoder. If the current dynamic parameter changes are not significant, the current encoder configuration continues to output H.265 compressed encoding. The prerequisite for frame rate adjustment is that specific dynamic parameter(s) (channel status, flight status, light changes, battery level, etc.) changes by a certain magnitude. These adjustment conditions should be preset.

[0065] Step 3: The Schottky detector circuit on the wireless device side obtains the current signal strength and determines whether the current signal is suitable for image transmission. If the current signal strength does not meet the requirements, such as a sudden drop or interruption, gain adjustment or channel switching will be performed based on adaptive allocation logic (refer to the aforementioned logic for adaptive channel allocation based on the Schottky diode detector circuit). The signal is then returned to the monitoring and judgment via the Schottky detector circuit until it is determined to be acceptable. This step should be performed periodically.

[0066] Step 4: The Schottky detector circuit on the wireless device side obtains the current channel modulation state, and the FPGA determines whether it can maintain transmission in a high-order modulation state. If signal distortion occurs due to channel resource occupation, it will be determined to switch to low-order modulation. At this time, the image transmission signal will continue to be transmitted, and a request will be made to the encoder to readjust the dynamic frame rate to ensure the continuity of the current image transmission. The signal strength judgment in Step 3 and the channel modulation judgment in Step 4 will be re-performed. When channel resources are sufficient, the image transmission signal will be transmitted to the ground remote controller while maintaining the high-order modulation state. This step should be performed periodically.

[0067] Step 5: The data transmission signal sent from the wireless device to the ground is assessed for signal strength in the same way as the image transmission channel strength assessment in Step 3. The data returned from the ground to the wireless device includes necessary UAV status information such as sensor data and periodic patrol commands, which are used by the remote controller for flight control analysis and prediction.

[0068] Step 6: The image transmission data sent from the wireless device to the ground, after demodulation, first arrives at the buffer, which contains at least four frames of image signals. These include the previous frame, the predicted current frame, the actual current frame (raw data received from the drone), and the predicted next frame. The current actual frame is compared and compensated with the data predicted based on the previous frame for data correction. The corrected data is then decoded, rendered, and output to the screen. The corrected image data is also input into the prediction algorithm, which predicts the next frame. The next frame is then transmitted to the buffer. The prediction algorithm, as an AI-assisted model (which can be trained using existing publicly available models), obtains data for decoding more quickly through comparison and correction between the actual and predicted frames. It uses inter-frame interpolation, frame freezing, and background filling to mitigate perception loss. The data transmission signal returned by the wireless device (drone) is input to the flight control system, providing variables to the prediction algorithm based on network status.

[0069] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A video optimization method based on wireless transmission, characterized in that, Includes the following steps: Step 1: The wireless device collects video data; Step 2: When the wireless device determines that the frame rate used for transmission needs to be adjusted based on the dynamic parameters, it simplifies the frame group structure configuration in the encoder. If no adjustment is needed, it continues to maintain the current encoder configuration output. Step 3: If the current image transmission signal strength does not meet the transmission requirements, the adaptive allocation method based on the Schottky diode detection circuit is used to adjust the gain or switch the channel. This step is repeated until it is determined to be successful. Step 4: If the current channel modulation state cannot be maintained in the high-order modulation state, it will switch to low-order modulation and continue to send the image transmission signal. At the same time, it will request the encoder to readjust the dynamic frame rate and repeat step 3. When the channel resources are sufficient, it will maintain the high-order modulation state and send the image transmission signal to the ground. Step 5: If the current data transmission signal strength does not meet the transmission requirements, the adaptive allocation method based on the Schottky diode detection circuit is used to adjust the gain or switch the channel. This step is repeated until it is determined that the signal is clear, and then the data transmission signal is sent to the ground; the ground returns data to the wireless device. Step 6: Compare and correct the current actual frame data of the image transmission data sent by the wireless device with the frame data predicted based on the previous frame. After the corrected data is decoded, it is rendered and output to the screen. The corrected image data is input into the prediction algorithm, which then outputs the prediction data for the next frame. The adaptive allocation method based on the Schottky diode detection circuit includes: when the interference signal is too large, the FPGA modifies the transmission frequency to avoid the interference segment; If the RF signal is too low, adjust the gain of the VGA or LNA. If the signal is still too low after the gain adjustment, adjust the encoding parameters of the FPGA.

2. The video optimization method based on wireless transmission according to claim 1, characterized in that, In step 2, the frame rate used for transmission is adjusted when the dynamic parameters change to the preset conditions.

3. The video optimization method based on wireless transmission according to claim 1, characterized in that, The reason why the high-order modulation state cannot be maintained in step 4 is that the signal is distorted.

4. The video optimization method based on wireless transmission according to claim 1, characterized in that, In step 6, the image transmission data sent from the wireless device to the ground is demodulated and stored in the buffer. The prediction algorithm outputs the next frame prediction data, which is transmitted to the buffer. The current actual frame data and the frame data predicted based on the previous frame are both extracted from the buffer.

5. A video optimization system based on wireless transmission, comprising a wireless device and a flight control system, characterized in that, To implement the video optimization method based on wireless transmission as described in any one of claims 1-4, the wireless device includes a wireless transmission module. The wireless transmission module is used to buffer video data collected from the network and sensors, evaluate the data transmission and image transmission channels using an adaptive allocation method, select an appropriate channel or gain, adopt an appropriate channel modulation state, and select an appropriate encoding and transmission method. The flight control system receives the data transmitted by the wireless transmission module based on the remote controller receiver, and performs data comparison and calibration, image processing, prediction, image rendering, and final output.

Images