Unmanned aerial vehicle channel-aware adaptive image transmission device

Abstract

The application relates to a kind of unmanned aerial vehicle channel sensing adaptive image transmission devices.The device includes: radio frequency signal detection module real-time detection current transmission channel parameter;Image acquisition module uses the real-time image data obtained by camera, and the image data is denoised and output;Image detection and encoding control module detects and analyzes whether obvious abnormal image appears to the image data, and combines the current transmission channel parameter, and dynamically adjusts encoding parameter to carry out encoding control and frame rate control;Wireless transmission module carries out real-time transmission to the image data after encoding processing through radio frequency module, can sense channel state in real time, automatically adjusts transmission code rate and frame rate according to complex environment and change scene, to meet the image transmission requirements of different environments, automatically adapt transmission bandwidth can slow down the overall energy consumption, can better realize low delay transmission.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) image transmission technology, and in particular to a UAV channel-aware adaptive image transmission device. Background Technology

[0002] With technological advancements, drone aerial photography has entered the era of intelligence. Features such as real-time image transmission, autonomous obstacle avoidance, and one-click panorama have demystified aerial photography, making it easy for beginners to quickly master and significantly improving safety. Moreover, drones are small, portable, and can be taken off anytime, anywhere, truly achieving flexibility and convenience in aerial photography. With its comprehensive advantages, drone aerial photography has revolutionized the field, and it will continue to soar, creating even more possibilities in the future.

[0003] Drone image transmission involves wirelessly transmitting video captured by a camera to a remote location in real time. This transmission is achieved via an image transmission antenna and a control terminal. Currently, the most widely used transmission technologies for drones include Orthogonal Frequency Division Multiplexing (OFDM), Wireless Fidelity (Wi-Fi), and real-time image transmission (Lightbridge, DJI) high-definition long-range digital image transmission technology.

[0004] Existing UAV image transmission technologies employ simple signal strength grading control, failing to comprehensively consider multiple channel parameters. Alternatively, they may implement bit rate adjustment but not optimize image coding parameters. The quality of wireless channels changes dynamically, and fixed transmission parameters cannot adapt. Traditional solutions rely on preset threshold adjustments and lack precise quantization models. Image quality deteriorates significantly in weak signal environments, resulting in unstable image transmission quality in dynamic wireless environments.

[0005] In summary, there is an urgent need for a UAV channel-aware adaptive image transmission device that can perceive channel conditions in real time and automatically optimize image transmission parameters. Utility Model Content

[0006] Therefore, it is necessary to provide a UAV channel-aware adaptive image transmission device that can perceive channel status in real time and automatically optimize image transmission parameters to address the above-mentioned technical problems.

[0007] A channel-aware adaptive image transmission device for unmanned aerial vehicles (UAVs), the device comprising:

[0008] Radio frequency signal detection module for real-time detection of current transmission channel parameters;

[0009] An image acquisition module used to denoise and output real-time image data acquired by a camera;

[0010] An image detection and encoding control module is used to detect and analyze the image data to see if there are obvious abnormal images, and to adaptively and dynamically adjust the encoding parameters to control encoding and frame rate in combination with the current transmission channel parameters.

[0011] A wireless transmission module for transmitting the encoded image data in real time via a radio frequency module.

[0012] In one embodiment, the image acquisition module is connected to the camera via an HDMI, USB, or MIPI interface.

[0013] In one embodiment, the MIPI output interface of the image acquisition module and the MIPI input interface of the image detection and encoding control module are connected.

[0014] In one embodiment, the SPI interface of the radio frequency signal detection module is connected to the SPI interface of the main controller.

[0015] In one embodiment, the USB interface of the wireless transmission module is connected to the USB interface of the image detection and encoding control module.

[0016] In one embodiment, the image detection and encoding control module includes a Y component extraction submodule and an image distortion detection calculation submodule.

[0017] In one embodiment, the image detection and encoding control module further includes an encoding decision submodule and an encoding execution submodule.

[0018] The aforementioned UAV channel-aware adaptive image transmission device uses a radio frequency signal detection module to detect the current transmission channel parameters in real time. These parameters include, but are not limited to, received signal strength, signal-to-noise ratio, and packet error rate. An image acquisition module uses real-time image data acquired by a camera to denoise the image data and output it. An image detection and encoding control module analyzes the image data to detect any obvious abnormal images. Based on the current transmission channel parameters, it adaptively and dynamically adjusts the encoding parameters for encoding control and frame rate control. Abnormal images are those containing pixelation or / and mosaic effects. A wireless transmission module transmits the encoded image data in real time via a radio frequency module. This device can perceive the channel status in real time and automatically adjust the transmission bit rate and frame rate according to complex environments and changing scenarios to meet the image transmission requirements of different environments. Automatically adapting the transmission bandwidth reduces overall energy consumption and better achieves low-latency transmission. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the module architecture of a UAV channel-aware adaptive image transmission device in one embodiment.

[0020] Figure 2 This is a schematic diagram of the radio frequency signal detection module in one embodiment;

[0021] Figure 3 This is a schematic diagram of the image detection and encoding control module in one embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] In one embodiment, such as Figure 1 As shown, a UAV channel-aware adaptive image transmission device is provided, comprising: a radio frequency signal detection module 11, an image acquisition module 12, an image detection and encoding control module 13, and a wireless transmission module 14, wherein:

[0024] The radio frequency signal detection module 11 has its SPI interface connected to the SPI interface of the main controller. It is used to detect the current transmission channel parameters in real time. These channel parameters include, but are not limited to, received signal strength (RSSI), signal-to-noise ratio (SINR), and packet error rate (PER). The hardware block diagram of the radio frequency signal detection module is shown below. Figure 2 As shown; when the received signal strength RSSI is greater than -75dBm, the link transmission is within the normal range; when the received signal strength RSSI is less than -85dBm, the risk of link interruption increases; when the signal-to-noise ratio SINR is greater than 15dB, it is within the normal range; when the signal-to-noise ratio SINR is less than 10dB, the bit error rate increases exponentially; when the packet error rate PER is less than 5%, it is within the acceptable range; when the packet error rate PER is greater than 15%, video stuttering / screen tearing will occur.

[0025] The image acquisition module 12 is connected to the camera via an HDMI, USB, or MIPI interface. The MIPI output interface of the image acquisition module is connected to the MIPI input interface of the image detection and encoding control module. It is used to perform noise reduction processing on the real-time image data acquired by the camera and output the output video signal to provide a real-time data source for subsequent modules.

[0026] The image detection and encoding control module 13 is used to detect and analyze the image data to determine if any obvious abnormal images appear. Based on the current transmission channel parameters, it adaptively and dynamically adjusts the encoding parameters for encoding control and frame rate control. The abnormal images refer to images containing artifacts such as pixelation, mosaic, green screen, and image tearing. The hardware block diagram of the image detection and encoding control module is shown below. Figure 3 As shown, the modules include a Y component extraction submodule, an image distortion detection calculation submodule, an encoding decision submodule, and an encoding execution submodule.

[0027] The wireless transmission module 14 has its USB interface connected to the USB interface of the image detection and encoding control module. It is used to transmit the encoded image data in real time via the radio frequency module. The encoded image data is H.264 / H.265 encoded. By transmitting the H.264 / H.265 encoded data processed by the image detection and encoding control module in real time via the radio frequency module, the user can receive, decode, and display the real-time streaming video at the receiving end.

[0028] Furthermore, the image detection and encoding control module includes a Y component extraction submodule and an image distortion detection and calculation submodule, wherein...

[0029] The Y-component extraction submodule is used to extract image brightness, where image brightness is the Y component of image pixels.

[0030] Y(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y)

[0031] Where (x,y) is the coordinate position of the current pixel, Y(x,y) is the Y component value at (x,y), R(x,y) is the red component value at (x,y), G(x,y) is the green component value at (x,y), and B(x,y) is the blue component value at (x,y).

[0032] The image distortion detection calculation submodule is used to calculate the second-order mixture partial derivative of each pixel in the image based on the image brightness.

[0033] G xy (x,y)=(1 / 4)*[-I(x-1,y-1)+I(x+1,y-1)+I(x-1,y+1)-I(x+1,y+1)]

[0034] Where x and y are the coordinates of the current pixel, I is the grayscale value or pixel value of the image, and G... xy (x,y) is the second-order mixed partial derivative;

[0035] The image distortion detection calculation submodule is further configured to calculate an evaluation score based on the second-order mixed partial derivative of each pixel, and to determine whether the image data contains obvious abnormal images.

[0036]

[0037] Where BlockScore is the evaluation score for identifying block artifacts, N is the number of valid block boundary points, W is the image width, H is the image height, and Mask(x,y) is the block boundary mask function; Mask(x,y) is 1 if it is at the block boundary, and 0 otherwise. The changes in local blocks in the image (via G...) are then analyzed. xy The block artifact score in an image is calculated by combining the block artifacts measured by (x,y) and the masking function M(x,y). This is achieved by weighted summation of the contributions of all pixels, followed by normalization, to obtain a metric used to assess the presence of significant block artifacts in the image. A higher block artifact score indicates the presence of more obvious block artifacts. Preferably, an evaluation score greater than 0.35 indicates a significant image anomaly.

[0038] Furthermore, the image detection and encoding control module also includes an encoding decision submodule and an encoding execution submodule, wherein...

[0039] The coding decision submodule is used to calculate the overall quality index (QI) based on the current transmission channel parameters.

[0040] QI = w score ·BlockScore+w r ·R'+w s ·S'+w p ·P'

[0041] Among them, w score It is the image detection distortion weight, w r It is the received signal strength weight, w s Signal-to-interference-plus-noise ratio weight, w p Packet error rate, R' is the received signal strength normalized in [-100, 0], S' is the signal-to-interference-plus-noise ratio normalized in [0, 30], and P' is the packet error rate normalized in [0, 100].

[0042] The coding decision submodule is also used to determine the block artifact-channel coupling factor λ.

[0043]

[0044] Where k = 8, S is the signal-to-interference-plus-noise ratio (SINR), and e is the natural constant;

[0045] The coding decision submodule is further configured to calculate the target code rate B based on the comprehensive quality index and the block artifact-channel coupling factor. t ,

[0046] B t =B max ×[α·QI+(1-α)·λ]×Γ

[0047] Among them, B max Given the theoretical maximum bitrate at the current resolution, the dominant weight of the quality index is α = 0.6. When the BlockScore is less than 0.3, Γ = 1.2; when the BlockScore is greater than 0.7, Γ = 0.7; otherwise, Γ = 1 - BlockScore * 0.4.

[0048] The encoding execution submodule is used to adjust the target frame rate according to the target bitrate.

[0049] Specifically, the encoding execution submodule is used for:

[0050] When the overall quality index is greater than or equal to 0.7, the target frame rate is set to 30 frames per second.

[0051] When the overall quality index is greater than or equal to 0.5 and less than 0.7, the target frame rate is set to 25 frames per second.

[0052] When the overall quality index is less than 0.5, the target frame rate (fps) = 15 + 10 x λ. After calculating the current bitrate and frame rate using the above method, adjusting the encoding parameters accordingly can achieve the goal of dynamically optimizing and adjusting the image transmission quality.

[0053] In the aforementioned UAV channel-aware adaptive image transmission device, the radio frequency signal detection module detects the current transmission channel parameters in real time. These channel parameters include, but are not limited to, received signal strength, signal-to-noise ratio, and packet error rate. The image acquisition module uses real-time image data acquired by the camera to perform noise reduction processing on the image data and outputs it. The image detection and encoding control module detects and analyzes the image data to check for obvious abnormal images. Based on the current transmission channel parameters, it adaptively and dynamically adjusts the encoding parameters for encoding control and frame rate control. The abnormal images are those containing screen tearing or / and mosaic patterns. The wireless transmission module transmits the encoded image data in real time through the radio frequency module. This device can perceive the channel status in real time and automatically adjust the transmission bit rate and frame rate according to complex environments and changing scenarios to meet the image transmission requirements of different environments. Automatically adapting the transmission bandwidth can reduce overall energy consumption and better achieve low-latency transmission.

[0054] Specific limitations regarding the UAV channel-aware adaptive image transmission device can be found in the limitations of the UAV channel-aware adaptive image transmission method below, and will not be repeated here. Each module in the aforementioned UAV channel-aware adaptive image transmission device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0055] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0056] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the utility model patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A UAV channel-aware adaptive image transmission device, characterized in that, The device includes: Radio frequency signal detection module for real-time detection of current transmission channel parameters; An image acquisition module used to denoise and output real-time image data acquired by a camera; An image detection and encoding control module is used to detect and analyze the image data to see if there are obvious abnormal images, and to adaptively and dynamically adjust the encoding parameters to control encoding and frame rate in combination with the current transmission channel parameters. A wireless transmission module for transmitting the encoded image data in real time via a radio frequency module.

2. The apparatus according to claim 1, characterized in that, The image acquisition module is connected to the camera via an HDMI, USB, or MIPI interface.

3. The apparatus according to claim 1, characterized in that, The MIPI output interface of the image acquisition module is connected to the MIPI input interface of the image detection and encoding control module.

4. The apparatus according to claim 1, characterized in that, The SPI interface of the radio frequency signal detection module is connected to the SPI interface of the main controller.

5. The apparatus according to claim 1, characterized in that, The USB interface of the wireless transmission module is connected to the USB interface of the image detection and encoding control module.

6. The apparatus according to claim 1, characterized in that, The image detection and encoding control module includes a Y component extraction submodule and an image distortion detection and calculation submodule.

7. The apparatus according to claim 6, characterized in that, The image detection and encoding control module also includes an encoding decision submodule and an encoding execution submodule.

Images