Adaptive bit rate spacecraft image encoder under limited bandwidth

Abstract

The application provides a spacecraft image encoder with adaptive code rate under limited bandwidth, which comprises a video acquisition processing unit, an image compression coding unit and an image data transmission management unit. The video acquisition processing unit is used for photoelectric conversion and image data processing in a space environment. The image compression coding unit is used for image coding after image compression and conversion. The image data transmission management unit is used for video coding and transmission according to a preset management method. The application can solve the problem that multiple frames of video images with large field of view cannot be transmitted in real time due to the limitation of channel bandwidth in a space environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of space vehicle data transmission, in particular to a method for developing a space vehicle image encoder with adaptive code rate under limited bandwidth. BACKGROUND

[0002] With the rapid development of China's space technology, the spacecraft orbit gradually develops from the mature near-earth orbit exploration to the deep space exploration field. Due to the immature communication network link networking of deep space exploration and the complexity of spacecraft design, the downlink transmission bandwidth of the spacecraft is greatly limited. On the basis of meeting the main task of the spacecraft, the huge amount of data generated by video monitoring causes great transmission pressure on the channel. Once the image data volume bursts, not only the imaging effect will be affected, but also other important function tasks under the same channel of the spacecraft will be affected, causing immeasurable loss. SUMMARY

[0003] The purpose of the present application is to provide a space vehicle image encoder with adaptive code rate under limited bandwidth to solve the problem of limited bandwidth allocation of existing spacecraft video transmission and large channel pressure caused by sudden increase of transmission data.

[0004] To achieve the above purpose, the present application provides a space vehicle image encoder with adaptive code rate under limited bandwidth, which comprises:

[0005] A video acquisition and processing unit for photoelectric conversion and image preprocessing function in space environment;

[0006] An image compression and encoding unit for image encoding after image compression and conversion;

[0007] An image data transmission management unit for video encoding and transmission according to a certain management method.

[0008] Preferably, the video acquisition and processing unit comprises a visible light lens, an image sensor chip and an FPGA chip. The visible light lens collects optical corresponding spectral signals to the corresponding target surface of the image sensor chip. The image sensor chip converts the optical signals collected by the target surface into electrical signals and transmits them to the FPGA chip through the LVDS interface. The FPGA chip preprocesses the collected image electrical signals, converts the image BT1120 data, receives and transmits the compressed and encoded data.

[0009] Preferably, the visible light lens is a self-developed 125° visible light lens, the image sensor chip is a CMOS image sensor, and the FPGA chip is a field programmable gate array.

[0010] Preferably, the image compression coding unit uses the DSP processor unit of the VDCE coprocessor module with self-provided image scaling data to compress the BT1120 data transmitted by the FPGA chip, and then encodes the compressed image by using the MPEG4 coding library.

[0011] Preferably, the DSP processor unit is a professional image digital processing chip.

[0012] Preferably, the image data transmission management unit uses the Q value adjustment management method, the frame frequency control method and the code stream limitation control method to achieve adaptive code rate video transmission under limited bandwidth.

[0013] Preferably, the Q value adjustment management method uses the method of dynamically adjusting the Q value according to the encoding result, the frame frequency control method uses the method of dynamically adjusting the frame frequency according to the encoding result, and the code stream limitation control method uses the method of judging the output gate through the first cache processing according to the output interface parameters, and transmitting the data according to the low bandwidth requirement.

[0014] The present application has the following advantages:

[0015] (1) The present application is based on the video image processing platform with space flight experience for control design;

[0016] (2) The present application has the ability of dynamically adjusting the image coding parameters and dynamically adjusting the image scaling ratio;

[0017] (3) The present application has the ability of monitoring the channel bandwidth load and controlling the data stream of the video output interface;

[0018] (4) The present application is suitable for adaptive code rate video transmission under limited bandwidth. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 It is the overall flow diagram of the adaptive code rate spacecraft encoder under limited bandwidth of the present application;

[0020] Figure 2 It is the principle block diagram of the video acquisition processing unit of an embodiment of the present application;

[0021] Figure 3 It is the principle block diagram of the image compression coding unit of an embodiment of the present application;

[0022] Figure 4 It is the principle block diagram of the image data transmission management unit of an embodiment of the present application;

[0023] Figure 5The flow chart of the method for realizing Q value adjustment management, frame frequency control and code stream limitation control in the image data transmission management unit of an embodiment of the present application is shown in the figure.

[0024] Figure 6 The final code rate control effect diagram of an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described and discussed in detail below with reference to the drawings of the present application. Obviously, only some of the embodiments of the present application are described here, and all other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0026] In order to facilitate the understanding of the embodiments of the present application, the embodiments will be further explained and described below with reference to the drawings and specific examples, and each embodiment does not constitute a limitation on the embodiments of the present application.

[0027] Reference Figure 1 As shown in the figure, the embodiment provides a self-adaptive code rate spacecraft image encoder under limited bandwidth, which comprises:

[0028] A video acquisition and processing unit, which is used for photoelectric conversion and image preprocessing in a space environment.

[0029] An image compression and encoding unit, which is used for image encoding after compression and conversion of the acquired images.

[0030] An image data transmission management unit, which is used for video encoding and transmission according to a certain management method for image compression and encoding and data transmission.

[0031] In order to reduce the channel bandwidth pressure, reduce the video transmission code rate and ensure the continuity of video monitoring, the present application adjusts the self-adaptive code rate of video transmission under limited bandwidth based on the existing spacecraft image encoding hardware platform, which is an important research direction for future spacecraft video transmission. The image encoder of the present application is used for: firstly, using a data conversion engine to horizontally and vertically scale the acquired video images, compressing the high-resolution images under the premise of ensuring the original field of view, so as to reduce the size of the original image; then using an MPEG4 encoding library to compress and encode each original image again; finally, managing the video compression parameters Q value and frame frequency according to the data amount of each compressed image according to a certain method, so as to avoid the channel congestion caused by the sudden increase of transmission data amount. Using the present application, the problem that multiple frame frequency large field of view video images cannot be transmitted in real time due to the limitation of channel bandwidth in a space environment can be solved.

[0032] Reference Figure 2As shown, the video acquisition processing unit in the embodiment includes a lens group, a light sensor, an FPGA, a DDR2, a memory, a crystal oscillator, an external interface, and an internal bus. The lens group uses a customized optical lens. The optical lens collects visible light signals and transmits the collected light signals to the light sensor. The light sensor chip converts the collected optical signals into electrical signals. The image pixel resolution is 2048*2048. The image is transmitted to the FPGA through a parallel synchronous video interface. The FPGA chip pre-processes the collected image electrical signals, converts the image BT1120 data, adjusts the resolution to 16:9 standard 1920*1080 resolution, and transmits the image data stream to the internal bus. The internal bus receives the compressed image data stream and sends the data stream at a required rate through the external interface. The DDR2 is used for data caching of the FPGA chip. The memory is used as a program storage chip of the FPGA chip. The crystal oscillator provides a clock source for the FPGA chip. The internal bus is used for data interaction with the image compression and encoding unit. The external interface receives the FPGA constant rate image data and external control signals. In the embodiment, the FPGA sends the processed original image data (BT1120) to the image compression and encoding unit through the internal bus for further encoding and video transmission control. In addition, the FPGA monitors the MPEG4 image encoding data obtained from the image data transmission management unit. When the data amount exceeds a threshold, the FPGA notifies the image data transmission management unit to adjust the code stream size by limiting the compressed image data.

[0033] The lens group in the embodiment is a self-developed visible light lens with a large field of view of 125°. The light sensor is a CMV4000 supporting a 2048*2048 pixel target surface. The FPGA is an XQ4VSX55.

[0034] Reference Figure 3 As shown, the image compression and encoding unit in the embodiment uses a DSP chip DM6467 as the main processing chip and is supplemented by peripheral circuits. The DDR2 is a data caching chip, and the memory is a program storage chip of the DSP chip. The crystal oscillator provides a clock source for the DSP chip. The DSP chip has heterogeneous dual-core processing capability of an ARM control core and a DSP encoding core.

[0035] ARM control core, responsible for receiving the original image data in the form of internal bus BT1120, extracting each frame of image data and storing in the off-chip DDR2 cache, the resolution is 1920*1080, the frame rate is 25fps, informing the VDCE coprocessor of each frame of image data after extraction to compress the 1920*1080 resolution field of view to CIF format YUV420 format 352*288 pixel image data, reducing the amount of original image data from the source, informing the DSP encoding core to encode 352*288 according to YUV420, and transmitting the received encoded data to the video acquisition processing unit;

[0036] DSP encoding core, responsible for receiving the compressed image data of the ARM control core, obtaining the image data transmission management unit calculation parameters, using the self-provided encoding coprocessor to call the MPEG4 encoding library for video encoding according to the corresponding parameters, further compressing the video image data, transmitting the length of the encoded data to the image data transmission management unit, and sending the encoded data to the ARM control core.

[0037] The VDCE coprocessor in the embodiment is a video data conversion engine (VDCE) provided in a 6000 series chip, and the MPEG4 encoding library selects a highly efficient video compression algorithm library developed based on a high-definition video / image coprocessor (HDVICP) provided in an image data processing chip.

[0038] Reference Figure 4 As shown, the image data transmission management unit of the embodiment has a total of Q value adjustment management method, frame frequency control method, and code stream limit control method.

[0039] The Q value adjustment management method returns the encoding length of the current frame according to the MPEG4 encoding library encoding, judges the next frame Q value parameter transmission value through the coefficient lookup table method, adjusts the encoding effect of each frame, and solves the problem of data amount exceeding the channel bandwidth after encoding; the frame frequency control method analyzes whether to encode the next frame according to the length after MPEG encoding, solves the problem of exceeding the channel bandwidth which cannot be solved by the Q value adjustment management through reducing the frame frequency; the code stream limit control method judges whether the encoded data is sent to the video acquisition processing unit according to the feedback of the FPGA chip cache size of the video acquisition processing unit, so as to limit the data stream burst.

[0040] Reference Figure 5 and 6 As shown, the image data transmission management unit provided in the embodiment realizes the Q value adjustment management, frame frequency control, and code stream limit control method flow chart and the final code rate control effect diagram.

[0041] The image data transmission management unit provided by the embodiment provides a Q value adjustment management method as shown in the flow chart, two parameters are set, current P frame encoding length WP and reference P frame encoding length 2000, according to the dynamic comparison with the reference P frame, the Q value is dynamically adjusted in the range of 2 to 31 according to the flow chart, the overall idea is that the larger the current encoding length WP, the larger the dynamic adjustment Q value, and the maximum cannot exceed 31.

[0042] The image data transmission management unit provided by the embodiment provides a frame frequency control method, by judging the encoding length Nal_size, if the Nal_size length is greater than half of the resolution size, the next frame P frame encoding data is discarded, for example, the 352*288 encoding data amount should be no more than 352*144 Byte, and the 1920*1080 data amount should be no more than 1920*540 Byte.

[0043] The image data transmission management unit provided by the embodiment provides an image code stream limitation control method, by judging the buffer size of the FPGA chip in the video acquisition processing unit in the DDR2, if the image encoding buffer in the FPGA reaches half of the total buffer size due to the sending rate limitation, that is, the half-full flag is set to 1, the image data transmission management unit will buffer the image encoding data, wait for the next half-full signal to be 0, and then send the subsequent data frame to the video acquisition processing unit together, wherein the data buffer of the image data transmission management unit can store 1MB, which can store more than 20 frames of 384*288 resolution image data, if the buffer is full, no storage is performed, and data storage is performed only after there is space.

[0044] The Q value of the embodiment represents the serial number of the corresponding quantization step, and the value ranges from 2 to 31. The smaller the value, the smaller the quantization step, and the higher the quantization accuracy, which means that the data amount generated may be larger under the same picture quality. The quantization step is doubled every 6 Q value increases.

[0045] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any deformation or replacement of the present application within the technical range disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A bandwidth-limited, adaptive bit rate spacecraft image encoder, comprising: The application relates to a video acquisition and processing device. The video acquisition and processing device comprises the following parts: a video acquisition and processing unit for photoelectric conversion and image data processing in a space environment; an image compression and coding unit for image coding after image compression and conversion, wherein the image compression and conversion is horizontal and vertical scaling of the acquired video image by using a data conversion engine; an image data transmission management unit for video coding and transmission according to a preset management method of image compression and coding and data transmission; the image data transmission management unit uses a Q value adjustment management method, a frame frequency control method and a code stream limitation control method to meet the video transmission under limited bandwidth; the Q value adjustment management method returns the coding length of the current frame according to the MPEG4 coding library coding, and judges the next frame Q value parameter input value by using a coefficient lookup table method; the frame frequency control method uses a method of dynamically adjusting the frame frequency according to the coding result, and discards the next frame P frame coding data if the Nal_size length is greater than half of the resolution size; 2. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 1 wherein, the code stream limitation control method uses the output interface parameter to judge the output gate through one-stage cache processing; the FPGA chip in the video acquisition and processing unit judges the cache size in the DDR2; if the image coding cache in the FPGA reaches half of the total cache size due to the sending rate limitation, that is, the half-full flag bit is set to 1, then the image data transmission management unit will cache the image coding data, wait for the next half-full signal to return to 0, and then send the subsequent data frame to the video acquisition and processing unit; if the cache is full, the data storage is not performed until there is space. The video acquisition and processing unit comprises a visible light lens, an image sensor chip and an FPGA chip, wherein the visible light lens is used for collecting optical corresponding spectrum signals to the corresponding target surface of the image sensor chip; the image sensor chip is used for converting the optical signals collected by the target surface into electric signals and transmitting the electric signals to the FPGA chip through an LVDS interface; 3. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 2 wherein, the FPGA chip is used for pre-processing the collected image electric signals, converting the image BT1120 data, receiving the compressed and coded data and transmitting the data.

4. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 2 wherein, The visible light lens is a 125-degree large field of view visible light lens.

5. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 2 wherein, The image sensor chip is a CMOS type image sensor.

6. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 2 wherein, The FPGA chip is a field programmable gate array.

7. The limited bandwidth down adaptive bit rate spacecraft image encoder of claim 2 wherein, The image compression and coding unit uses a DSP processor unit of a VDCE coprocessor module of the image scaling data conversion to compress the BT1120 data transmitted by the FPGA chip, and then codes the compressed image by using an MPEG4 coding library. The DSP processor unit is an image digital processing chip, the chip has a VDCE image scaling data conversion coprocessor and supports the MPEG4 coding library.

Images