Airborne video data acquisition system and method based on dynamic adjustment mechanism

Abstract

The invention belongs to the technical field of aviation airborne video acquisition and processing, and particularly relates to an airborne video data acquisition system and method based on a dynamic adjustment mechanism. The acquisition system comprises a video acquisition module, a video compression coding module, a video quality evaluation module and an output module. The video acquisition module is used for receiving various multi-channel video signals output by a video source on a receiver and carrying out format conversion on the various multi-channel video signals to obtain a unified original digital video code stream; the video compression coding module outputs a video image to the image quality evaluation module, receives an optimal compression coding parameter output by the video quality evaluation module, and performs dynamic adjustment compression coding processing on an original digital video code stream output by the video acquisition module according to the compression coding parameter; the video quality evaluation module is used for receiving a video image output by the video compression coding module, and predicting an optimal compression coding parameter of the video compression coding module in a current system by adopting a video image quality evaluation and compression parameter prediction model based on deep learning; the predicted optimal compression coding parameter is output to the video compression coding module; and the output module outputs the compressed data packet output by the video compression coding module. The processing parameters of each video channel are dynamically adjusted and set through the quality evaluation model, so that more channels of video data are processed as far as possible under the condition that processing resources are limited, and meanwhile, the video quality is ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of aviation airborne video acquisition and processing, and in particular relates to an airborne video data acquisition system and method based on a dynamic adjustment mechanism. Background Art

[0002] In flight tests, telemetry is an important means of communication between aircraft and ground monitoring systems. Through telemetry, flight data can be transmitted to the ground monitoring system in real time, providing a basis for decision-making for ground commanders, aircraft designers, test pilots, etc. Among them, the video images in the onboard data can intuitively and accurately describe the status of the aircraft, and are an important part of collecting telemetry data.

[0003] As technology gradually advances and demand increases, more and more video data needs to be collected, for example, more cameras are installed on aircraft to monitor target status. At the same time, in addition to the increase in the number of channels, the amount of video data is also increasing due to the gradual improvement of resolution, frame rate, quality, etc. of each channel of video. Therefore, the processing pressure of airborne video data acquisition equipment is also gradually increasing. When responding to different flight test tasks, if the acquisition equipment is configured according to the minimum acquisition processing requirements, the processing capacity of the collector will be wasted in some cases, and more acquisition equipment will be required. Summary of the invention

[0004] The purpose of the present invention is to propose an airborne video data acquisition system and method based on a dynamic adjustment mechanism. For multiple video data inputs, the processing parameters of each video channel are dynamically adjusted and set through a quality evaluation model, so as to process as many video data as possible under limited processing resources while ensuring video quality.

[0005] Technical solution of the present invention: In order to achieve the above-mentioned invention object, according to the first aspect of the present invention, an airborne video data acquisition system based on a dynamic adjustment mechanism is proposed, comprising: a video acquisition module, a video compression encoding module, a video quality evaluation module, and an output module;

[0006] The video acquisition module receives various types of multi-channel video signals output by the video source on the machine, converts the formats of the various types of multi-channel video signals, and obtains a unified original digital video code stream; the video compression and coding module outputs the video image to the image quality evaluation module, and receives the optimal compression and coding parameters output by the video quality evaluation module, and dynamically adjusts the compression and coding processing of the original digital video code stream output by the video acquisition module according to the compression and coding parameters; the video quality evaluation module receives the video image output by the video compression and coding module, and uses a video image quality assessment and compression parameter prediction model based on deep learning to predict the optimal compression and coding parameters of the video compression and coding module in the current system, and outputs the predicted optimal compression and coding parameters to the video compression and coding module; the output module outputs the compressed data packet output by the video compression and coding module.

[0007] In a possible embodiment, the video acquisition module receives the time code on the machine and superimposes the time code on the video screen in real time to achieve time code synchronization.

[0008] In a possible embodiment, the video compression encoding module performs hard encoding in H.264 or H.265 compression encoding format.

[0009] In a possible embodiment, the compression coding parameters include: selecting the number of channels for compression coding processing, the video image resolution of each channel, the coding method, the compressed bit rate, and the i-frame interval.

[0010] In a possible embodiment, the output module encapsulates the compressed data packets output by the video compression encoding module, and can encapsulate them into TS format for output according to application requirements, or split them into data packets according to H.264 or H.265 raw streams for output. It can also split each frame of the compressed video data and package them into PCM data packets for output as telemetry coded data.

[0011] According to a second aspect of the present invention, an airborne video data acquisition method based on a dynamic adjustment mechanism is proposed, using the above-mentioned airborne video data acquisition system based on a dynamic adjustment mechanism, comprising the following steps:

[0012] Step 1: Collect various multi-channel video signals output by the video source on the machine, obtain data and convert the formats of various formats of videos on the machine, and obtain a unified original digital video code stream;

[0013] Step 2: compress and encode the original digital video code stream; for the first time, the compression encoding uses the initial compression encoding parameters; the subsequent compression encoding uses the optimal encoding parameters predicted in step 3 to achieve dynamic adjustment of the compression encoding parameters;

[0014] Step 3: Decode the compressed coded video image, and use the deep learning-based video image quality assessment and compression parameter prediction model to predict the optimal compression coding parameters;

[0015] Step 4: Output the compressed data packet obtained by compression encoding in step 2.

[0016] In a possible embodiment, in step 2, the compression coding parameters include: selecting the number of channels for compression coding processing, the video image resolution of each channel, the coding method (such as VBR, CBR, CVBR method), the compressed bit rate, and the i-frame interval.

[0017] In a possible embodiment, in step 3, the video image quality assessment and compression parameter prediction model based on deep learning is specifically as follows:

[0018]

[0019] Among them, L VQA is the objective function for video image quality evaluation, is the quality evaluation objective function for each video channel, i=1…N, N is the number of input video channels; It is defined using the existing peak signal-to-noise ratio function or structural similarity function, that is, or where x i is the original image frame, It is the compressed and decoded image frame;

[0020] L cof is the compression coding parameter regression objective function, is the encoding parameter regression objective function for each video channel, i=1…N, N is the number of input video channels; The conventional mean square error loss function can be used. where c j The jth current parameter of the compression encoding for this channel, is the jth target parameter of the channel compression encoding; L is the target function based on video image quality evaluation L VQA and compression coding parameter regression objective function L cof The constructed weighted fusion loss function, α is the weight factor.

[0021] The advantages of the present invention are: for multi-channel video data input, the processing parameters of each video channel are dynamically adjusted and set through a quality evaluation model, so that as many channels of video data as possible can be processed under limited processing resources while ensuring video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A schematic diagram of the composition of an airborne video data acquisition system according to a preferred embodiment of the present invention;

[0023] Figure 2 A flow chart of an airborne video data acquisition method based on a dynamic adjustment mechanism according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.

[0026] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] like Figure 1 As shown, an airborne video data acquisition system based on a dynamic adjustment mechanism includes: a video acquisition module, a video compression encoding module, a video quality evaluation module, and an output module;

[0028] The video acquisition module receives various types of multi-channel video signals output by the video source on the machine, converts the formats of the various types of multi-channel video signals, and obtains a unified original digital video code stream; the video compression and coding module receives the compression and coding parameters output by the video quality evaluation module, and dynamically adjusts the compression and coding processing of the original digital video code stream output by the video acquisition module; the video quality evaluation module receives the video image output by the video compression and coding module, predicts the optimal compression and coding parameters of the video compression and coding module in the current system by evaluating the quality of the video image, and outputs the predicted optimal compression and coding parameters to the video compression and coding module; the output module outputs the data packet output by the video compression and coding module.

[0029] The video acquisition module receives various multi-channel video signals output by the video source on the machine. The received input video stream can be an analog signal or a digital signal. Common video formats include PAL, VGA, LVDS, DVI, SDI, GigE, etc. Corresponding to various formats of video, it uses mature SoC chips or DSP chips or FPGAs for data acquisition and format conversion to obtain a unified digital video code stream. The video acquisition module receives the time code on the machine and superimposes the time code on the video screen in real time to achieve time code synchronization.

[0030] The video compression coding module compresses and codes the original digital video code stream output by the video acquisition module. A DSP chip or SoC chip is used to hard-code the H.264 or H.265 compression coding format, and the compression coding parameters output by the video quality evaluation module are received, and the compression coding processing is dynamically adjusted according to the parameters. Among them, the parameters of the video compression coding module include: the number of channels for selecting the compression coding processing, the video image resolution of each channel, the coding mode (such as VBR, CBR, CVBR mode), the compressed bit rate, and the i-frame interval.

[0031] The video quality evaluation module receives the video image output by the video compression coding module, predicts the optimal compression coding parameters of the video compression coding module in the current system by evaluating the quality of the video image, and outputs the predicted optimal compression coding parameters to the video compression coding module.

[0032] The output module encapsulates the H.264 or H.265 compressed data packets output by the video compression encoding module. According to application requirements, it can be encapsulated into TS format for output or split into data packets according to H.264 or H.265 raw streams for output. It can also split each frame of the compressed video data and package them into PCM data packets for output as telemetry encoded data.

[0033] Based on the above system, the flowchart of the airborne video data acquisition method based on the dynamic adjustment mechanism is as follows:

[0034] The present invention also proposes an airborne video data acquisition method based on a dynamic adjustment mechanism, such as Figure 2 As shown, the specific steps are as follows:

[0035] Step 1: Collect various multi-channel video signals output by the video source on the machine, obtain data and convert the formats of various formats of videos on the machine to obtain a unified digital video code stream; receive the time code on the machine, and superimpose the time code on the video screen in real time to achieve time code synchronization. The received input video stream can be an analog signal or a digital signal. Common video formats include: PAL, VGA, LVDS, DVI, SDI, GigE, etc.

[0036] Step 2: compress and encode the original digital video code stream output by the video acquisition module. Preferably, step 2 is implemented using H.264 or H.265 compression encoding format.

[0037] Among them, the parameters of the video compression encoding can be dynamically adjusted, specifically including: selecting the number of channels for compression encoding processing, the video image resolution of each channel, the encoding method (such as VBR, CBR, CVBR method), the compressed bit rate, and the i-frame interval.

[0038] If the image resolution in the compression encoding parameters is smaller than the original input image resolution, the image size needs to be reduced before compression encoding of each frame of the image, which can be achieved by traditional methods such as interpolation processing.

[0039] Step 3: Video quality evaluation to dynamically adjust encoding parameters. Decode the compressed encoded video image, such as H.264 or H.265 compression decoding, perform image quality evaluation based on the decoded RGB image and the original RGB image, and predict the optimal compression encoding parameters of the video compression encoding in step 2, and output the predicted optimal compression encoding parameters to step 2 as parameters for the next stage of video compression encoding.

[0040] Among them, a video image quality assessment and compression parameter prediction model based on deep learning is adopted, and the compression coding parameters are used as the regression parameters of the model to form a fusion loss function as follows:

[0041]

[0042] Among them, L VQA is the objective function for video image quality evaluation, The quality evaluation objective function for each video channel, i=1…N, N is the number of input video channels. It can be defined by using the existing peak signal-to-noise ratio (PSNR) function or structural similarity (SSIM) function, that is, or where x i is the original image frame, It is the compressed and decoded image frame.

[0043] L cof is the compression coding parameter regression objective function, The encoding parameter regression objective function for each video channel, i = 1...N, N is the number of input video channels. The conventional mean square error loss function can be used. where c j The jth current parameter of the compression encoding for this channel, is the jth target parameter for compression coding of the channel. The compression coding parameters specifically include: video image resolution of each channel, coding mode (such as VBR, CBR, CVBR), compressed bit rate, and i-frame interval.

[0044] The model is implemented based on deep learning-based full-reference video quality assessment models such as DVQA and DeepVQA. When inputting the model, both the original image frames before compression and the image frames after decompression are input at the same time, and the compression encoding parameter output is added.

[0045] Before training, a data set needs to be established, including: original images of multiple types of video streams in typical scenes, optimal encoding parameters corresponding to various types of original videos, and video stream images obtained by compressing and encoding various types of original videos under the optimal encoding parameters.

[0046] Step 4: Encapsulate the H.264 or H.265 compressed data packet of the video compression encoding described in step 2. According to the application requirements, it can be encapsulated into TS format for output or split into data packets according to H.264 or H.265 raw stream for output. Alternatively, each frame of the compressed video data can be split and packaged into PCM data packets and output as telemetry encoded data.

Claims

1. An airborne video data acquisition system based on a dynamic adjustment mechanism, characterized in that: include: Video acquisition module, video compression encoding module, video quality evaluation module, output module; The video acquisition module receives various types of multi-channel video signals output by the video source on the receiver, converts the formats of the various types of multi-channel video signals, and obtains a unified original digital video code stream; The video compression and coding module outputs the video image to the image quality evaluation module, and receives the optimal compression and coding parameters output by the video quality evaluation module, and dynamically adjusts the compression and coding processing of the original digital video code stream output by the video acquisition module according to the compression and coding parameters; the video quality evaluation module receives the video image output by the video compression and coding module, and uses a video image quality assessment and compression parameter prediction model based on deep learning to predict the optimal compression and coding parameters of the video compression and coding module in the current system, and outputs the predicted optimal compression and coding parameters to the video compression and coding module; the output module outputs the compressed data packet output by the video compression and coding module.

2. The airborne video data acquisition system based on a dynamic adjustment mechanism according to claim 1, characterized in that: The video acquisition module receives the time code on the machine and superimposes the time code on the video screen in real time to achieve time code synchronization.

3. The airborne video data acquisition system based on a dynamic adjustment mechanism according to claim 1, characterized in that: The video compression encoding module performs hard encoding in H.264 or H.265 compression encoding format.

4. The airborne video data acquisition system based on a dynamic adjustment mechanism according to claim 1, characterized in that: The compression encoding parameters include: the number of channels selected for compression encoding processing, the video image resolution of each channel, the encoding method, the compressed bit rate, and the i-frame interval.

5. The airborne video data acquisition system based on a dynamic adjustment mechanism according to claim 1, characterized in that: The output module encapsulates the compressed data packets output by the video compression encoding module. According to application requirements, it can be encapsulated into TS format for output or split into data packets according to H.264 or H.265 raw stream for output. It can also split each frame of the compressed video data and package them into PCM data packets for output as telemetry coded data.

6. An airborne video data acquisition method based on a dynamic adjustment mechanism, using an airborne video data acquisition system based on a dynamic adjustment mechanism as claimed in any one of claims 1 to 5, characterized in that: The steps include: Step 1: Collect various multi-channel video signals output by the video source on the machine, obtain data and convert the formats of various formats of videos on the machine, and obtain a unified original digital video code stream; Step 2: compress and encode the original digital video code stream; for the first time, the compression encoding uses the initial compression encoding parameters; the subsequent compression encoding uses the optimal encoding parameters predicted in step 3 to achieve dynamic adjustment of the compression encoding parameters; Step 3: Decode the compressed coded video image, and use the deep learning-based video image quality assessment and compression parameter prediction model to predict the optimal compression coding parameters; Step 4: Output the compressed data packet obtained by compression encoding in step 2.

7. The method for collecting airborne video data based on a dynamic adjustment mechanism according to claim 6, characterized in that: In the step 2, the compression encoding parameters include: selecting the number of channels for compression encoding processing, the video image resolution of each channel, the encoding method (such as VBR, CBR, CVBR method), the compressed bit rate, and the i-frame interval.

8. The method for collecting airborne video data based on a dynamic adjustment mechanism according to claim 6, characterized in that: In step 3, the video image quality assessment and compression parameter prediction model based on deep learning is as follows: L=L VQA +αL cof Among them, L VQA is the objective function for video image quality evaluation, is the quality evaluation objective function for each video channel, i=1…N, N is the number of input video channels; It is defined using the existing peak signal-to-noise ratio function or structural similarity function, that is, or where x i is the original image frame, It is the compressed and decoded image frame; L cof is the compression coding parameter regression objective function, is the encoding parameter regression objective function for each video channel, i=1…N, N is the number of input video channels; The conventional mean square error loss function can be used. where c j The jth current parameter of the compression encoding for this channel, is the jth target parameter of the channel compression encoding; L is the target function based on video image quality evaluation L VQA and compression coding parameter regression objective function L cof The constructed weighted fusion loss function, α is the weight factor.

Images