Video high-temporal-spatial-resolution signal processing method combining optical flow method and deep network

Abstract

The invention discloses a video high-temporal-spatial-resolution signal processing method combining an optical flow method and a deep network. The video high-temporal-spatial-resolution signal processing method adopts a signal and information processing algorithm-based mode to recover and reconstruct a high-spatial-resolution and high-temporal-resolution video sequence, i.e., a video super-resolution reconstruction method. The video high-temporal-spatial-resolution signal processing method comprises the following steps: taking frame sequences of videos in sequence; starting from the third frame of the video, performing optical flow method motion estimation on each frame and front and back frames of the frame; synthesizing the four generated motion estimation images and the intermediate frame into high-dimensional image blocks of five images; constructing an OF deep convolution super-resolution network, extracting image information by a shallow network, and reconstructing a super-resolution image by a last sub-pixel convolution layer; sending the high-dimensional image blocks into a deep convolutional network for training; and finally, sending the degraded video frame or the video frame with relatively low resolution into a network for reconstruction. According to the video high-temporal-spatial-resolution signal processing method, the reconstruction quality is good, and the reconstruction speed is high, and the reconstruction effect is good compared with a traditional video super-resolution model, and real-time video reconstruction can be carried out.

Description

technical field

[0001] The design of the present invention belongs to the application field of image processing and deep learning, and specifically relates to a video signal processing method with high spatio-temporal resolution combined with optical flow method and deep network. Background technique

[0002] In the process of video recording, transmission and storage, the resolution reduction problem of high-frequency signal loss often occurs. The low-resolution video frames reconstructed by video super-resolution can directly obtain high-resolution images, which is fast and efficient. It is an effective video Signal processing method. Video super-resolution reconstruction can be applied to monitoring, video recording, high-definition video and TV broadcasting, etc.

[0003] The current mainstream super-resolution reconstruction methods include interpolation method, reconstruction method and learning method. The principle of the method based on interpolation is simple, an...

Examples

Embodiment Construction

[0037] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0038] A video high temporal and spatial resolution signal processing method combining an optical flow method and a deep network, comprising the following steps:

[0039] Step 1, extract the frame sequence of the video file in sequence.

[0040] Step 2: Starting from the third frame of the video, each frame is subjected to optical flow motion estimation with the two frames before and after; the generated 4 motion estimation images and the intermediate frame are synthesized into 5 high-dimensional image blocks of images.

[0041] The high-dimensional image block construction method described in step 2 is:

[0042] Step 2-1, select 5 consecutive video frames in the video frame sequence, we mark this frame as the nth frame, then the first two frames are respectively n-2 and n-1 frames, and the last two frames are n+1, n+ 2 frames.

...

technical field

[0001] The design of the present invention belongs to the application field of image processing and deep learning, and specifically relates to a video signal processing method with high spatio-temporal resolution combined with optical flow method and deep network. Background technique

[0002] In the process of video recording, transmission and storage, the resolution reduction problem of high-frequency signal loss often occurs. The low-resolution video frames reconstructed by video super-resolution can directly obtain high-resolution images, which is fast and efficient. It is an effective video Signal processing method. Video super-resolution reconstruction can be applied to monitoring, video recording, high-definition video and TV broadcasting, etc.

[0003] The current mainstream super-resolution reconstruction methods include interpolation method, reconstruction method and learning method. The principle of the method based on interpolation is simple, an...

Examples

Embodiment Construction

[0037] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0038] A video high temporal and spatial resolution signal processing method combining an optical flow method and a deep network, comprising the following steps:

[0039] Step 1, extract the frame sequence of the video file in sequence.

[0040] Step 2: Starting from the third frame of the video, each frame is subjected to optical flow motion estimation with the two frames before and after; the generated 4 motion estimation images and the intermediate frame are synthesized into 5 high-dimensional image blocks of images.

[0041] The high-dimensional image block construction method described in step 2 is:

[0042] Step 2-1, select 5 consecutive video frames in the video frame sequence, we mark this frame as the nth frame, then the first two frames are respectively n-2 and n-1 frames, and the last two frames are n+1, n+ 2 frames.

...

Images