Video compressed sensing system and method based on interval observation, equipment and storage medium

Abstract

The invention discloses a video compressed sensing system and method based on interval observation, equipment and a storage medium. The system comprises an image compressed sensing network and a frame synthesis network; the image compressed sensing network is used for carrying out interval observation on a key frame, generating initial recovery of the key frame according to an observation value, and performing the image enhancement for the initial recovery; the frame synthesis network is used for collecting multi-level features of key frames, predicting optical flows among the multi-level features, estimating features of intermediate frames among adjacent key frames based on the optical flows, and synthesizing the intermediate frames. According to the invention, the time redundancy of video reconstruction can be reduced, and the quality and efficiency of the reconstructed video frame can be improved.

Description

technical field [0001] The invention belongs to the technical field of video processing, and in particular relates to a video compression sensing system and method, equipment and storage medium based on interval observation. Background technique [0002] Compressed Sensing (CS) is a theory of signal compression sampling. This technology obtains the observation value of the signal at a sampling rate lower than the Nyquist rate, and restores the original signal with a high probability through a reconstruction algorithm. CS has been applied to many signals. Processing fields, such as medical imaging, cameras, radar imaging, and video transmission, etc., with the emergence and popularization of hardware systems such as single-pixel cameras, compressive sensing is applied to static image compression, showing excellent potential. Shortening the imaging time and improving the imaging quality in imaging have put forward new research directions; now, compressed sensing is not limited...

AI Technical Summary

Problems solved by technology

[0004] The existing spatial video compression sensing method obtains observations from the scene frame by frame. Commonly used observation strategies include uniform distribution strategy and non-uniform distribution strategy. Under the uniform distribution strategy, the number of observations per frame is equal, and the corresponding reconstruction algorithm Use the image compression sensing method to restore each frame separately; under the non-uniform distribution strategy, the number of observations of key frames is higher than that of non-key frames, which reduces the redundancy in time to a certain extent, and key frames can improve the accuracy of non-key frames. reconstruction quality; although the non-uniform distribution strategy divides video frames into key frames and non-key frames to improve the utilization of information, this observation method still has a lot of redundancy in the time dimension, and its The similarity between adjacent frames is very high, and the time structure of the video is not fully utilized, so that the improvement of the reconstruction effect is limited, and the potential of the video signal still needs to be further explored.

Images