Time-space video compressed sensing method based on convolutional network

A video compression and convolutional network technology, applied in the field of video processing, can solve the problems of reducing video reconstruction results, insufficient correlation, and difficult information recovery, etc., to achieve the goals of reducing network parameters, high video reconstruction, and improving balance Effect

Active Publication Date: 2018-11-30
XIDIAN UNIV
View PDF3 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since these methods only perform compression on a single dimension of space (or time), also known as observations, the obtained observations have very low resolution in the compressed dimension
As a result, when performing video reconstruction, the reconstruction result has insufficient correlation between pixels in the compressed dimension, and the information of the compressed dimension is difficult to recover, thereby reducing the result of the entire video reconstruction.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Time-space video compressed sensing method based on convolutional network
  • Time-space video compressed sensing method based on convolutional network
  • Time-space video compressed sensing method based on convolutional network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0028] Today compressed sensing is no longer limited to still images, but has been extended to video. Compared with static images, the video compression process also needs to consider the correlation in the time dimension of the image, so it is more complicated to use the compressed sensing (VCS) theory to process video. Video compression sensing compresses and samples video, reduces storage space and greatly increases transmission speed. The reconstructed video obtained after reconstructing the transmitted data can perform more complex tasks, such as target detection and tracking. Existing methods only Compression, also called observations, in a single dimension of space (or time) results in observations obtained with low resolution in the compressed dimension. As a result, when performing video reconstruction, the reconstruction result has insufficient correlation between pixels in the compressed dimension, and the information of the compressed dimension is difficult to reco...

Embodiment 2

[0040] The spatio-temporal video compression sensing method based on the convolutional network is the same as implementation 1, the network structure of the design spatio-temporal video compression sensing method described in step 2), see figure 2 , including the following steps:

[0041] 2a) The three-dimensional convolution layer setting of the observation part of the space-time video compression sensing method based on the convolutional network: the size of the convolution kernel of the three-dimensional convolution layer is set to T×3×3, where T=16 is the convolution kernel in the time dimension 3×3 is the size on the spatial dimension, and the convolution process is set without zero padding, and the step size is 3; the size of the input video block is T×H×W, and T is the number of frames contained in the input video block, H×W is the spatial dimension of each frame, and both H and W are multiples of 3; when the number of convolution kernels is 1, the spatial compression ...

Embodiment 3

[0047] The spatio-temporal video compression sensing method based on convolutional network is the same as implementation 1-2, and each "spatial-temporal block" described in step 2c) is serially connected by a three-dimensional convolutional layer and a residual block, see image 3 , including the following steps:

[0048] 2c1) The 3D convolutional layer settings in each "space-time block": the size of the convolutional kernel of the 3D convolutional layer is 16×1×1, the number is 16, and the zero padding is not set during the convolution process, and the step size is 1 ; Since the spatial dimension of the convolution kernel is 1×1, it can integrate the inter-frame information of each spatial position, and has the ability to enhance the inter-frame relationship.

[0049] 2c2) Residual block settings in each "spatial-temporal block": The residual block contains three 3D convolutional layers, and the sizes of the convolution kernels of the 3D convolutional layers are 16×3×3, 64×1...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a time-space video compressed sensing method based on a convolutional network, and mainly aims at solving the problems that in the prior art, the video compression time-space balance and the video reconstruction real-time performance are poor. The scheme of the method comprises the steps that a training data set is prepared; a network structure of a time-space video compressed sensing method is designed; training and testing files are written according to the designed network structure; a network of the time-space video compressed sensing method is trained; and the network of the time-space video compressed sensing method is tested. The network of the time-space video compressed sensing method adopts an observation technology of simultaneous time-space compression are conducted simultaneously and a reconstruction technology of using 'time-space blocks' to enhance the time-space correlation, not only can real-time video reconstruction be achieved, but also the reconstruction result has the high time-space balance, the reconstruction quality is high and stable, and the network can be used for compressed transmission of a video and follow-up video reconstruction.

Description

technical field [0001] The invention belongs to the technical field of video processing, and mainly relates to video compression perception, in particular to a spatio-temporal video compression perception method based on a convolutional network, which can be used to realize real-time high-quality video compression perception reconstruction. Background technique [0002] Compressed Sensing (CS) is a signal compression sampling theory, which can sample the signal at a rate lower than the Nyquist sampling rate, and restore the original signal by a reconstruction algorithm. The theory has been successfully applied in various signal processing fields, such as medical imaging, radar imaging, etc. After the emergence and popularization of hardware systems such as single-pixel cameras, compressed sensing has been applied to still image compression and has shown excellent potential. Today compressed sensing is no longer limited to still images, but has been extended to video. Compa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/24H04L12/26H04N19/136H04N19/149H04N19/154H04N19/176H04N19/30H04N19/85
CPCH04L41/044H04L41/0823H04L41/145H04L43/08H04N19/136H04N19/149H04N19/154H04N19/176H04N19/30H04N19/85
Inventor 谢雪梅刘婉赵至夫汪芳羽石光明
Owner XIDIAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap