Video frame sampling method, device, electronic equipment and storage medium
A technology of video frame and sampling position, applied in the field of computer and video understanding, can solve the problems affecting classification performance, and achieve the effect of improving performance
Active Publication Date: 2021-10-08
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF10 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The current manually designed frame extraction strategies such as random sampling, uniform sampling, or interval sampling will inevitably sample irrelevant content in the video, thereby affecting the overall classification performance
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0075] Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
[0076] figure 1 is a schematic diagram of a video frame sampling method according to Embodiment 1 of the present application. The video frame sampling method includes:
[0077] Step S110, determining the initial sampling position in the sequence of video frames;
[0078] Step S120, performing feature extraction on the video frame corresponding to the initial sampl...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The application discloses a video frame sampling method, device, electronic equipment and storage medium, and relates to the technical field of video understanding. The specific implementation plan is: determine the initial sampling position in the video frame sequence; perform feature extraction on the video frame corresponding to the initial sampling position through the convolutional neural network, obtain the video frame feature, and input the video frame feature into the gated recurrent unit network; The control cycle unit network generates the hidden layer state according to the video frame characteristics, and inputs the hidden layer state to the strategy network; the strategy network outputs the probability distribution of the action space according to the hidden layer state, and the action space includes the action set for adjusting the sampling position; according to The probability distribution of the action space selects the action from the set of actions to adjust the sampling position. The embodiment of the present application implements sampling of image frames in a video based on a convolutional neural network, a gated recurrent unit network, a policy network, and an action space, and improves the performance of video recognition by adaptively sampling video content.
Description
technical field [0001] The present application relates to a technical field of computers, and in particular to a technical field of video understanding. Background technique [0002] An important part of video understanding is to identify video content, that is, to classify videos into categories such as film and television, entertainment, and sports. At present, most video recognition methods select video frames from video sequences, and classify based on the image features of the selected video frames. In this type of method, the selection methods in the video are basically manually designed frame extraction strategies, such as random sampling, uniform sampling, or interval sampling and other frame extraction strategies. For a video sequence, not every picture frame in the video is suitable for extracting features for recognition, especially for longer videos, the picture frames in the video that have nothing to do with the video category will bring negative effects to th...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04N21/234H04N21/44H04N21/2343H04N21/4402G06N3/04G06K9/00
CPCH04N21/23418H04N21/44008H04N21/234381H04N21/440281G06V20/41G06N3/045
Inventor 何栋梁谭啸文石磊孙昊
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD