A key frame extraction method for ship surveillance video based on bidirectional GRU and attention mechanism

A technology for monitoring video and extraction methods, applied in neural learning methods, computer parts, character and pattern recognition, etc., can solve the problems of lack, negative key frames, and departure from video key frame extraction standards.

Active Publication Date: 2019-03-22
HANGZHOU DIANZI UNIV
View PDF8 Cites 17 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1. The existing key frame extraction often ignores the connection between video frames. For the video key frame extraction of video semantics, we not only need to use the visual features of the previous video frame, but also need to use the relationship between video frames. The connection in time will largely deviate from t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A key frame extraction method for ship surveillance video based on bidirectional GRU and attention mechanism
  • A key frame extraction method for ship surveillance video based on bidirectional GRU and attention mechanism
  • A key frame extraction method for ship surveillance video based on bidirectional GRU and attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] The technical solutions provided by the present invention will be further described below in conjunction with the accompanying drawings.

[0051] In the present invention, the key frame prediction of ship monitoring video is regarded as a structure prediction problem. The input is a sequence of video frames, and the output is a binary vector indicating whether the frame is selected as a keyframe. The bidirectional GRU can be used to uniformly encode the video frame information of the front and back time, and the attention mechanism gives different attention to each moment, which is more in line with the standard for human extraction of key frames. The parameters of the model are optimized using the crossover loss function and batch stochastic gradient descent. For this reason, the present invention provides the key frame extraction method based on the two-way GRU of ship video and attention mechanism.

[0052] see figure 1 and figure 2 , shown as the flow chart of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a ship monitoring video key frame extraction method based on a two-way GRU and an attention mechanism, comprising the following steps: step S1: training a two-way GRU and an attention mechanism model by using a large number of seaside ship monitoring video data sets. Step S2, extracting key frames from ship monitoring video by using trained two-way GRU and attention mechanism model, and providing a set of key frames for quickly retrieving ship. By adopting the technical proposal of the invention, the key frames are extracted and applied to the ship monitoring video, a large number of redundant video frames are eliminated, efficient search and browsing of ship events are provided, and the overhead of video storage is saved. At the same time, two-way GRU and attentionmechanism are used to model the relationship between the video frames, and the temporal information is fused into the model, and the information at each time is given different weight, that is, eachtime gives different attention degree, so that the key frame set which is more consistent with human semantics can be obtained.

Description

technical field [0001] The invention relates to fast retrieval based on ship video content and lightweight storage of ship data, in particular to a ship monitoring video key frame extraction method based on a two-way GRU and an attention mechanism. Background technique [0002] Video has become one of the most common sources of visual information. The scale of video data is rapidly expanding. For the videos uploaded to Youtube every day, individual users need more than 100 years to watch all of them. Then automatic tools for analyzing and understanding video content are very important. In particular, automatic video keyframe extraction technology can help users browse video data. A set of video key frames with good effect can succinctly represent the original video, extract important events, and represent the content of the original video with a short and readable key frame summary. With the deployment of seaside surveillance cameras, a large amount of video of ships is g...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06N3/04G06N3/08
CPCG06N3/084G06V20/46G06N3/045
Inventor 刘俊林贤早
Owner HANGZHOU DIANZI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap