A method for summarizing videos using a hierarchical self-attention network combined with video descriptions
A video description and attention technology, applied in neural learning methods, biological neural network models, selective content distribution, etc., can solve the problem that the video summary task cannot provide more accurate
Active Publication Date: 2021-03-26
HANGZHOU YIWISE INTELLIGENT TECH CO LTD
View PDF9 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The purpose of the present invention is to solve the problems in the prior art. In order to overcome the problem that the prior art cannot provide more accurate video summaries for video summarization tasks, the present invention provides a layered self-attention network combined with video descriptions. Ways to summarize the video
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment
[0106] The present invention conducts training on the ActivityNet experimental data set, and conducts test experiments on the SumMe experimental data set and the TVSum experimental data set. In order to objectively evaluate the performance of the algorithm of the present invention, the present invention respectively adopts unsupervised method and supervised method to evaluate the effect of the present invention in the selected test set, and aims at unsupervised method and Supervised methods all use the evaluation standard of F score to evaluate the effect of the present invention. According to the steps described in the specific embodiment, the experimental results obtained are shown in Table 1-2, and this method is expressed as HSAN:
[0107]
[0108] Table 1 The present invention is aimed at the test result of unsupervised method
[0109]
[0110] Table 2 The present invention is aimed at the test results of the supervised method.
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a method for summarizing videos by using a hierarchical self-attention network combined with video description. The method mainly comprises the following steps of: 1) designinga hierarchical self-attention network model and an enhanced title generator model to obtain comprehensive importance degree scores of all frames in a video and improve the accuracy of the importancedegree scores; and 2) training to obtain a final layered self-attention network model, and generating an answer of the video summary task by using the model. Compared with a common video summary tasksolution, the video summary task can be better completed by using the video description information. Compared with a traditional method, the effect obtained in the video summary task is better.
Description
technical field [0001] The present invention relates to video summarization tasks, and more particularly to a method for summarizing videos using a hierarchical self-attention network combined with video descriptions. Background technique [0002] The video summarization task is a very challenging task that has attracted a lot of attention currently. In this task, the system needs to target a specific video, give the key frames in the video, and complete the task of summarizing the video. Current research on video summarization tasks is still in the exploratory stage. Research on video summarization tasks can be applied in many fields. [0003] The existing video summarization task solutions generally focus on solving the diversity and representativeness of the video summarization task, and pay less attention to the semantic information contained in the video. Traditional video summarization task solutions mainly use hand-tuned heuristic methods to select important frames...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04N21/8549H04N21/845G06N3/08G06N3/04G06K9/00
Inventor 赵洲王昕然
Owner HANGZHOU YIWISE INTELLIGENT TECH CO LTD