Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

An unsupervised video summarization model and its establishment method

A technology for video summarization and model building, applied in character and pattern recognition, instruments, computing, etc., can solve problems such as information loss, lack of extraction of long-term video dependencies, network pre-training, etc., to improve performance and realize end-to-end End-to-end training, the effect of eliminating subjectivity

Active Publication Date: 2021-06-22
SHANGHAI JIAO TONG UNIV
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the performance of the unsupervised video summarization technology based on reinforcement learning depends on the artificially designed reward function. Although the unsupervised video summarization technology based on the generative confrontation network can learn the confrontation loss function from the data, thereby avoiding the artificial design of the loss function, but The existing methods based on generative confrontation networks have information loss, and some networks need to be pre-trained
[0005] At the same time, the existing supervised and unsupervised video summarization techniques are only based on the recurrent neural network to obtain the relationship between frames in the video, and generally lack the extraction of long-term dependencies of the video

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An unsupervised video summarization model and its establishment method
  • An unsupervised video summarization model and its establishment method
  • An unsupervised video summarization model and its establishment method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The following describes a preferred embodiment of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned herein.

[0035] like image 3 Shown, the unsupervised video summarization model of the present invention, its establishment method comprises the following steps:

[0036] Step 1, input T frame video, utilize the pre-trained convolutional neural network to extract the original video frame feature, usually the original video frame feature extracted is a feature vector, the present embodiment uses GoogLeNet as the feature extraction network, then the feature of each frame is 1024-dimensional feature vector.

[0037] Step 2, the unsupervised video model is trained, and the input of the model is the original video frame featur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an unsupervised video summarization model, which relates to the field of video processing in the direction of computer vision. The model includes a conditional feature selection module, a frame-level self-attention module and a conditional generation confrontation network module, wherein the conditional feature selection The module selects the more important frame features in the output video to guide the training of the conditional generation confrontation network module, so that it can pay more attention to the characteristics of this area; the frame-level self-attention module can obtain the long-term dependence of frames in the video , so that the model can better learn global features and avoid removing frames that are visually similar but have a certain time distance; Any manual calibration data can overcome the information loss of the original method, some networks need to be pre-trained, and the model is complex, etc., and can achieve end-to-end training and improve the performance of video summarization.

Description

technical field [0001] The invention relates to the field of video processing in the direction of computer vision, in particular to an unsupervised video summarization model and its establishment method. Background technique [0002] Video summarization is one of the research problems in the field of computer video processing. With the explosive growth of video data in recent years, such as video data captured by massive mobile phones and surveillance cameras, it has brought huge challenges to storing and browsing videos. The purpose of the video summary is to shorten the length of the original video while allowing the shortened video to fully reflect the story line of the original video. The shortened video is also called a video summary (VideoSummary). Video summarization has a wide range of applications, which can reduce the pressure on video storage, save users' time browsing videos, generate highlights of videos, and so on. At present, mainstream methods in this field...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/738G06K9/00G06K9/62
CPCG06F16/739G06V20/46G06F18/22
Inventor 马汝辉何旭峰华扬宋涛管海兵
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products