Video description method and system based on multi-level coding-decoding device

A video description and multi-level technology, applied in the field of video processing, can solve problems such as unclear interrelationships, inaccurate description of fine-grained elements, etc., and achieve the effect of improving description performance

Active Publication Date: 2021-04-30
SUN YAT SEN UNIV
View PDF13 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the existing methods, most models ignore the multi-grained hierarchical structure and the relationship modeling between semantic elements, which often makes the descrip

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video description method and system based on multi-level coding-decoding device
  • Video description method and system based on multi-level coding-decoding device
  • Video description method and system based on multi-level coding-decoding device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033]The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, it is only provided for convenience of explaining that the order between steps does not do any defined, and the execution order of each step in the embodiment can be adapted according to the understanding of the art. Sex adjustment.

[0034]Referfigure 1 withimage 3 The present invention provides a video description method based on a multi-hierarchical coding-decoder, which comprises the following steps:

[0035]S1, obtain video and encoding processing based on multi-hierarchical encoders, to build target maps and event diagrams;

[0036]Specifically, multi-hierarchical encoder structuresfigure 2 The present invention constructs two types of graphs to represent hierarchies in the video, that is, target maps and event diagrams, on a small scale, we construct a few separate target maps, each target diagram represe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a video description method and system based on a multi-level codec. The method comprises the steps: obtaining a video, carrying out the coding processing based on a multi-level encoder, and constructing a target graph and an event graph; and decoding the target graph and the event graph based on a multi-level decoder to obtain a sentence sequence and a word sequence, and completing a text description task and a sentence label prediction task based on multi-task learning. The system comprises an encoding module and a decoding module. Based on the multi-level encoder and the multi-level decoder, the relation of fine grit in the statement can be mined, and the description performance can be improved. The video description method and system based on the multi-level codec can be widely applied to the field of video processing.

Description

Technical field[0001]The present invention belongs to the field of video processing, and more particularly to a video description method and system based on a multi-hierarchical coder.Background technique[0002]The goal of intensive video description tasks is to perform time position detection and natural language descriptions in the unusual video, which has attracted more and more researchers in recent years. Interesting video descriptions include two subtasks, timing event nominations tasks, and event description tasks. The purpose of the former is to detect the time range of the event, and the latter is a natural language description to the event. Event Description The network needs to be accurate, powerful event features as input, and the precision time border of the event is the basis for the feature construct, so most of the existing models are completed two-step complex description: first implement accurate event nomination forecast, further Perform an event description. In th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N21/84H04N19/42H04N21/44H04N21/234G06N3/04
CPCH04N21/84H04N19/42H04N21/44008H04N21/23418G06N3/045
Inventor 郑慧诚余明静王腾刘泽华
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap