Video intensive description method for fine-grained semantic perception assisted boundary positioning

Abstract

The invention provides a video dense description method for fine-grained semantic perception assisted boundary positioning. The video dense description method comprises the following steps: step 1, carrying out feature extraction and coding; 2, constructing an event coding module; step 3, decoding events with enhanced semantic perception; 4, optimizing a loss function; a decoder is regarded as a multi-head prediction task to overcome the problem that only finite correlation is generated between a positioning stage and a description stage due to the fact that only event starting and ending time is adopted as direct supervision information; on the basis, aiming at the defect that the finite correlation problem is more difficult to position under the condition that the boundary is excessively smooth, the invention further provides a method for enhancing the difference characteristics between the model effective learning foreground and background by introducing language semantic perception information into a framework. The problem of unclear positioning caused by excessive smoothness is solved by increasing the boundary sensing capability, the capability of learning robust features of noise event instances is improved, and more accurate positioning is realized.

Images