Video dense event description method based on generative adversarial network

Abstract

The invention discloses a video dense event description method based on a generative adversarial network, and belongs to the field of deep learning and image recognition. The method comprises the steps of constructing a video dense event description network; the network comprises a video feature extraction module for performing feature extraction on video frames to obtain video frame features; thetime sequence action feature extraction module is used for performing video frame feature learning by utilizing the characteristics of forward propagation and reverse propagation of the video to obtain each time sequence action feature; the natural language encoder fuses the video frame features and the time sequence action features by utilizing an attention mechanism to obtain natural statements; the discriminator is used for enhancing the accuracy of natural statements; and performing video dense event description by using the trained video dense event description model. According to the method, the characteristic of video two-way propagation is fully considered, video characteristics and time sequence action characteristics are fully utilized when the natural sentences are learned to be generated, and the grammar discriminator and the content discriminator are constructed, so that the accuracy of the natural sentences is effectively enhanced.

Description

technical field [0001] The invention belongs to the field of deep learning and image recognition, and more specifically, relates to a description method for video-intensive events based on a generative confrontation network. Background technique [0002] In recent years, with the gradual popularization of high-definition video surveillance and the rapid development of video apps such as short video social software and live broadcast software, video data has exploded. How to intelligently analyze these massive video data has also become a hot spot in the field of visual analysis. Generally speaking, the video-intensive event description algorithm is to perform multiple descriptions for a video, including three parts, one is video feature extraction, the other is video timing motion detection, and the third is video description generation. [0003] Among them, the task of video description generation is to generate corresponding natural language descriptions for videos. The c...

AI Technical Summary

Problems solved by technology

[0004] However, most of the existing video description generation tasks only consider the characteristics of video timing, but ignore the features covered by temporal action detection; at the same time, the existing video description generation models do not have a good model for the generated sentences To judge whether the generated sentence is grammatical and appropriate to the event itself, so it is necessary to design a good network model to solve the above problems

Images