Video content semantic understanding method based on a recurrent convolutional neural network

Abstract

The invention relates to a video content semantic understanding method based on a recurrent convolutional neural network. The method is used for carrying out content analysis and classification on video data such as network videos and monitoring videos. According to the method, a convolutional neural network is placed in a recurrent neural network to serve as a kernel; Presenting video state concepts, organic combination of target detection and inter-frame association in the video data is realized; By performing recursive convolution operation between video frames, accurate and efficient extraction of video features is realized, video representations with more semantic representations are obtained, and on the basis, tasks of video classification, event detection, scene recognition and thelike are completed by adopting an artificial neural network full-connection classifier. The method provided by the invention overcomes the problems of information loss, poor feature characterization capability, training convergence difficulty and the like in the traditional method, and is an accurate, efficient and advanced method with a wide application prospect.

Description

Technical field [0001] The invention relates to a method for semantic understanding of video content based on a recursive convolutional neural network, and belongs to the technical field of computer vision. Background technique [0002] Video content understanding is one of the important basic issues in computer vision. Its goal is to extract features from images in the video and model the relationship between video frames, and finally obtain the feature representation of the entire video to facilitate subsequent image analysis The semantic understanding of video and video can be used in technical fields such as autonomous driving, surveillance video real-time intelligent detection and network video auditing. [0003] Traditional video content processing methods include simple single-frame image processing, optical flow method, feature extraction method based on convolutional neural network, feature extraction method based on recurrent neural network, or a combination of multiple m...

AI Technical Summary

Problems solved by technology

[0007] First, the accurate feature representation of the convolutional neural network requires a high-dimensional model output, and the recurrent neural network training and application calculation costs require a low-dimensional input data. The contradiction between the two leads to the cascade combination of the two. There is a critical bottleneck in the method, and a large amount of key information is lost. Therefore, it is impossible to effectively use the inter-frame relationship of the video to provide effective supervision for the training of the neural network model. In practical applications, it is difficult to accurately make a reasonable detection of the overall content of the video.

[0008] Second, such traditional methods center on object detection and estimate video content based on objects of different shapes between multiple frames, which separates the two concepts of object detection and inter-frame change

There are a lot of objects of different shapes in the real scene, and their accurate representation leads to the dimensionality disaster problem in the field of deep learning, and the requirement for the scale of the training data set is too high

Moreover, the robustness to action length and scene switching is poor

[0009] The video content understanding model of the cascaded convolutional neural network and recursive neural network is based on the above limitations, and the feature output dimension of the convolutional neural network is limited. effective modeling of relationships

Images