Video event recognition method based on deep residual long-short term memory network

Abstract

The invention discloses a video event recognition method based on a deep residual long-short term memory network, which comprises 1) the design of spatial-temporal feature data connection layer, thatis, spatial-temporal feature data is synchronously parsed through a long-short term memory (LSTM) unit and then forms a spatial-temporal feature data connection unit DLSTM (double-LSTM), and the consistency of spatial and temporal information is highlighted; (2) the design of a DU-DLSTM (dual unidirectional DLSTM) structure which expands the width of the network and increases the feature selectionrange; (3) the design of an RDU-DLSTM (residual dual unidirectional DLSTM) module which solves a deeper problem of network gradient disappearance; and 4) the design of a 2C-softmax objective functionwhich reduces the distance within classes while expanding the distance between the classes. The video event recognition method has the advantages that the problem of gradient disappearance is solvedthrough constructing the deep residual network framework, and the video event recognition accuracy is improved by using the consistency fusion of temporal network and spatial network features at the same time.

Description

technical field [0001] The invention relates to a video event recognition technology, in particular to a video event recognition method based on a deep residual long-short-term memory network. Background technique [0002] Video event recognition refers to the recognition of spatio-temporal visual patterns of events from videos. With the wide application of video surveillance in real life, surveillance video event recognition has received extensive attention and a series of research results have been achieved. However, event recognition of surveillance video still faces great challenges and difficulties, such as surveillance in natural scenes. Factors such as complex video background, severe object occlusion in the event area, and changes in camera viewing angles lead to small inter-class distances and large intra-class distances. [0003] In the existing technology, in order to solve the problem of difficult event recognition in surveillance video, the traditional solution...

AI Technical Summary

Problems solved by technology

With the wide application of video surveillance in real life, surveillance video event recognition has received extensive attention and a series of research results have been achieved. However, event recognition of surveillance video still faces great challenges and difficulties, such as surveillance in natural scenes. Due to complex video background, severe object occlusion in the event area, and changes in camera angle of view, the distance between event classes is small and the distance within a class is large.

[0003] In the existing technology, in order to solve the problem of difficult event recognition in surveillance video, the traditional solution is to use the method based on the visual bag of words model and the method based on motion trajectory for event recognition in surveillance video, but this manual feature recognition method is difficult to further Improve recognition accuracy; with the development of the times, deep learning has become a research hotspot in the field of artificial intelligence, and has begun to be applied to surveillance video event detection, behavior recognition and other fields, for example, the dual-stream CNN network for behavior recognition, among them, time CNN The network uses the static frame information of the video, and the spatial CNN network uses the optical flow information of the video, but the method represented by the dual-stream CNN network only uses the short-term dynamic features of the video, and does not effectively use the long-term dynamic features of the video. There are still some defects in video event recognition, so the method of long-term recursive convolution network (LRCN) is used to make up for the above defects. LRCN uses CNN network to extract features, and then sends them to LSTM network to obtain recognition results. Among them, LSTM is also called The long-term short-term memory network can recursively learn long-term dynamic features from the input sequence, so it can handle tasks with typical time sequences, such as speech recognition and behavior recognition. Therefore, the recognition ability of CNN and LSTM networks can be improved through deep architectures. However, whether it is CNN or LSTM, as the depth of the network increases, it will encounter the problem of gradient disappearance, and it is difficult to train a deeper network.

Images