Optical flow multilayer frame feature propagation and aggregation method for video target detection

Abstract

The invention provides an optical flow multilayer frame feature propagation and aggregation method for video target detection, and relates to the technical field of computer vision. The method comprises the following steps: firstly, extracting multilayer features of adjacent frames through a feature network, extracting optical flow through an optical flow network, then propagating multilayer framelevel features of a previous frame of a current frame and a next frame of the current frame to the current frame by utilizing the optical flow, and performing up-sampling or down-sampling on the optical flow by layers with different step lengths to obtain multilayer propagation features; and then sequentially aggregating propagation characteristics of each layer by layer, and finally generating multi-layer aggregated frame level characteristics for final video target detection. According to the optical flow multilayer frame feature propagation and aggregation method oriented to video target detection provided by the invention, the output frame level aggregation feature has the advantages of high shallow network resolution and deep network high-dimensional semantic feature, the detection performance can be improved, and the detection performance of the multilayer feature aggregation method on a small target is improved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to an optical flow multi-layer frame feature propagation and aggregation method for video target detection. Background technique [0002] At present, video target detection methods at home and abroad can be mainly divided into two categories, one is frame-level methods, and the other is feature-level methods based on optical flow. In recent years, researchers have focused on the high semantic feature level extracted by deep neural networks, modeling the motion information between video frames through optical flow, using the optical flow between frames to propagate the features of adjacent frames to the current frame, and predicting Or enhance the features of the current frame. The advantage of this method is that it is clear in thinking, simple and effective, and can train the model end-to-end. Although optical flow can be used for spatial transformation at the feature leve...

AI Technical Summary

Problems solved by technology

Although optical flow can be used for spatial transformation at the feature level, there are errors in propagating inter-frame features using optical flow information. For example, when DFF and FGFA propagate features between frames, they use the last residual block res5 of the residual network to extract However, due to errors in the optical flow network, the local features are not aligned, causing two problems: one is that the feature extracted by res5 has low resolution and high semantic level, and each pixel contains rich semantic information. If Direct detection or aggregation on these error-prone propagation features, without some methods to correct these error pixels, will directly affect the performance of detection; the second is that each pixel of the residual block res5 extracts features in The receptive field on the original image is large, some smaller targets in the video are lower than 64×64 resolution, and the range of eigenvalues ​​corresponding to the residual block res5 is lower than 4×4, and the error of a single pixel point is relatively small for these small targets. The impact of the detection is much larger than the detection of large objects with a resolution higher than 150×150

Images