A video semantic segmentation method and device based on prediction for feature propagation

Abstract

The invention discloses a video semantic segmentation method and device based on prediction for feature propagation, and the method comprises the steps: predicting the semantic difference of video frames according to a superficial neural network, and obtaining a plurality of key frames and a plurality of non-key frames in the video frames; Obtaining high-order semantic features of a plurality of key frames according to the picture semantic segmentation network, and predicting the high-order semantic features of a plurality of non-key frames according to sequential information of the high-ordersemantic features; And classifying the high-order semantic features of the plurality of key frames and the plurality of non-key high-order semantic features, and performing sampling to a preset sizeto generate a video semantic segmentation result. According to the method, hypotheses do not need to be made for high-order and low-order features, video semantic segmentation is obtained through prediction and fine adjustment, and the time complexity of the algorithm can be reduced on the premise that the video segmentation accuracy is guaranteed.

Description

technical field [0001] The invention relates to the technical field of video frame feature propagation, in particular to a video semantic segmentation method and device for feature propagation based on prediction. Background technique [0002] Feature propagation technology plays a vital role in real-time tasks. The feature propagation technology can reuse the obtained features, and consider the continuity of the sequence data in time, and propagate it to the task at the next moment to obtain the features at that moment. According to this, the feature propagation technology can significantly reduce the time complexity of sequence data feature acquisition, and ensure that the obtained features have high accuracy while considering the feature timing information. Feature propagation technology can be used in sequence data tasks such as video and audio. In this patent, the semantic segmentation task in video data is taken as an example to illustrate our proposed video semantic...

AI Technical Summary

Problems solved by technology

[0010] In related technologies, the first is to directly transfer the image-based semantic segmentation method to process video. This type of method has high accuracy, but needs to process each frame of the video, and this type of method cannot use the timing information of the video. Therefore, the time complexity of the algorithm is often relatively high

Although the method of non-propagation features can achieve excellent performance, most of these methods only focus on the accuracy of the algorithm and ignore the efficiency of the algorithm.

Propagation-based metho

Images