Unsupervised video target segmentation method based on local and global memory mechanism

Abstract

The invention discloses an unsupervised video target segmentation method based on a local and global memory mechanism, and belongs to the technical field of feature learning and video target segmentation. The method comprises the following steps: firstly, extracting embedding features of a pair of same videos; selecting global memory candidate frames in the video, and extracting global memory candidate features, wherein each global memory candidate feature corresponds to each node corresponding to the graph convolution network, and the enhanced global memory feature expression is carried out; extracting mutual attention information between a pair of frames through a local memory module, and alternately regarding the mutual attention information as a target and a search role in an attention mechanism for mutual attention enhancement; and finally, obtaining a predicted target mask through a decoder, calculating loss by using cross entropy loss, and updating the whole model so as to obtain a final segmentation network. According to the method, local and global memory mechanisms are considered at the same time, and reliable short-time and long-time video inter-frame correlation information is obtained at the same time, so that unsupervised video target segmentation is realized.

Description

technical field [0001] The invention relates to the field of deep learning and the field of weakly supervised video object segmentation, in particular to a feature expression method in unsupervised video object segmentation, which can obtain more accurate segmentation results on video object segmentation data sets. Background technique [0002] With the development of visual big data technology, video information has gradually become an important information transmission medium, and the information it carries includes both spatial and temporal aspects. How to obtain valuable scene target information from this space-time carrier has become the top priority in the development of computer vision today. While the existing video information target analysis tasks bring convenience and attention to the society, they also bring certain challenges. For example, how to use only limited categories to segment foreground targets without specifying targets to be segmented online, so as t...

AI Technical Summary

Problems solved by technology

Although the methods in this category can achieve good results, they lack global guidance information, and the effect will be relatively poor when faced with large appearance changes.

Images