Video object segmentation method and system based on diversified interaction

Abstract

The invention discloses a video object segmentation method and system based on diversified interaction, and belongs to the technical field of video image processing, and the method comprises the steps: constructing a network segmentation model for interactive video object segmentation, and carrying out the user interaction processing of a training set, and obtaining an expected segmented image; performing bidirectional propagation on the segmented single-frame image, and segmenting each frame image in the propagation process to obtain a propagation output image; performing difference fusion on the propagation output image to obtain a fused segmentation result; and training the network segmentation model by using the segmented image, the propagated output image, the fused segmentation result and the training set corresponding to the segmented image until a convergence condition is satisfied, and performing video object segmentation by using the trained network segmentation model, so that the precision of the segmentation result can be effectively improved through various interaction factors.

Description

technical field [0001] The invention belongs to the technical field of video image processing, and in particular relates to a method and system for segmentation of video objects based on diverse interactions. Background technique [0002] Video object segmentation aims to perform high-quality segmentation of target objects in input video sequences, and has a wide range of applications in video understanding and video editing. Existing VOS methods can be classified according to the user's input type, semi-supervised methods require pixel labeling of the first frame, and interactive VOS methods take user interaction as input, and users can iteratively optimize the results until they are satisfied. Interactive VOS has more applications in video editing because typical user interactions like clicking or scribbling are much easier than specifying full annotations. Interactive VOS can be thought of as a combination of two tasks, interaction understanding and time propagation. ...

AI Technical Summary

Problems solved by technology

[0006] The existing video object segmentation technology mostly adopts the linear weighting method when fusing video frames with different segmentation results. This method cannot effectively convey the user's intention during the fusion process.

This can make it difficult to blend the differences between video frames

In addition, the existing technology cannot make good use of the precise location information of the features, making it difficult for the model to obtain multi-dimensional feature information

At the same time, the existing techniques cannot well separate the two processes of segmentation mask generation and segmentation mask propagation.

So, not great versatility and performance

And the existing technology is strongly related to the interaction method, resulting in the model can only rely on a single input, and the generalization is very low

Images