Video instance segmentation method and system based on inter-frame relationship

Abstract

The invention discloses a video instance segmentation method and system based on an inter-frame relationship, and belongs to the field of video processing, and the method comprises the steps: obtaining a target Mask R-CNN network and a target FlowNet network through end-to-end training; sequentially acquiring each frame of image in the to-be-processed video, judging the type of each frame of image, if the type of each frame of image is a key frame, performing instance segmentation by utilizing the target Mask R-CNN network, otherwise, acquiring a corresponding key frame, and performing instance segmentation by utilizing the video instance segmentation model; wherein the video instance segmentation model is used for performing instance segmentation on the key frame by using a target Mask R-CNN network, and the video instance segmentation model also uses a target FlowNet network to calculate a flow field and a scale field between the key frame and the non-key frame, propagates pyramid features of the key frame to the non-key frame according to the flow field and the scale field, and performs instance segmentation on the non-key frame according to the features of the non-key frame. According to the method, the speed of video instance segmentation can be increased while high-quality instance segmentation is completed.

Description

technical field [0001] The invention belongs to the field of video processing, and more specifically relates to a video instance segmentation method and system based on inter-frame relationships. Background technique [0002] Instance segmentation (Instance segmentation) divides the scene image into multiple regions, each region corresponds to an object, and judges the category of the region. Instance segmentation is a computer vision task that is closest to the real visual experience of human beings. It has high application value especially in the field of automatic driving, such as detecting lane lines, pedestrians, obstacles, etc. through instance segmentation to guide automatic driving. [0003] The SDS (Simultaneous Detection and Segmentation) algorithm proposed in 2014 can be said to be the earliest instance segmentation algorithm and the basis of existing instance segmentation algorithms. The algorithm combines detection with segmentation, specifically, object detect...

AI Technical Summary

Problems solved by technology

This video instance segmentation method completely ignores the temporal continuity and the relationship between the front and rear frames in the video, and treats each frame in the video as an independent individual, and each frame of image must be reprocessed, which leads to the processing of video The calculation of the frame is relatively large, and the processing takes a long time, which cannot meet the real-time requirements of video processing

Images