A method and a system for determining binocular scene flow based on semantic segmentation

Abstract

The invention discloses a method and a system for determining binocular scene flow based on semantic segmentation. Firstly, the scene in binocular image is semantically segmented, and the semantic optical flow is calculated by adding semantic segmentation label information, then the disparity information is calculated by semi-global matching algorithm, and then the motion parameters of countless facet regions are calculated and optimized by combining the semantic optical flow and disparity information fitting. In the process of optimization, the initial scene flow is obtained by superpixel segmentation, and then the motion of the superpixel blocks in the semantic tag is optimized, so the motion of the superpixel blocks in the semantic tag is consistent and the edge information of the moving object is protected well. Semantic information is added into the optical flow information, so that the edge of the object is protected, and the reasoning process of the occlusion problem is greatlysimplified. In addition, the motion inference at the semantic tag level makes the scene flow of pixels on the surface of the same moving object approximately consistent, and ultimately the goal of optimizing the scene flow is achieved.

Description

technical field [0001] The present invention relates to the technical field of scene flow optimization, in particular to a method and system for determining binocular scene flow based on semantic segmentation. Background technique [0002] Scene flow is a three-dimensional motion field formed by the three-dimensional motion of scenes in space. The proposal of scene flow will extend the motion estimation of objects from two-dimensional to three-dimensional, and it is at the core of 3D reconstruction and visual navigation. The common scene flow determination method is the scene flow calculation method based on binocular vision. The principle is to construct a corresponding The motion scene of image pixels in three-dimensional space, that is, scene flow. Scene flow includes the structure and motion characteristics of 3D scenes, and is the core issue in many vision applications, such as video tracking and monitoring, autonomous robot navigation, virtual reality, 3D video compre...

AI Technical Summary

Problems solved by technology

However, even the state-of-the-art scene flow methods still perform poorly near moving edges and object occlusion boundaries, and the problem of moving edge blur and occlusion has always been a difficult problem in scene flow estimation algorithms.

Images