Binocular stereo matching method based on convolutional neural network

Abstract

The invention discloses a binocular stereo matching method based on a convolutional neural network. For matching cost calculation, context information is integrated by using dense blocks on the basisof initial features. For matching cost aggregation, a regularization cost amount of a small coding and decoding structure is provided. And for parallax calculation, a differentiable soft argmin operation is executed on the parallax dimension of the cost quantity to obtain an initial parallax. For parallax refinement, the initial parallax is guided to be refined by taking a residual block as a mainpart and taking similarity measurement as an auxiliary part. According to the method, four stages of a stereo matching algorithm are strictly followed, and four steps are integrated into one network,so that the network can be trained end to end. According to the stereo matching method, contextual information is integrated in the feature extraction process, mismatching of pixel points in an ill-conditioned area is effectively relieved, memory occupation and running time in the network training / speculation period are remarkably reduced through small and medium coding and decoding structures inthe regularization process, and parallax prediction precision is improved.

Description

technical field [0001] The invention relates to the fields of robot navigation and three-dimensional reconstruction of computer vision, and in particular to a binocular stereo matching method based on a convolutional neural network. Background technique [0002] Depth estimation from stereo image pairs is the core problem of many stereo vision tasks and has applications in many fields, such as 3D reconstruction, autonomous driving, object detection, robot navigation and virtual reality, augmented reality, etc. The purpose of stereo matching is to estimate the correspondence of all pixels between two rectified images. Given a pair of rectified stereo images, the goal of disparity estimation is to compute the disparity d for each pixel in the reference image. Parallax refers to the horizontal displacement between a pair of corresponding points in a reference image and a target image. For a certain pixel of the reference image (x, y), if the corresponding pixel is found at th...

AI Technical Summary

Problems solved by technology

However, these methods still have some limitations

First, the network model often cannot accurately find the matching points corresponding to pixels in ill-healthy areas such as occluded areas, repeated textures, and reflective surfaces.

Second, existing network operations have huge memory consumption and require powerful computing power

Images