Binocular parallax matching method and system based on shared features and attention upsampling

Abstract

The invention discloses a binocular parallax matching method and system based on shared features and attention upsampling. The method comprises the following steps: preprocessing left and right images, and then extracting 1 / 2-scale, 1 / 4-scale, 1 / 8-scale and 1 / 16-scale feature maps; constructing a matching cost matrix for the 1 / 16-scale feature maps of the left image and the right image, and generating a 1 / 16-scale initial disparity map; estimating a 1 / 16-scale parallax residual image by using the 1 / 16-scale initial parallax image and the 1 / 16-scale image feature image, and realizing up-sampling by using an attention mechanism to generate a 1 / 8-scale parallax image; using the 1 / 8-scale disparity map, the 1 / 4-scale disparity map, the 1 / 2-scale disparity map and the corresponding image feature map to respectively generate a 1 / 4-scale disparity map, a 1 / 2-scale disparity map and an original-scale disparity map; carrying out model training and storing optimal model parameters; loading pre-training parameters, inputting image frames, and obtaining disparity maps of different scales. Shared feature design and attention mechanism up-sampling are adopted, the parallax matching precision andspeed are effectively improved, and a high-precision parallax image can be generated in real time.

Description

technical field [0001] The invention relates to the technical field of binocular parallax matching, in particular to a binocular parallax matching method and system based on shared features and attention upsampling. Background technique [0002] Depth estimation is the core problem of many practical applications, such as autonomous driving, 3D reconstruction, virtual reality, etc. At present, methods for obtaining depth values ​​include lidar, structured light, and binocular vision. Among them, the binocular vision method is widely used because of its low cost and easy deployment. The binocular vision method is based on a binocular camera. The binocular camera captures two views at the same time. The corresponding disparity map is obtained from the left and right views, and then the depth image is calculated according to the parameters of the binocular camera. [0003] Traditional disparity matching methods can be divided into four steps: matching cost calculation, cost ag...

AI Technical Summary

Problems solved by technology

[0005]However, the current disparity matching calculation based on the deep convolutional neural network still has certain limitations. The specific performance is: 1) too many parameters, resulting in excessive memory usage; 2) The amount of calculation is too large to support high real-time applications; 3) The amount of calculation and precision cannot be adjusted in real time according to the demand

Images