Sensor fusion depth reconstruction data driving method based on attention mechanism

Abstract

The invention discloses a sensor fusion depth reconstruction data driving method based on an attention mechanism. According to the method, a convolutional neural network structure is designed for an SPAD array detector with a resolution of 32 * 32, and a low-resolution TCSPC histogram is mapped to a high-resolution depth map under the guidance of an intensity map; and a network adopts a multi-scale method to extract input features and fuses depth data and intensity data based on an attention model. In addition, a loss function combination is designed and is suitable for a network for processing TCSPC histogram data. According to the method, a spatial resolution of depth original data can be successfully improved by four times, the depth reconstruction effect of the method is verified on simulation data and collected data, and the method is superior to other algorithms in quality and data indexes.

Description

technical field [0001] The invention belongs to data-driven technology, in particular to a sensor fusion depth reconstruction data-driven method based on an attention mechanism. [0002] technical background [0003] Inferring correct depth information from perceived scenes is critical for many applications, such as autonomous driving, virtual reality, augmented reality, and robotics. Lidar is the leading technology in depth imaging. At present, most Lidar systems adopt a single-point / scanning method, using coaxially aligned laser diodes and single-photon detectors. The laser emits laser light, and the detector time mark is reflected by the scene. The coming back arrives at the photon. Although the scanning lidar system can obtain relatively accurate depth information, the acquisition speed is slow, and it usually takes several hours to complete a scene information acquisition. However, more and more applications require fast scene acquisition. Under this demand, single pho...

AI Technical Summary

Problems solved by technology

The first method and the second method use linear SPAD detectors and single-point SPAD detectors respectively, and need to scan and image the scene. It often takes several hours to obtain the depth information of a scene, which cannot meet the needs of more and more applications. Real-time requirements; the third method is mainly aimed at the problem of depth super-resolution, which needs to select important features from the original data of the detector in advance, that is, denoise the original data first, and then input it into the network, which has more preprocessing steps, and the network Upsampling only

Images