Monocular multi-modal depth map generation method, system, device and storage medium

Abstract

The invention provides a method, system, device and storage medium for generating a depth map based on monocular multimodality, which belongs to the technical field of image processing and solves the technical problem that the existing depth map acquisition methods cannot meet the requirements of all-weather multi-scenes. The method includes: creating a dual-branch sensory neural network, inputting infrared images and visible light images into the dual-branch sensory neural network to generate infrared feature maps and visible light feature maps; The feature map and the visible light feature map are mutually perceptually fused across modalities to obtain a feature fusion map; the feature fusion map is up-sampled by a dual sensory neural network to generate a brand new depth map. Based on the image data of the two modalities of infrared images and visible light images, cross-modal fusion is performed at the feature level, and finally a brand new depth map that combines the advantages of visible light images and infrared images is generated, which can be obtained in all-weather and multi-scene conditions. .

Description

technical field [0001] The present invention relates to the technical field of image processing, in particular to a method, system, device and storage medium for generating a monocular multimodal depth map. Background technique [0002] The depth map is also known as the distance map, and its pixel values ​​represent the distance from the image collector to each point in the scene. This depth information helps to understand the geometric relationship between the object and the environment. In augmented reality, in focusing, object detection It plays an important role in assisting the blind to perceive the environment. The depth map can be obtained by the depth camera. The imaging methods of the existing depth camera can be roughly divided into three types, namely, based on structured light, ToF (Time of flight) and pure binocular. However, none of these three imaging methods can meet the depth map acquisition of all-weather and multi-scene. In the prior art, single modalit...

AI Technical Summary

Problems solved by technology

[0003] In view of the above problems, the embodiment of the present invention is based on a monocular multi-modal depth map generation method, system, device, and storage medium to solve the technical problem that the existing depth map acquisition methods cannot meet the requirements of all-weather multi-scenario

Images