An unsupervised multi-view and multi-modal smart glasses image registration method and device

Abstract

The present invention provides an image registration method for unsupervised multi-view and multi-modal smart glasses, comprising: alternately training an auxiliary feature extraction network and a high-dimensional feature similarity discrimination network, and using the auxiliary feature extraction network to extract the same modal image and / or Or different modal images, the same perspective and / or different perspective image pairs to train the high-dimensional feature similarity discrimination network; fix the high-dimensional feature similarity discrimination network and auxiliary feature extraction network, train the registration network, among which, the registration network The input is the image to be registered, the template image and the difference image between the two, the height output by the auxiliary feature extraction network inserted in the middle layer is the feature vector, and the output is the dense registration field; the registration network and the auxiliary feature extraction network are used for registration. The image is predicted to obtain the registered image.

Description

technical field [0001] The present invention relates to the field of computers, in particular to a method and device for image registration of unsupervised multi-view and multi-modal smart glasses. Background technique [0002] Obtaining environmental images is an important condition for obstacle avoidance. At present, there are various devices for obtaining images, such as RGB cameras, structured light cameras, TOF cameras, etc. Each camera has its own advantages and disadvantages and applicable scenarios. It is impossible to use a certain camera alone. Adapt to the actual scene, so in the obstacle avoidance scene, multiple cameras are usually used in combination. The imaging results of various cameras vary greatly due to their different placement positions, imaging time, and imaging forms. Therefore, the fusion of multi-view and multi-modal image information is an important technical link in obstacle avoidance tasks. SUMMARY OF THE INVENTION [0003] The present inventi...

AI Technical Summary

Problems solved by technology

[0002] Obtaining environmental images is an important condition for obstacle avoidance. Currently, there are many devices for obtaining images, such as RGB cameras, structured light cameras, and TOF cameras. Each camera has its own advantages and disadvantages and applicable scenarios. Using a single camera alone cannot Adapt to the actual scene, so in the obstacle avoidance scene, multiple cameras are usually used in combination

Images