Multi-modal image registration method and device based on generative model and self-supervised learning, and medium

By combining generative models and self-supervised learning methods with regional feature intensity and structural importance for multimodal image registration, the problems of low accuracy and high computational complexity in existing technologies are solved, achieving efficient and accurate image alignment, which is applicable to fields such as remote sensing and urban planning.

CN120976274APending Publication Date: 2025-11-18ZHEJIANG LANYING INTELLIGENT TECH CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202511083768.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-04
Publication Date
2025-11-18

AI Technical Summary

Technical Problem

Existing multimodal image registration methods face problems of low registration accuracy and high computational complexity when processing images acquired by different types of sensors, especially due to difficulties in feature extraction and inaccurate matching caused by modal differences.

Method used

We employ a generative model and self-supervised learning approach. We use the Pix2Pix-Transformer generative model for mode transformation and combine it with the DINO self-supervised learning model to extract global and local features for adaptive regional registration. We then assign weights based on the strength of regional features and the importance of structures to implement high-precision nonlinear registration and low-precision linear registration.

Benefits of technology

It improves the accuracy and efficiency of multimodal image registration, reduces computational resource consumption, and is suitable for multimodal image registration tasks in fields such as remote sensing images and urban planning.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a multi-modal image registration method and device based on a generative model and self-supervised learning, and a medium, and the method comprises the steps: carrying out the modal conversion of a source image through employing a Pix2Pix-Transform generative model, extracting global and local features from the source image and a target image through a DINO model in an image registration process, carrying out the region division of the image based on the features, and carrying out the recognition of a target image through a DINO model; for each region, weighted distribution is carried out according to the feature intensity and importance of the region, so that the key region can obtain higher weight and precision in the registration process, and the weight of the background region can be properly reduced. A region self-adaptive registration technology is adopted, high-precision non-linear registration is carried out on a key region, low-precision linear registration is carried out on a background region, and the overall registration efficiency is ensured. According to the self-adaptive weighted registration method, the overall precision and efficiency of image registration are effectively improved, and particularly, when a multi-modal image is processed, the registration difficulty caused by modal difference can be remarkably reduced.
Need to check novelty before this filing date? Find Prior Art