Position recognition model construction method and system based on multi-view cross-modal matching

By employing a multi-view cross-modal matching method, combined with panoramic images and natural language descriptions, the accuracy and computational complexity issues of existing visual position recognition technologies in complex environments are addressed. This results in high-precision and robust position recognition, applicable to fields such as autonomous driving and robot navigation.

CN119887911BActive Publication Date: 2026-06-12QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
Filing Date
2024-12-25
Publication Date
2026-06-12

AI Technical Summary

Technical Problem

Existing visual position recognition technologies suffer from low accuracy, loss of feature details, and high computational complexity in complex environments. Some methods rely on point cloud data acquisition, which is costly. Furthermore, the language descriptions of cross-modal position recognition methods are too simplified and cannot meet the needs of complex scenarios.

Method used

By segmenting 360° images acquired by a panoramic camera into multiple viewpoints, natural language descriptions are generated. Text features are extracted using GPT-4 and frozen T5 models, and image features are extracted by combining ViT and Sinkhorn algorithms. Cross-modal matching is achieved through contrastive learning and multi-view feature stitching, optimizing feature distance and similarity.

Benefits of technology

It achieves high-precision and highly adaptable location recognition in complex scenarios, improves the robustness and computational efficiency of the system, supports matching in cases where some text or images are missing, significantly improves positioning accuracy, and is applicable to fields such as autonomous driving, robot navigation, and logistics delivery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119887911B_ABST
    Figure CN119887911B_ABST
Patent Text Reader

Abstract

The application relates to a position recognition model construction method and system based on multi-view cross-modal matching, relates to the technical field of computer vision and natural language processing, and aims at the problem that a traditional visual position recognition method is difficult to maintain high precision in a complex environment and a multi-view scene and cannot effectively process natural language description. In order to solve the problem, the application combines multi-view images and natural language text description, adopts text coding and visual coding to respectively extract features of the text and the images, then uses a clustering algorithm to cluster the image features, and splices the multi-view image features of each position into global image features, and finally, position matching is performed by calculating the similarity between the text features and the image features. By combining visual and text information, the application solves the problem of poor robustness and accuracy of the traditional method in a complex scene and view change, and can be widely applied to the navigation field of unmanned systems.
Need to check novelty before this filing date? Find Prior Art