Cross-modal image-text retrieval method, system and equipment

CN119357422APending Publication Date: 2025-01-24SHAANXI CHANGYIN CONSUMER FINANCE CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411272509.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2025-01-24

AI Technical Summary

Technical Problem

The prior art has problems of semantic inconsistency and poor results in cross-modal remote sensing image-text retrieval, especially in low data volume scenarios, large models and high computing resource requirements, making it difficult to directly apply.

Method used

Using context and uncertainty prompt network models, including remote sensing prompt-clip modules and uncertain estimation modules, use pre-trained CLIP multi-mode graphic models to compare learning, and through uncertain symmetrical cross-entropy loss functions, cross-crossing The modal relationship loss function and the Kullback-LeiBler's dispersal loss function are adjusted to improve the accuracy of graphic matching.

Benefits of technology

By introducing context and uncertainty perception mechanisms, the model can more deeply understand the complex relationship between remote sensing images and text, improve the accuracy and consistency of image-text matching, and reduce computational complexity and parameter training volume.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119357422A_ABST
    Figure CN119357422A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of remote sensing image-text retrieval, in particular to a cross-modal image-text retrieval method, system and device, and the method comprises the steps: obtaining target remote sensing image data and corresponding text data; inputting the remote sensing image data and the corresponding text data into a pre-trained image-text retrieval model to obtain an image-text retrieval result; the image-text retrieval model adopts a context and uncertain sensing prompt network model, the network model comprises a remote sensing prompt-CLIP module and an uncertainty estimation module, and the remote sensing prompt-CLIP module is used for performing comparative learning according to a prompt mark and transferring pre-trained CLIP knowledge into an image-text retrieval task to obtain an image-text understanding result; and the uncertainty estimation module is used for reversely adjusting the remote sensing prompt (CLIP) module according to the image-text understanding result, and aligning similarity in modes among different modes to obtain a final image-text search result. The method can comprehensively and accurately capture the features and information of the object, and the retrieval result is more accurate.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Image-text retrieval method based on prompt learning and related equipment

    CN121980013A