Multi-modal image retrieval method and device, electronic equipment and readable storage medium

By acquiring and processing image and text information, generating comprehensive features and generating sentence-level prompts through text encoders, the problem of low image retrieval accuracy and generalization in the prior art is solved, and more efficient and accurate image retrieval is achieved.

CN119691213APending Publication Date: 2025-03-25SHENZHEN XUMI YUNTU SPACE TECH CO LTD
0 Cites 1 Cited by

Patent Information

Application Number
CN202411625265.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-13
Publication Date
2025-03-25

AI Technical Summary

Technical Problem

In the prior art, the accuracy and generalization of image retrieval results are low, especially when dealing with multi-object and complex text descriptions, it is impossible to effectively decouple and reflect complex changes, and mainly focuses on direct relationships and ignores richer associations.

Method used

By obtaining the image information and text information of the object to be retrieved, the image embedding features and text embedding features are determined, and the image information is input to the multimodal encoder to obtain the image description features. Then, sentence-level prompts are generated through the text encoder based on the comprehensive features, enhancing the cross-modal capability of image retrieval.

Benefits of technology

It improves the accuracy and efficiency of image retrieval, enhances the text-based image retrieval capability, improves the accuracy and generalization capability of image retrieval, and can more effectively decouple complex changes.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of image processing, and provides a multi-modal image retrieval method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring image information and text information of a to-be-retrieved object; determining an image embedding feature based on the image information, determining a text embedding feature based on the text information, and inputting the image information to a preset multi-mode encoder to obtain an image description feature for describing the image information; obtaining a comprehensive feature based on the image embedding feature, the text embedding feature and the image description feature; processing the comprehensive feature through a preset text encoder to obtain a target feature; and obtaining to-be-matched image features corresponding to each target image in the target image library, and determining a target retrieval result of the to-be-retrieved object based on the to-be-matched image features and the target features. Through the method provided by the invention, the precision and generalization ability of image retrieval are improved.
Need to check novelty before this filing date? Find Prior Art