Cross-modal search system

Abstract

The invention discloses a cross-modal search system, which relates to the technical field of cross-modal search. The cross-modal search system comprises a cross-modal retrieval module, a resource library generation module and a quantization library, wherein the cross-modal retrieval module is used for serving as information input to obtain retrieval features and coding results input by a user, storing the information and determining corresponding high-dimensional vectors; the resource library generation module is used for establishing a corrected resource and a quantized code thereof and storing the corrected resource and the quantized code thereof into a specified modal region of a quantized library so as to provide support for retrieval operation; and the quantification library is used for storing all the to-be-retrieved resources. Fusion and supplement of various modal information are realized, semantic understanding of the information during cross-modal search can be effectively improved, the problem that resources of different modals cannot be directly and mutually retrieved is solved, and the resources can be converted into a uniform and normative computable quantification form.

Description

technical field [0001] The present invention relates to the technical field of cross-modal search, in particular to a cross-modal search system. Background technique [0002] In recent years, the development of artificial intelligence has made great breakthroughs, so that artificial intelligence has been deeply applied in many fields. Deep learning technology in artificial intelligence has developed rapidly in recent years, especially in the field of natural language processing. With the continuous development of deep learning, for some tasks, the ability of deep learning models to complete tasks has exceeded human beings. Modality refers to the existence form of data, such as text, audio, image, video and other file formats. Some data exist in different forms, but they all describe the same thing or event, and data of different modalities are often required for information retrieval. To enrich our cognition of the same thing or event, cross-modal search is used to realize...

AI Technical Summary

Problems solved by technology

The traditional single-modal search system can no longer meet the growing needs of users, because its query data and search result data are limited to a single modality

Images