A cross-modal retrieval method, device, equipment and medium

Abstract

The embodiment of the present application discloses a cross-modal retrieval method, device, device and medium, wherein the method includes: obtaining the query text to be retrieved; searching in the multimedia database according to the query text to obtain a candidate multimedia set; according to N The object information in the candidate multimedia data generates the missing content of the query text, and obtains confirmation feedback for the missing content; optimizes the candidate multimedia set according to the confirmation feedback, and obtains the retrieval result. The embodiments of the present application are applicable to the situation that the query text is incomplete, and can effectively improve the retrieval performance and the accuracy of the retrieval results.

Description

technical field [0001] The present application relates to the field of computer technology, specifically to the field of artificial intelligence, and in particular to a cross-modal retrieval method, a cross-modal retrieval device, a cross-modal retrieval device, and a computer-readable storage medium. Background technique [0002] Cross-modal retrieval is a retrieval method in which the retrieval content to be retrieved and the retrieved retrieval results belong to different modalities; the so-called modalities can refer to document forms, including but not limited to text, images, audio and video, etc.; for example, The modality of the retrieval content to be retrieved is text, and the modality of the retrieval result is an image. [0003] Current cross-modal retrieval methods can be broadly classified into two categories, single-shot retrieval methods or interactive retrieval methods. Taking the text-image cross-modal retrieval as an example, the single retrieval method o...

AI Technical Summary

Problems solved by technology

This method is post-dependent, that is, it completely relies on the user's feedback on the previous retrieval results to improve the next retrieval results. If the user refuses to provide feedback or provides inaccurate feedback, it will affect the improvement of retrieval performance and cause the accuracy of retrieval results. lower

Practice has found that the current cross-modal retrieval methods rely entirely on the query text for retrieval, and generally have the problem of low accuracy. If there is an incomplete query text, it will also greatly negatively affect the retrieval results. , further reducing retrieval performance and retrieval accuracy

Images