Cross-modal retrieval algorithm based on text concept extension

Abstract

The invention relates to the technical field of cross-modal retrieval, in particular to a cross-modal retrieval algorithm based on text concept extension, which comprises the following steps: preprocessing a video to obtain video embedded feature representation; preprocessing the text to obtain text embedding feature representation; calculating the similarity between the video embedding feature representation and the text embedding feature representation to obtain the hidden space similarity; deepening the video through an encoder to obtain a video concept; analyzing the text by using grammar analysis to obtain an initial concept; expanding the initial concept through a text concept expansion mechanism to obtain a text concept; calculating the similarity between the video concept and the text concept to obtain the concept space similarity; performing weighted calculation on the hidden space similarity and the concept space similarity to obtain a fusion similarity model; and based on the retrieval instruction, retrieval sorting is performed by using the fusion similarity model, so that the problem that the retrieval performance is reduced due to imbalance of the video information amount and the text information amount is solved.

Description

technical field [0001] The invention relates to the technical field of cross-modal retrieval, in particular to a cross-modal retrieval algorithm based on text concept expansion. Background technique [0002] Cross-modal retrieval needs to use given modal data to find semantically related data from massive information. Therefore, the main problem of cross-modal technology is how to align the semantics of different modal data. [0003] Existing video text retrieval algorithms find a common embedding space for samples of different modalities, and realize the relationship measurement between heterogeneous modalities in this shared space. The advantage of this type of method lies in the diversity of video and text encoders, which can obtain features with strong representation of different modalities, thereby achieving more accurate retrieval. [0004] The above methods use conceptual and deep features as bridges, respectively, but lack in-depth exploration of video text retrieva...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a cross-modal retrieval algorithm based on text concept expansion, aiming to solve the problem of unbalanced video information and text information, which reduces retrieval performance

Images