Cross-modal retrieval method based on multilevel feature representation alignment

Abstract

The invention discloses a cross-modal retrieval method based on multilevel feature representation alignment, and relates to the technical field of cross-modal retrieval. In the cross-modal fine-grained accurate alignment stage, the global similarity, the local similarity and the relation similarity between two different modal data of an image and a text are calculated respectively, the image-text comprehensive similarity is obtained through fusion, in the neural network training stage, a corresponding loss function is designed, cross-modal structure constraint information is mined, parameter learning of a retrieval model is constrained and supervised from multiple angles, and finally a retrieval result of a test query sample is obtained according to the image-text comprehensive similarity, so that the accuracy of cross-modal retrieval is effectively improved by introducing a fine-grained association relationship between two different modal data of an image and a text, and the method has wide market requirements and application prospects in the fields of image-text retrieval, pattern recognition and the like.

Description

technical field [0001] The invention relates to the technical field of cross-modal retrieval, in particular to a cross-modal retrieval method based on multi-level feature representation alignment. Background technique [0002] With the rapid development of new-generation Internet technologies such as mobile Internet and social networks, multi-modal data such as text, images, and videos have shown explosive growth. The cross-modal retrieval technology aims to realize the spanning retrieval between different modal data by mining and utilizing the associated information between different modal data, and its core is to realize the similarity measurement between cross-modal data. In recent years, cross-modal retrieval technology has become a research hotspot at home and abroad, and has attracted extensive attention from academia and industry. It is one of the important research fields of cross-modal intelligence and an important direction for the future development of information...

AI Technical Summary

Problems solved by technology

[0006] In order to solve the above-mentioned problems existing in the prior art, the present invention provides a cross-modal retrieval method based on multi-level feature representation alignment, which can accurately measure the similarity between images and texts through cross-modal multi-level representation associations, effectively Provide retrieval accuracy, so as to solve the technical problems that the existing cross-modal retrieval methods are not precise enough, and the cross-modal correlation is not sufficient. At the same time, the training of the retrieval model is supervised by using the cross-modal structural constraint information

Images