Cross-media retrieval method based on Resnet-Bert network model

Abstract

The invention provides a cross-media retrieval method based on a Resnet-Bert network model. The method comprises the following steps of carrying out classified retrieval on at least one of image data,text data, video data and audio data by adopting a Resnet-Bert network model, and returning a corresponding classification result. Compared with the traditional mutual retrieval of two media types, the mutual retrieval of the four media types can realize wider market application; a Resnet convolutional neural network model with a good effect and a Bert model leading in 11 natural language processing aspects at present are adopted, and the model itself can obtain higher-layer, more abstract and richer feature expressions; the used four modals of data and information are mutually migrated, associated learning is enhanced, and stronger knowledge representation capability is realized; The invention benefits from the improvement of computer performance, the Resnet-Bert network model can achieve a better cross-media training effect and a better cross-media retrieval effect through complex calculation.

Description

technical field [0001] The present invention relates to a cross-media retrieval method based on Resnet-Bert network model, belonging to. Background technique [0002] In the era of big data, a variety of media data types, such as text, images, video, audio, etc., have become the main data forms for people to acquire knowledge. More and more users are eager to learn and master more comprehensive knowledge information through a variety of media data content and the interrelationships between them, so as to assist their own cognition and problem solving. [0003] Retrieval is one of the common ways for users to acquire knowledge. Traditional cross-media retrieval research mainly focuses on two types of media data: search for images by text and search for text by images. In fact, with the advent of the era of big data, people will generate a large amount of text data through the Internet, such as news reports, Weibo Taobao and other comment data, WeChat chat records, bullet scr...

AI Technical Summary

Problems solved by technology

However, the existing cross-media retrieval technology is limited to two kinds of media data. In fact, this kind of search can no longer meet people's increasing data retrieval needs, especially the problem of low accuracy of cross-modal retrieval.

Images