Cross-media retrieval method based on hybrid migration network

A hybrid transfer and cross-media technology, applied in multimedia data retrieval, multimedia data query, instruments, etc., can solve the problems of insufficient training data, limit unified representation learning, ignore knowledge transfer of different media, etc., to improve accuracy and retrieval. The effect of accuracy

Active Publication Date: 2017-09-29
PEKING UNIV
View PDF9 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Existing methods can only use cross-media datasets for training, which can easily cause overfitting due to insufficient training data and reduce the retrieval effect; or only perform knowledge transfer between the same media, ignoring the knowledge transfer between different media, It makes the knowledge transfer process not comprehensive enough, which limits the effect of unified representation learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-media retrieval method based on hybrid migration network
  • Cross-media retrieval method based on hybrid migration network
  • Cross-media retrieval method based on hybrid migration network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0022] A cross-media retrieval method based on a hybrid migration network of the present invention, its flow is as follows figure 1 shown, including the following steps:

[0023] (1) Establish a single-media database containing one media type, and simultaneously establish a cross-media database containing multiple media types, and divide the data in the cross-media database into a training set and a test set.

[0024] In this embodiment, the media types included in the single-media database are images, and the media types included in the cross-media database are images and texts. For images, the AlexNet-based convolutional neural network structure is used as the feature extractor in the network. This method also supports other convolutional neural network structures for image feature extraction, such as VGG-19; for text, word frequen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a cross-media retrieval method based on a hybrid migration network. The method comprises the following steps that 1, a single-media database and a cross-media database are set up, and data in the cross-media database is divided into a training set and a test set; 2, the hybrid migration network is trained by means of data in the single-media database and the training set of the cross-media database, and used for learning unified representation of different media data; 3, by means of the trained hybrid migration network, unified representation of the data in the test set of the cross-media database is obtained, and then the cross-media similarity is calculated; 4, one media type in the cross-media test set is used as a query set, the other media type serves as a retrieval database to conduct retrieval, and the final retrieval result is obtained according to the similarity. Accordingly, knowledge transfer from single-media to cross-media is achieved, the unified representation more suitable for cross-media retrieval is generated by emphasizing semantic association of a target domain, and the accuracy rate of the cross-media retrieval is increased.

Description

technical field [0001] The invention belongs to the field of multimedia retrieval, and in particular relates to a cross-media retrieval method based on a hybrid migration network. Background technique [0002] With the advancement of human civilization and the development of science and technology, multimedia data such as images, texts, videos, and audios has grown rapidly, and has gradually become the main form of information storage and dissemination. In this case, cross-media retrieval has become one of the important applications of artificial intelligence. Cross-media retrieval is a new form of retrieval, which can return retrieval results with related semantics but different media types according to user queries of any media type. For example, users can use an image as a query to retrieve related text, or use text as a query to retrieve an image that matches its description. Compared with single-media retrieval, cross-media retrieval can provide Internet users with a ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F17/27G06N5/02
CPCG06N5/02G06F16/43G06F40/30
Inventor 黄鑫彭宇新
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap