Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A Cross-media Retrieval Method Based on Hybrid Transfer Network

A hybrid migration and cross-media technology, applied in multimedia data retrieval, multimedia data query, semantic analysis, etc., can solve the problems of restricting unified representation learning, insufficient training data, ignoring knowledge transfer of different media, etc., to improve retrieval accuracy, The effect of improving accuracy

Active Publication Date: 2020-12-22
PEKING UNIV
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Existing methods can only use cross-media datasets for training, which can easily cause overfitting due to insufficient training data and reduce the retrieval effect; or only perform knowledge transfer between the same media, ignoring the knowledge transfer between different media, It makes the knowledge transfer process not comprehensive enough, which limits the effect of unified representation learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Cross-media Retrieval Method Based on Hybrid Transfer Network
  • A Cross-media Retrieval Method Based on Hybrid Transfer Network
  • A Cross-media Retrieval Method Based on Hybrid Transfer Network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021]The present invention will be further described in detail below in conjunction with the drawings and specific embodiments.

[0022]The cross-media retrieval method based on the hybrid migration network of the present invention has a process such asfigure 1 As shown, it includes the following steps:

[0023](1) Establish a single media database containing one media type, and at the same time establish a cross-media database containing multiple media types, and divide the data in the cross-media database into a training set and a test set.

[0024]In this embodiment, the media types included in the single-media database are images, and the media types included in the cross-media database are images and text. For images, the AlexNet-based convolutional neural network structure is used as the feature extractor in the network. This method also supports other convolutional neural network structures for image feature extraction, such as VGG-19, etc.; for text, the word frequency vector is use...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a cross-media retrieval method based on a hybrid migration network. The method comprises the following steps that 1, a single-media database and a cross-media database are set up, and data in the cross-media database is divided into a training set and a test set; 2, the hybrid migration network is trained by means of data in the single-media database and the training set of the cross-media database, and used for learning unified representation of different media data; 3, by means of the trained hybrid migration network, unified representation of the data in the test set of the cross-media database is obtained, and then the cross-media similarity is calculated; 4, one media type in the cross-media test set is used as a query set, the other media type serves as a retrieval database to conduct retrieval, and the final retrieval result is obtained according to the similarity. Accordingly, knowledge transfer from single-media to cross-media is achieved, the unified representation more suitable for cross-media retrieval is generated by emphasizing semantic association of a target domain, and the accuracy rate of the cross-media retrieval is increased.

Description

Technical field[0001]The invention belongs to the field of multimedia retrieval, and specifically relates to a cross-media retrieval method based on a hybrid migration network.Background technique[0002]With the progress of human civilization and the development of science and technology, the rapid growth of multimedia data such as images, texts, videos and audios has gradually become the main form of information storage and dissemination. In this case, cross-media retrieval has become one of the important applications of artificial intelligence. Cross-media search is a new form of search, which can return search results with relevant semantics and different media types based on user queries of any media type. For example, the user can use an image as a query to retrieve related text, or use the text as a query to retrieve an image that matches its description. Compared with single-media retrieval, cross-media retrieval can provide Internet users with a more flexible retrieval experi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/43G06F40/30G06N5/02
CPCG06N5/02G06F16/43G06F40/30
Inventor 黄鑫彭宇新
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com