An antagonistic cross-media retrieval method based on bilingual semantic space

An adversarial, cross-media technology, applied in the fields of pattern recognition, multimedia retrieval, and natural language processing.

Active Publication Date: 2019-02-15
PEKING UNIV SHENZHEN GRADUATE SCHOOL
View PDF3 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Most of the existing methods map heterogeneous features to a single homogeneous space to eliminate the "semantic gap". However, such processing is accompanied by a la

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An antagonistic cross-media retrieval method based on bilingual semantic space
  • An antagonistic cross-media retrieval method based on bilingual semantic space
  • An antagonistic cross-media retrieval method based on bilingual semantic space

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0064] Below in conjunction with accompanying drawing, further describe the present invention through embodiment, but do not limit the scope of the present invention in any way.

[0065] The present invention provides an adversarial cross-media retrieval method based on dual semantic spaces. By establishing text subspace and image subspace, internal features of different modalities are retained respectively, and rich semantic information in multimedia data is mined through confrontational training. So as to realize effective cross-media retrieval.

[0066] The method provided by the invention includes: a feature generation process, a dual semantic space construction process and an adversarial semantic space optimization process; figure 1 Shown is the flow process of method provided by the present invention, and concrete steps are as follows:

[0067] 1) Assuming that there are n sets of training data, the image and text data are sent to the CNN network and the BoW model respe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an antagonistic cross-media retrieval method based on bilingual meaning space, which relates to the technical fields of pattern recognition, natural language processing, multimedia retrieval and the like. Including: feature generation process, bilingual semantic space construction process and antagonistic semantic space optimization process. The invention realizes that theoriginal image and the text information are kept to the maximum extent while the semantic gap is eliminated by establishing an isomorphic bilingual meaning space, namely a text subspace and an image subspace. And through antagonism training to optimize the distribution of isomorphic subspace data, mining the rich semantic information in multimedia data, and fitting the vector distribution of different modes in the semantic space under the condition that the categories remain unchanged and the modes can be distinguished. The method of the invention can effectively eliminate the heterogeneity ofdifferent modal information, realize effective cross-media retrieval, and has wide market demand and application prospect in the fields of picture and text retrieval, pattern recognition and the like.

Description

technical field [0001] The present invention relates to technical fields such as pattern recognition, natural language processing, and multimedia retrieval, and in particular to an adversarial cross-media retrieval method based on dual semantic spaces, which mainly uses feature mapping of public spaces to eliminate semantic gaps, and data of different modalities Matching is carried out to achieve the purpose of retrieval, and the effectiveness of this method is verified in the classic database of cross-media retrieval. Background technique [0002] In recent years, Internet technology has developed rapidly, followed by the explosive growth of multimedia information. Users are more inclined to obtain multimedia information results through retrieval. Information related to the state, such as images of lions, roars of lions, and videos related to lions, etc. From this point of view, traditional retrieval techniques cannot meet users' requirements for diversity and comprehensiv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/48G06F16/435
CPCG06F16/435G06F16/48
Inventor 王文敏夏雅娴韩梁王荣刚李革高文
Owner PEKING UNIV SHENZHEN GRADUATE SCHOOL
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap