Two-stage network image text cross-media retrieval method

A cross-media, level network technology, applied in still image data retrieval, metadata still image retrieval, unstructured text data retrieval, etc., to achieve the effect of high recall rate

Active Publication Date: 2019-07-26
GUANGXI NORMAL UNIV
View PDF14 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, for an item in one modality, there may exist multiple semantically different items with the same modality, simply matching representations through common subspaces is not enough, and better networks are needed model to match representation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Two-stage network image text cross-media retrieval method
  • Two-stage network image text cross-media retrieval method
  • Two-stage network image text cross-media retrieval method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail by taking cross-media retrieval as an example below.

[0035] The present invention proposes a kind of image text cross-media retrieval method of two-level network, and it comprises the following steps:

[0036] Step 1. Construction of a cross-media two-level model stage:

[0037] The constructed cross-media two-level model includes a global generative adversarial network and a local cross-media attention network. The present invention constructs a global generative adversarial network and a local cross-media attention network to explore multi-layer alignments, which contain two sub-networks for global and local respectively. Using multi-level alignment for mutual promotion, complementary cues for cross-media related learning can be learned, and different representations for cross-media retrieval can be learned.

...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a two-stage network image text cross-media retrieval method, which comprises the following steps of: firstly, exploring two-stage alignment by constructing a cross-media two-stage network, which respectively comprises two subnets for global and local; and training the cross-media two-stage model by using the training data set to determine network parameters in the cross-media two-stage model, thereby obtaining the trained cross-media two-stage mode; And finally, carrying out similarity retrieval on the to-be-retrieved image and the to-be-retrieved text by using the trained cross-media two-stage model. Experiments show that the cross-media retrieval method achieves a good effect in the application of cross-media retrieval.

Description

technical field [0001] The invention relates to the field of computer cross-media retrieval, in particular to a two-level network image text cross-media retrieval method. Background technique [0002] Cross-media is manifested not only in the mixed coexistence of complex media objects including network text, images, audio, and video, but also in the formation of complex associations and organizational structures of various media objects, and in the fact that media objects with different modalities cross media or The platform is highly interactive and integrated. "Cross-media" can express the same semantic information from their respective sides, and can reflect specific content information more comprehensively than a single media object and its specific modality. The same content information cross-spreads and integrates across various media objects. Only by integrating and analyzing these multi-modal media can we understand the content information contained in this cross-me...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/583G06F16/58G06F16/33
CPCG06F16/583G06F16/5866G06F16/334
Inventor 李志欣凌锋张灿龙周韬
Owner GUANGXI NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap