Cross-modal retrieval method based on deep correlation network

An associative network and cross-modal technology, applied in biological neural network models, multimedia data retrieval, special data processing applications, etc., can solve the problems of unstable retrieval effect, single types of deep network components, and low retrieval accuracy, and achieve Good performance, high precision, good stability

Inactive Publication Date: 2018-03-23
GUILIN UNIV OF ELECTRONIC TECH
View PDF5 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although the second type of method has been proved to be more suitable for cross-modal retrieval tasks in practice; however, the existing second type of algorithm

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal retrieval method based on deep correlation network
  • Cross-modal retrieval method based on deep correlation network
  • Cross-modal retrieval method based on deep correlation network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0106] Assume that we have o pairs of text and image data with known correspondence, that is, training set data; k text data and image data with unknown correspondence, that is, test set data; take image retrieval text as an example to illustrate, then retrieve The target is an image s in the test set, and the retrieval library contains k retrieval members in the test set, and the retrieval members are all text modal data; for example Figure 7 shown, including the following three steps:

[0107] 1) Step 701: use the initial feature method to extract the features of the text and image data of the known correspondence in the training set o to form a primary vector, and extract the features of k in the test set to the text data and image data of the unknown correspondence to form a primary vector;

[0108] The original data of different modalities has its mature initial feature extraction method; the retrieval target is the data of the image modality, and the data of the image m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cross-modal retrieval method based on a deep correlation network. The method comprises the following steps that a primary vector of first modal data and a primary vector of second modal data are obtained through an initial feature extraction method; secondly, a deep correlation network model is established and trained, and high-level representation vectors corresponding to a retrieval target and retrieval library members are obtained through the deep correlation network model; thirdly, similarity matching is conducted on the retrieval target and each retrieval memberin a retrieval library through the high-level representation vectors, that is, the Euclidean distance is calculated; fourthly, calculation results of the Euclidean distance are successfully distributed from small to large, and a cross-modal retrieval result list of the retrieval target is obtained. According to the method, multiple layers of corresponding incidence relations are built between dataof different modals, and meanwhile various neural networks are fused, so that a deep model has a better representation effect; the precision of the cross-modal retrieval is higher, and the stabilityis better.

Description

technical field [0001] The invention relates to a multimedia data retrieval technology, in particular to a cross-modal retrieval method based on a deep association network. Background technique [0002] In the era of explosive growth of multimedia information, people are more inclined to search for diversified results, not just for single-modal retrieval. For example, if you see a landscape painting, if you submit this picture to the retrieval system, the retrieval system will not only retrieve similar landscape paintings, but also retrieve information such as audio or text related to this picture at the same time, which will improve the retrieval effect. will be more influential. This process of using data from one modality to retrieve data from another modality is called cross-modal retrieval. [0003] Traditional cross-modal retrieval, such as text retrieval of images, is often based on matching the text annotation information of the image with the retrieved text, so it...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06N3/04
CPCG06F16/43G06N3/045
Inventor 蔡国永冯耀功
Owner GUILIN UNIV OF ELECTRONIC TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap