Cross-modal data retrieval method and system based on graph regularization and modal independence

A data retrieval, cross-modal technology, applied in multimedia data retrieval, digital data information retrieval, multimedia data query and other directions, to achieve the effect of improving accuracy

Active Publication Date: 2019-08-30
SHANDONG NORMAL UNIV
View PDF12 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this context, the traditional text search methods can no longer meet people's special needs, so cross-media retrieval is proposed, and the correlation between different modalities in cross-media retrieval has also attracted more and more attention.
[0004] However, the inventors discovered during the research and development process that most retrieval methods, such as text retrieval, image retrieval, and video retrieval, etc., mostly focus on single-modal retrieval, and can only provide queries and retrievals of the same modality.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal data retrieval method and system based on graph regularization and modal independence
  • Cross-modal data retrieval method and system based on graph regularization and modal independence
  • Cross-modal data retrieval method and system based on graph regularization and modal independence

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0063] According to an aspect of one or more embodiments of the present disclosure, a cross-modal data retrieval method based on graph regularization and modality independence is provided.

[0064] Such as figure 1 As shown, a cross-modal data retrieval method based on graph regularization and modality independence, the method includes:

[0065] Step 1. Receive the original data of different modes and carry out feature extraction, set up the multimodal data set that is made up of one-to-one corresponding image text pair, described multimodal data set comprises training set and test set;

[0066] Step 2. By optimizing the predefined objective function, the feature matrix of different modal data in the training set is projected to the common subspace to obtain the image projection matrix and the text projection matrix;

[0067] Step 3. according to described image projection matrix and text projection matrix, the feature matrix of different mode data in test set is projected to...

Embodiment 2

[0132] According to an aspect of one or more embodiments of the present disclosure, there is provided a computer-readable storage medium.

[0133] A computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor of a terminal device and executing the cross-modal data retrieval method based on graph regularization and modality independence.

Embodiment 3

[0135] According to an aspect of one or more embodiments of the present disclosure, a terminal device is provided.

[0136] A terminal device, which includes a processor and a computer-readable storage medium, the processor is used to implement instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the described one A Cross-Modal Data Retrieval Method Based on Graph Regularization and Modality Independence.

[0137] These computer-executable instructions, when executed in a device, cause the device to perform the methods or processes described in accordance with various embodiments in the present disclosure.

[0138] In this embodiment, a computer program product may include a computer-readable storage medium carrying computer-readable program instructions for performing various aspects of the present disclosure. A computer readable storage medium may be a tan...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cross-modal data retrieval method and system based on graph regularization and modal independence, and the method comprises the steps of receiving the original data of different modalities, carrying out the feature extraction, building a multi-modal data set which consists of the image text pairs in one-to-one correspondence, wherein the multi-modal data set comprises a training set and a test set; projecting the feature matrixes of different modal data in the training set to a public subspace through optimizing a predefined objective function to obtain an image projection matrix and a text projection matrix; according to the image projection matrix and the text projection matrix, projecting the feature matrixes of different modal data in the test set to a publicsubspace; calculating the similarity between the projected matrix and other projection matrixes in the public subspace; and performing descending sort according to the similarity to obtain the data corresponding to the plurality of first feature projection matrixes, and performing the cross-mode retrieval.

Description

technical field [0001] The disclosure belongs to the technical field of data retrieval, and relates to a cross-modal data retrieval method and system based on graph regularization and modality independence. Background technique [0002] The statements in this section merely provide background technical information related to the present disclosure and do not necessarily constitute online technology. [0003] In the era of big data with the rapid growth of multimedia information, the types of media information have developed from the original text to images, texts, and videos; the ways of obtaining information have also evolved from broadcasting and newspapers to websites, Weibo, WeChat, and QQ. In this context, the traditional text search methods can no longer meet people's special needs, so cross-media retrieval is proposed, and the correlation between different modalities in cross-media retrieval has attracted more and more attention. [0004] However, the inventors disco...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/43G06F16/432
CPCG06F16/43G06F16/434
Inventor 计华计晓赟王冠华
Owner SHANDONG NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap