Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Cross-modal multimedia data retrieval method and system with tags embedded in online hash

A multimedia data and multimedia technology, which is applied in multimedia data retrieval, multimedia data query, special data processing applications, etc. , the effect of reducing computational complexity

Active Publication Date: 2021-03-12
SHANDONG UNIV
View PDF15 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of the existing hashing methods are batch-based, and few online hashing methods have been proposed, resulting in low efficiency and low accuracy for cross-modal multimedia data retrieval.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal multimedia data retrieval method and system with tags embedded in online hash
  • Cross-modal multimedia data retrieval method and system with tags embedded in online hash
  • Cross-modal multimedia data retrieval method and system with tags embedded in online hash

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0035] This embodiment provides a cross-modal multimedia data retrieval method in which a tag is embedded in an online hash;

[0036] like figure 1 As shown, the cross-modal multimedia data retrieval method with tags embedded in online hashing includes:

[0037] S101: Obtain multimedia training data; obtain a multimedia training label matrix, a feature matrix of different modalities of the multimedia training data, and a feature matrix of different modalities of samples to be retrieved according to the multimedia training data;

[0038] S102: Construct a label semantic similarity block matrix based on the multimedia training label matrix; embed the label semantic similarity block matrix into the Hamming space, and obtain the hash code of the multimedia training data;

[0039] S103: According to the hash code of the multimedia training data and the feature matrix of different modalities of the multimedia training data, obtain a projection matrix for mapping each modal feature ...

Embodiment 2

[0133] This embodiment provides a cross-modal multimedia data retrieval system in which tags are embedded in online hash;

[0134] A cross-modal multimedia data retrieval system with tags embedded in online hashing, including:

[0135] The obtaining module is configured to: obtain multimedia training data; obtain multimedia training label matrix, feature matrix of different modes of multimedia training data and feature matrix of different modes of sample to be retrieved according to multimedia training data;

[0136] The building module is configured to: construct a label semantic similarity block matrix based on the multimedia training label matrix; embed the label semantic similarity block matrix into the Hamming space, and obtain the hash code of the multimedia training data;

[0137] Mapping module, it is configured to: According to the hash coding of multimedia training data and the feature matrix of the different modes of multimedia training data, obtain the projection m...

Embodiment 3

[0144] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are programmed Stored in the memory, when the electronic device is running, the processor executes one or more computer programs stored in the memory, so that the electronic device executes the method described in Embodiment 1 above.

[0145] It should be understood that in this embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate array FPGA or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a label embedded online hash cross-modal multimedia data retrieval method and system, and the method comprises the steps: obtaining a multimedia training label matrix, featurematrixes of different modals of multimedia training data, and feature matrixes of different modals of a to-be-retrieved sample according to the multimedia training data; constructing a label semanticsimilarity block matrix based on the multimedia training label matrix; embedding the label semantic similarity block matrix into a Hamming space to obtain a hash code of the multimedia training data;solving a projection matrix of mapping each modal feature of the multimedia training data to the hash code of the multimedia training data according to the hash code of the multimedia training data and the feature matrixes of different modals of the multimedia training data; obtaining hash codes of the to-be-retrieved sample according to the projection matrix and the feature matrixes of differentmodes of the to-be-retrieved sample; and calculating the distance between the hash code of the to-be-retrieved sample and the hash code of the multimedia training data, and obtaining a sample similarto the to-be-retrieved sample from the multimedia training data.

Description

technical field [0001] The present disclosure relates to the technical field of multimedia data processing, in particular to a cross-modal multimedia data retrieval method and system in which tags are embedded in online hashes. Background technique [0002] The statements in this section merely mention background art related to the present disclosure and do not necessarily constitute prior art. [0003] Nearest Neighbor Retrieval (NN) is to find the item most similar to the target data from the database according to the similarity of the data. This similarity is usually quantified to the Euclidean or Manhattan distance between the data. However, with the explosive growth of Internet multimedia data in scale and dimension, NN becomes uncomputable. Approximate Nearest Neighbor Search (ANN), as a compromise between efficiency and accuracy, is gradually replacing NN in large-scale multimedia retrieval tasks. Among them, hash learning is widely concerned as a typical ANN algor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/43
CPCG06F16/43
Inventor 许信顺王永欣罗昕
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com