Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and system for cross-modal retrieval of multimedia data using tag hierarchy information

A multimedia data and modal technology, applied in the field of cross-media retrieval, can solve problems such as ignoring label level information and not considering cross-layer label related information, so as to achieve the effect of quality assurance

Active Publication Date: 2021-04-27
SHANDONG UNIV
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of the current existing technologies ignore the hierarchical information of labels
The inventors found that although few methods attempt to use this hierarchical information during the learning process, these cross-modal hashing methods have the following disadvantages: they generate hierarchical hash codes for each layer of the label hierarchy, and do not Consider the association information between cross-layer tags

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for cross-modal retrieval of multimedia data using tag hierarchy information
  • Method and system for cross-modal retrieval of multimedia data using tag hierarchy information
  • Method and system for cross-modal retrieval of multimedia data using tag hierarchy information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0035] This embodiment provides a method for cross-modal retrieval of multimedia data using label hierarchy information;

[0036] Such as figure 1 As shown, the cross-modal retrieval method of multimedia data using label hierarchical information includes:

[0037] S101: Obtain the first modal multimedia data to be retrieved;

[0038] S102: Perform feature extraction on the first modal multimedia data to be retrieved to obtain a first hash code;

[0039] S103: Calculate the distance between the first hash code and the known hash codes corresponding to all the pre-stored multimedia data of the second modality; select the multimedia data of the second modality corresponding to several hash codes with the closest distance, output as search results.

[0040] As one or more embodiments, in S101, the first modality multimedia data to be retrieved includes but not limited to: one or more of text data, image data, audio data or video data.

[0041] As one or more embodiments, in S1...

Embodiment 2

[0140] This embodiment provides a multimedia data cross-modal retrieval system utilizing label hierarchy information;

[0141] A cross-modal retrieval system for multimedia data utilizing tag-level information, including:

[0142] An acquisition module configured to: acquire the first modality multimedia data to be retrieved;

[0143] A feature extraction module configured to: perform feature extraction on the first modality multimedia data to be retrieved to obtain a first hash code;

[0144] The retrieval output module is configured to: calculate the distance between the first hash code and the known hash codes corresponding to all the multimedia data of the pre-stored second modality; Two-modal multimedia data is output as a retrieval result.

[0145]It should be noted here that the above acquisition module, feature extraction module and retrieval output module correspond to steps S101 to S103 in Embodiment 1, and the examples and application scenarios implemented by the ...

Embodiment 3

[0149] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are programmed Stored in the memory, when the electronic device is running, the processor executes one or more computer programs stored in the memory, so that the electronic device executes the method described in Embodiment 1 above.

[0150] It should be understood that in this embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate array FPGA or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multimedia data cross-modal retrieval method and system using label hierarchical information, comprising: acquiring the first modal multimedia data to be retrieved; performing feature extraction on the first modal multimedia data to be retrieved to obtain the first hash code; the known hash codes corresponding to the first hash code and all the pre-stored multimedia data of the second modality are subjected to distance calculation; the multimedia data of the second modality corresponding to the nearest several hash codes are selected, output as search results.

Description

technical field [0001] The present application relates to the technical field of cross-media retrieval, in particular to a method and system for cross-modal retrieval of multimedia data using tag level information. Background technique [0002] The statements in this section merely mention the background art related to this application, and do not necessarily constitute the prior art. [0003] With the explosive growth of multimedia data, data is usually represented in multiple modalities, such as images and texts. In the face of massive data, it is usually necessary to perform fast similarity comparison, which is the basic operation of managing and using data. Therefore, there is a growing need for fast cross-modal retrieval. To meet this need, cross-modal hashing methods that use data from one modality to retrieve similar samples in another modality have been proposed. [0004] Cross-modal hash learning belongs to hash learning and has the advantages of hash learning. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/432G06N3/04G06N3/08
CPCG06F16/432G06N3/08G06N3/045
Inventor 罗昕詹雨薇许信顺
Owner SHANDONG UNIV