Cross-media search method based on isomorphic subspace mapping and optimization

A homogeneous subspace and cross-media technology, applied in the field of cross-media retrieval based on homogeneous subspace mapping and optimization, to achieve the effect of good retrieval efficiency

Active Publication Date: 2014-08-20
WUHAN UNIV OF SCI & TECH
View PDF4 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of these current research works rely on direct semantic associations such as text annotations and web page links to establish association models between different types of multimedia samples such as images, audio, and video, and rarely analyze multimedia data from the level of underlying content characteristics. Latent Semantic Relations in Isomorphic Subspaces

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-media search method based on isomorphic subspace mapping and optimization
  • Cross-media search method based on isomorphic subspace mapping and optimization
  • Cross-media search method based on isomorphic subspace mapping and optimization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0067] Such as figure 1 As shown, the cross-media retrieval method based on isomorphic subspace mapping and optimization in this embodiment, its specific steps are as follows:

[0068] The first step, isomorphic subspace mapping based on audiovisual feature analysis

[0069] The underlying content features of different types of multimedia data are extracted, and the correlation-preserving mapping is performed in the high-dimensional kernel space to obtain the isomorphic subspace Z.

[0070] (1) Extract three visual features of color histogram, color aggregation vector and Tamura directionality from the image database to obtain the visual feature matrix A;

[0071] Extract the four auditory features of centroid, attenuation cut-off frequency, spectral flow and root mean square from the audio database, and use the method of fuzzy clustering to index the auditory features, and unify the auditory features of each audio sample to the same dimension, Get the auditory feature matri...

Embodiment 2

[0114] A method for cross-media retrieval based on isomorphic subspace mapping and optimization. as attached figure 2 As shown, taking the "explosion" audio clip as a query example to perform cross-media retrieval, the specific steps are as follows:

[0115] The first step, isomorphic subspace mapping based on audiovisual feature analysis

[0116] The underlying content features of different types of multimedia data are extracted, and the correlation-preserving mapping is performed in the high-dimensional kernel space to obtain the isomorphic subspace Z.

[0117] (1) Collect image database and audio database, including the following 8 different semantic categories: explosion, airplane, lightning, insect, car, dog, monkey, elephant, each category includes 80 images and 40 audio segments; Extract the three visual features of color histogram, color aggregation vector and Tamura directionality from the database, and obtain the visual feature matrix A, where the image samples of...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cross-media search method based on isomorphic subspace mapping and optimization. The method comprises the steps that firstly, visual features and audio features are extracted from an image database and an audio database respectively to obtain a corresponding visual feature matrix A and a corresponding audio feature matrix B, and typical correlation analysis based on high-dimensional kernel space is adopted for mapping to obtain isomorphic subspace Z on this basis; then, the distance relation of an image sample and an audio sample in the isomorphic subspace Z is analyzed, and then a cross-media weighting neighbour image G (V, E) is constructed to obtain a corresponding weight matrix W and a corresponding Laplacian matrix L; an objective function is solved to obtain the value of optimized isomorphic subspace Y; finally, according to the cosine distance in the optimized isomorphic subspace Y, the image sample and the audio sample which are most similar to a search sample are calculated as a cross-medial search result to be returned. According to the method, the isomorphic subspace capable of containing the image sample and the audio sample at the same time is constructed, optimization is carried out, and the good cross-medial search result is obtained.

Description

technical field [0001] The invention relates to the technical field of multimedia content analysis and semantic understanding, in particular to a cross-media retrieval method based on isomorphic subspace mapping and optimization. Background technique [0002] With the rapid development of multimedia technology and network technology, text is no longer the main multimedia content that people come into contact with. Different types of multimedia data such as images, audio and video have spread across various network terminals. These rich multimedia data express a large amount of semantic information and are intricately related to each other, such as: the statistical relationship on the underlying content features, the link relationship between web pages, and so on. How to effectively manage a large amount of different types of multimedia data and provide flexible and efficient cross-media retrieval is a new challenge in the field of multimedia content analysis and semantic un...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06F17/27
CPCG06F16/583G06F16/683
Inventor 张鸿聂加梅张延鹏
Owner WUHAN UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap