Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Social image retrieval method and system based on missing multi-modal hash

A social image, multi-modal technology, applied in still image data retrieval, metadata still image retrieval, special data processing applications, etc., can solve the problems of inaccurate and lack of retrieval.

Active Publication Date: 2020-05-01
SHANDONG ZHENGZHONG COMP NETWORK TECH CONSULTING
View PDF6 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In order to solve the deficiencies of the prior art, the present disclosure provides a method and system for social image retrieval based on missing multimodal hashing, by learning a shared latent representation for complete paired data and a unique latent representation for missing data at the same time , to analyze the relationship between the image and the label, thereby generating a hash code, which solves the problem of inaccurate retrieval caused by images lacking label descriptions and text labels lacking corresponding images in social image retrieval in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Social image retrieval method and system based on missing multi-modal hash
  • Social image retrieval method and system based on missing multi-modal hash
  • Social image retrieval method and system based on missing multi-modal hash

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0063] Such as figure 1 As shown, Embodiment 1 of the present disclosure provides a social image retrieval method based on unsupervised missing multimodal hashing, including:

[0064] S1: Obtain a multimodal retrieval dataset, where each sample includes paired image and text data of two modalities, and divide them into training set, test set and database set. Construct missing data sets for training set, test set and database set respectively;

[0065] This disclosure considers a social image dataset, including social image features and text features marked as labels Both image features and text features contain two parts: fully paired data features and missing data features. is n 1 social image features missing corresponding labels, is n 2 Text features of missing images, where d 1 and d 2 are the dimensions of image and text features, respectively. The goal of this example is to learn a shared hash code B ∈ [-1,1] n×r , where r represents the length of the hash...

Embodiment 2

[0136] Such as figure 2 As shown, Embodiment 2 of the present disclosure provides a social image retrieval method based on supervised missing multimodal hashing, including:

[0137] S1: Obtain multimodal retrieval datasets and construct missing datasets;

[0138] S2: Input the original data of the two modalities into the constructed deep feature extraction model to perform multimodal extraction on the training data set, and then map the extracted multimodal features to a low-dimensional space using the Gaussian kernel function;

[0139] S3: Use the pairwise semantic matrix to guide the projection learning process, and construct an objective function based on supervised missing multimodal hashing on this dataset;

[0140] Existing multimodal hashing methods mostly focus on unsupervised methods, while the development of supervised multimodal hashing methods is seriously lagging behind. Since supervised hashing utilizes discriminative label information to preserve the semantic...

Embodiment 3

[0184] Such as image 3 As shown, Embodiment 3 of the present disclosure provides a social image retrieval system based on missing multimodal hash, including:

[0185] The image preprocessing module is configured to: obtain a multimodal retrieval data set, wherein each sample includes paired image and text two modal data, and divide them into a training set, a test set and a database set. Construct missing data sets for training set, test set and database set respectively;

[0186] The nonlinear feature representation module is configured to: input the original data of the two modalities into the constructed deep feature extraction model to perform multimodal extraction on the training data set, and then use Gaussian The kernel function is mapped to a low-dimensional space;

[0187] The objective function construction module is configured to: for the training multimodal data set, construct the objective function f based on the unsupervised missing multimodal hash on the data...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a social image retrieval method and system based on missing multi-modal hash. A shared potential representation is learned for completely paired data; a unique potential representation is learned for missing data; the relations between different modes of images and labelsare explored, an online hash retrieval mode suitable for completely paired data and missing data at the same time is constructed, hash codes are directly solved by designing a new discrete optimization strategy, the quantization error of a relaxation strategy in the prior art is effectively reduced, andthe retrieval performance is improved; on the basis of an unsupervised missing multi-modal hash method, the invention is expanded to a supervised learning mode, an asymmetric hash learning method is used for guiding a projection learning process, the identification capability of hash codes is improved, the binary hash codes are directly solved, the speed is high, the operation is simple, and the learning efficiency is ensured.

Description

technical field [0001] The present disclosure relates to the technical field of multimodal retrieval, and in particular to a social image retrieval method and system based on missing multimodal hash. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. [0003] With the rapid development of mobile Internet technology, more and more people like to upload their photos to social networking sites. Social networking sites allow users to actively upload pictures and annotations with descriptive tags, and have become the most popular interactive platform with the highest user participation in the Internet age. However, the explosive growth of social images also leads to a great challenge, how to perform effective image retrieval from huge social image databases. [0004] Multimodal hashing can encode multimodal features from different modalities into compa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/58
CPCG06F16/5866
Inventor 朱磊郑超群
Owner SHANDONG ZHENGZHONG COMP NETWORK TECH CONSULTING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com