Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-modal retrieval method and system based on weak supervision hash learning

A weakly supervised, multi-modal technology, applied in the field of big data retrieval, can solve complex optimization problems, do not solve the semantic gap and other problems, achieve the effect of improving performance, bridging the cross-modal semantic gap, and improving retrieval accuracy

Active Publication Date: 2022-04-12
SHANDONG JIANZHU UNIV
View PDF7 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Existing hash retrieval methods for weakly supervised scenarios have the following problems: (1) The graph-based semi-supervised hash method adopts a label propagation framework, which can mine potential label information, but this framework also brings complex optimization problems , and ignore the incomplete pairing information between modalities
(2) Weak pairing cross-modal hashing method can deal with incomplete inter-modal pairing information, but this type of method simply uses the intra-modal neighborhood relationship to approximate the inter-modal similarity, but does not solve the problem of semantic gap

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal retrieval method and system based on weak supervision hash learning
  • Multi-modal retrieval method and system based on weak supervision hash learning
  • Multi-modal retrieval method and system based on weak supervision hash learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] refer to figure 1 with figure 2 , the present embodiment provides a multimodal retrieval method based on weakly supervised hash learning, which specifically includes the following steps:

[0045] S101: Obtain samples to be retrieved, and perform hash code calculation on them.

[0046]For image samples , generating its hash code ; for text samples , generating its hash code ; Among them, the sign function is a quantization function, the purpose is to quantize the real value output by the network into a discrete 0 / 1 code, which is convenient for subsequent retrieval.

[0047] S102: Perform a 0 / 1 XOR operation on the hash code of the sample to be retrieved and the hash code in the retrieval database to calculate the Hamming distance, and return similar data in order of the Hamming distance from small to large.

[0048] What needs to be explained here is that the construction of the retrieval database is constructed in an offline manner.

[0049] In the specific...

Embodiment 2

[0088] Such as image 3 As shown, the present embodiment provides a multimodal retrieval system based on weakly supervised hash learning, which specifically includes the following modules:

[0089] (1) Hash code calculation module, which is used to obtain samples to be retrieved and perform hash code calculation on them;

[0090] (2) Online retrieval module, which is used to perform 0 / 1 XOR operation between the hash code of the sample to be retrieved and the hash code in the retrieval database, calculate the Hamming distance, and return the similarity according to the Hamming distance from small to large data;

[0091] Wherein, the construction process of the retrieval database is:

[0092] Based on the intra-modal pairwise similarity, the inter-modal pairwise similarity and the completed label information of each modality, the objective function of semi-supervised semi-pair cross-modal hashing is established;

[0093] Obtain hash representation by optimizing the objective f...

Embodiment 3

[0103] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the multimodal retrieval method based on weakly supervised hash learning as described above are implemented.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of big data retrieval, and provides a multi-modal retrieval method and system based on weak supervision hash learning. In order to solve the problem of incomplete pairing information among modals, the method comprises the following steps: acquiring a to-be-retrieved sample, and carrying out hash code calculation on the to-be-retrieved sample; performing 0 / 1 XOR operation on the hash code of the to-be-retrieved sample and the hash code in the retrieval database to calculate a Hamming distance, and returning similar data from small to large according to the Hamming distance; the construction process of the retrieval database comprises the following steps: establishing a semi-supervised and semi-paired cross-modal hash target function based on intra-modal pairing similarity, inter-modal pairing similarity and complemented label information of each modal; hash representation is obtained by optimizing an objective function of semi-supervised and semi-paired cross-modal Hash, sampling is carried out from the Hash representation, then corresponding partial cross-modal similarity information is embedded into Hash function learning, and finally, a retrieval database is generated by utilizing the embedded Hash function. According to the method, the calculation complexity is reduced, and the retrieval precision is improved.

Description

technical field [0001] The invention belongs to the technical field of big data retrieval, and in particular relates to a multimodal retrieval method and system based on weakly supervised hash learning. Background technique [0002] In order to facilitate users to quickly retrieve useful or interesting content from massive data, Internet content providers not only need to filter duplicate and similar content, but also reorder similar content searched by users. In addition, the heterogeneity of multimedia data representation brings cross-modal retrieval requirements. For example, a web page contains both image content and text information. Users may need to search images by text or text by images. Therefore, how to realize the similarity search between different modalities has become a new challenge faced by Internet companies when processing multimedia signals. [0003] Compared with traditional similarity search methods such as exhaustive methods and methods based on space...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/903G06F16/907G06N20/10
CPCY02D10/00
Inventor 刘兴波张雪凝聂秀山王少华尹义龙
Owner SHANDONG JIANZHU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com