Online cross-modal retrieval method and system based on similarity re-learning

A re-learning, cross-modal technology, applied in the field of cross-modal retrieval, can solve the problems of small number of paired samples, difficult to maintain, and difficult to optimize the loss function, so as to improve retrieval accuracy and avoid cumulative quantization errors Effect

Active Publication Date: 2022-03-01
SHANDONG JIANZHU UNIV
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This type of method usually uses the method of constructing a similarity graph to mine the relationship between old and new data. However, due to the large difference in the size of old and new data, the number of paired samples is very small, making the update heavily dependent on unpaired samples, resulting in " The problem of "unbalanced update" makes the loss function difficult to optimize
At the same time, when searching across modalities, there is a semantic gap between the modalities, and it is difficult to maintain the similarity between samples when crossing modalities, which makes the learning of hash codes more difficult and hinders the accuracy of cross-modal retrieval

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Online cross-modal retrieval method and system based on similarity re-learning
  • Online cross-modal retrieval method and system based on similarity re-learning
  • Online cross-modal retrieval method and system based on similarity re-learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] In one or more embodiments, an online cross-modal retrieval method based on similarity re-learning is disclosed. First, the training samples are uniformly grouped to generate flow data, and then multi-metric matrix learning is performed. After that, the combined The matrix factorization approach learns the common mapping of old and new data as well as the common representation of different modalities. At the same time, in the learning process, two kinds of supervision information are introduced: 1) through the newly learned multi-metric matrix to measure the similarity relationship between old and new samples in multiple modalities; 2) to mine the category information between new samples through label embedding . Finally, a strategy is devised to efficiently update the hash codes of the original samples. During the retrieval process, the hash code is generated for the query sample through the out-of-sample extended mapping, and compared with the hash code in the update...

Embodiment 2

[0089] In one or more embodiments, an online cross-modal retrieval system based on similarity re-learning is disclosed. The system adopts the online cross-modal retrieval method based on similarity re-learning described in Embodiment 1 to realize Online cross-modal search.

Embodiment 3

[0091] In one or more embodiments, a terminal device is disclosed, including a server, the server includes a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor executes the The program realizes the online cross-modal retrieval method based on similarity re-learning in the first embodiment. For the sake of brevity, details are not repeated here.

[0092] It should be understood that in this embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate array FPGA or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

[0093] The memory may include read-only memory a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an online cross-modal retrieval method and system based on similarity re-learning, and the method comprises the steps: obtaining an original data sample, dividing the original data sample into a plurality of groups, and constructing a training set; constructing an objective function of Hash code learning, training the objective function by utilizing the training set to obtain a Hash code and a Hash function corresponding to each batch of data, and storing the Hash code and the Hash function into a retrieval library; generating a hash code of the to-be-queried sample according to the sample external expansion mapping; updating the hash code of the original sample data in the retrieval library based on the new sample data in the data stream; and comparing the hash code of the to-be-queried sample with the updated hash code in the retrieval library, sorting according to the Hamming distance from small to large, and returning a retrieval result. According to the method, the Hash representation is generated for the new data on the premise of not retraining the original data, and meanwhile, the retrieval precision is greatly improved by mining the similarity relationship between the new data and the old data and utilizing the label information of the new data.

Description

technical field [0001] The invention relates to the technical field of cross-modal retrieval in online scenarios, in particular to an online cross-modal retrieval method and system based on similarity re-learning. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] With the rapid development of Internet technology, a large amount of multimedia data has shown a blowout growth. At the same time, large-scale data retrieval has attracted more and more attention. [0004] In recent years, the approximate neighbor retrieval technology represented by hashing has been deeply researched and made rapid progress. The traditional hash learning method, under the premise of maintaining the similarity relationship of the original sample space, compresses the sample into a short binary code, so as to use a simple XOR operation to calculate the Hamming distan...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/903G06F16/9038G06N20/00
CPCG06F16/90335G06F16/9038G06N20/00
Inventor 刘兴波康潇聂秀山尹义龙郭杰
Owner SHANDONG JIANZHU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap