Multi-modal multi-class Boosting frame construction method and device for cross-modal retrieval

Abstract

The invention relates to multi-modal multi-class Boosting frame construction method and device for cross-modal retrieval. The multi-modal multi-class Boosting frame construction method for the cross-modal retrieval comprises the steps: constructing a target risk function including intra-mode loss of each mode and inter-mode loss among all modes; according to gradient descent strategy, sequentially updating a predictor of each mode in the risk function, but fixing predictors of other modes; after the predictors of all modes are updated, defining primary cyclic iteration, performing T times of cyclic iteration, and then learning an optimal predictor of each mode, which enables a target function to be minimum; and transferring a fitted edge generated by the optimal predictor of each mode into a common semantic space by utilizing a Sigmoid function, thereby implementing the cross-modal retrieval. The multi-modal multi-class Boosting frame construction method for the cross-modal retrieval considers intra-mode semantic relevance, can reinforce intra-mode semantic information with poor quality to a certain extent, and has relatively greater performance in a cross-modal retrieval task.

Description

technical field [0001] The invention belongs to the field of information retrieval, and in particular relates to a method and device for constructing a multi-modal and multi-class Boosting framework for cross-modal retrieval. Background technique [0002] The core idea of ​​the Boosting classification method is to combine multiple weak classifiers into a strong classifier. This method has been widely studied in computer vision and pattern recognition and other application fields, and has achieved good results. Nevertheless, traditional boosting methods only learn classification rules from a single modality dataset and cannot directly deal with multimodal datasets. In general, traditional boosting methods can be applied to cross-modal retrieval by individually mapping each modality's dataset to the semantic space. However, this scheme does not consider the crucial inter-modal information, which degrades the retrieval performance to some extent. [0003] At present, how to r...

AI Technical Summary

Problems solved by technology

[0012] The present invention provides a method and device for constructing a multi-modal multi-class Boosting framework for cross-modal retrieval, aiming to solve the problem of low retrieval performance when the traditional Boosting method is applied to cross-modal retrieval

Images