Text modal and image modal crossing type data retrieval method

Abstract

The invention discloses a text modal and image modal crossing type data retrieval method. The method comprises the following steps: dividing all text and image modal data into multiple samples, and carrying out co-clustering; carrying out main affinity calculation on all data according to a co-clustering center; taking sense vectors of the text modal data and the image modal data as an output expression of a Logistic regression classifier, centralizing a main affinity non-linear expression, and training the main affinity non-linear expression as an input expression, so as to obtain multiple classification functions; when a user needs to retrieve the text or image modal data sample, respectively calculating main affinities, inputting the main affinities into the classification functions to obtain a semantic layer expression of the text or image modal data sample, and carrying out normalization to generate a final expression; and calculating a retrieval result by virtue of an inner product calculation formula. According to the text modal and image modal crossing type data retrieval method, the modal crossing retrieval can be rapidly and effectively realized, and the modal crossing retrieval time can be remarkably shortened.

Description

technical field [0001] The invention relates to the technical fields of data retrieval such as computer vision, pattern recognition, and multimedia retrieval, and in particular relates to a data retrieval method across text modalities and image modalities. Background technique [0002] Currently in the era of mobile Internet, the amount of data is increasing day by day, especially the vast majority of data carry multimodal information. Among them, taking a web page as an example, a web page file includes both text information and image information, how to rationally use multi-modal information design to realize a more humanized search engine has attracted people's attention. However, it is worth noting that the two modalities, text modal and image modal, are not symmetrical at the level of feature expression. In addition, the length and distinguishing ability of these two modal features are quite different, which is Cross-modal retrieval poses great challenges. At present,...

AI Technical Summary

Problems solved by technology

On the one hand, these constraints increase the difficulty of the optimization solution, on the other hand, the optimized solution after scaling cannot guarantee the optimality of the solution

Among them, taking the low-rank constraint as an example, the multi-multiplier alternating direction method ADMM iterative optimization algorithm is usually used to deal with the low-rank constraint. In this way, the coexistence of the F norm and the nuclear norm will inevitably occur in each iteration. In this case, eigenvalue decomposition is used to optimize the solution. However, with the increase of samples, the time and space complexity of matrix eigenvalue decomposition is too large, which is not convenient for practical application.

Images