Mobile visual search framework based on CRBM and Fisher network

Abstract

The invention provides a mobile visual search framework based on a CRBM and Fisher network and relates to the image retrieval of a mobile terminal. The method comprises the following steps of 1) constructing and training a continuous restricted Boltzmann machine network; 2) constructing and training a Fisher layer network. According to the invention, the sub-space feature information of the localfeature nature of non-Gaussian distribution is found by adopting the nonlinear dimensionality reduction algorithm CRBM in the aggregation global compact binary feature algorithm. Meanwhile, more efficient global features are obtained through adopting a fisher-based network structure aggregation fisher vector. A compact self-adaptive feature is obtained by adopting the scalar quantization algorithmand the bit self-adaptation algorithm. Therefore, the length of the transmitted image feature information can be selected according to different self-adaptive selections of the network bandwidth of amobile terminal. During the retrieval stage, global features are roughly matched to obtain a candidate set, and the accurate matching of the geometric consistency test is conducted by adopting localfeatures. Therefore, the mobile visual search framework can be adapted to large-scale image retrieval tasks.

Description

technical field [0001] The invention relates to image retrieval on mobile terminals, in particular to a mobile visual search framework based on CRBM and Fisher network. Background technique [0002] The International Telecommunication Union of the United Nations (International Telecommunication Union) conducts statistics on the number of users accessing mobile broadband around the world every year. According to the statistical results of 2015 and 2016, the number of users accessing the Internet through mobile devices in the world increased from 3.2 billion in 2015 to 3.6 billion in 2016, with a net increase of 400 million. By 2016, about 47% of the world's population used mobile devices to access the Internet. The ITU also pointed out that the rapid increase in the number of users accessing the network through mobile devices is due to the remarkable achievements in the construction of network broadband infrastructure around the world. According to statistics, 84% of the cou...

AI Technical Summary

Problems solved by technology

[0007] Second, the current bandwidth of mobile devices is unstable and the speed is slow

[0008] Third, the hardware computing power and storage capacity of mobile devices are limited, and the real-time performance of visual feature extraction is a challenge. Although the hardware capacity and storage capacity of mobile devices continue to increase, for more complex visual feature extraction and multi-tasking, etc.

However, in this framework, the linear reduction algorithm PCA (Principal Component Analysis Algorithm) is used to reduce the dimensionality of the local feature SIFT of non-Gaussian statistics, which will greatly damage the local feature information; the traditional EM algorithm is used in the global feature Fisher Vector Estimate the parameters of the mixed Gaussian model containing K Gaussian functions, but the setting of the initial parameters has a great influence on the degree of convergence of the EM algorithm, and it is easy to converge to a local optimum

Images