Large-scale image data similarity searching method based on EMD (earth mover's distance)

Abstract

The invention discloses a large-scale image data similarity searching method based on an EMD (earth mover's distance). The method comprises the following steps that an image data mapping function f used for mapping to a one-dimension real number key value space Omega(phi) is designed; an operation MR1 is started, and a load of each key value in the Omega(phi) is estimated; the operation MR2 is started, the cutting is carried out on the Omega(phi) through a Map task on the basis of the estimated key value load, and data corresponding to the cutting region are sent to a Reduce task in a segmented way; image data received by each Reduce task is mapped to the key values in the Omega(phi) on the basis of f, and an index structure oriented to the EMD is built on the basis of the key values; the similarity searching based on the EMD is executed on the basis of the index structure; execution results of each Reduce task based on EMD similarity searching in the MR2 are subjected to union set taking and output. The large-scale image data similarity searching method has the advantages that the network transmission data quantity is lower, the calculation load distribution is more balanced, the similarity searching efficiency is higher, and the big data set analysis and processing expandability is better.

Description

Technical field [0001] The invention relates to a similarity search technology for computer image data, in particular to a large-scale image data similarity search method based on EMD distance. Background technique [0002] With the popularization of digital devices such as portable computers, smart phones, and digital cameras, multimedia data represented by images has increased and exploded. All this indicates that the era of image big data has arrived. At present, academia, industry and even government agencies have begun to pay close attention to the analysis and processing of image big data. [0003] To find important value image information from large-scale image data sets, the traditional text-based image retrieval method (Text-Based Image Retrieval, referred to as TBIR) obviously cannot meet the demand. Because TBIR technology relies on manual annotation of image content, when the number of images increases sharply, it brings two serious problems: first, the workload of ma...

AI Technical Summary

Problems solved by technology

[0003] To find valuable image information from large-scale image datasets, the traditional text-based image retrieval method (Text-Based Image Retrieval, TBIR for short) obviously cannot meet the demand

Because TBIR technology relies on manual annotation of image content, when the number of images increases sharply, it brings two serious problems: first, the workload of manual annotation is too large, and the cost of annotation is too high; second, the subjective If it is too strong, it will directly affect the reliability of image retrieval results.

In view of the high computational complexity of the EMD distance, it is obviously biased to estim

Images