An image service system and method based on distributed storage

Abstract

The invention discloses a distributed storage-based image service system. The system comprises a picture reception module, a feature analysis module, a feature library, a picture cache server, an HDFS, a feature matching module, a cache management module, a picture output module, a feedback processing module, an offline file optimization module and an image optimization module, wherein the cache management module is used for carrying out distributed caching on the picture searching result so that the picture searching result can be directly used next time; the picture output module is used for outputting the picture searching result; the feedback processing module is used for carrying out similar picture merging according to the picture searching result and restoring the pictures into the HDFS so as to optimize the picture storage and improve the picture similarity in the same large files; the offline file optimization module is used for carrying out offline partition de-weighting and integrating processing on the large files in the HDFS so as to obtain new large files and indexes; and the image optimization module is used for carrying out image analysis and processing on the pictures stored in the HDFS so as to optimize the image storage. The invention furthermore discloses a distributed storage-based image service method which is realized through the distributed storage-based image service system.

Description

Technical field [0001] The invention relates to the field of cloud computing and big data, in particular to an image service system and method based on distributed storage. Background technique [0002] HDFS has poor read and write performance for small files. Current ordinary pictures are almost all small files. If you need to store a large number of pictures in HDFS, you need to merge small files into large files, build indexes, and then store them in HDFS. When using image retrieval services, the result of image retrieval is often a collection of N pictures. In the worst case, N pictures are required. If each picture is in a different large file, you need to open and read it N large files. Since the collection of N pictures is obtained by feature similarity matching, the features are actually similar. Pictures with similar features may appear together again in future searches. However, there is no optimization for this situation. The pictures stored in HDFS may be duplicate...

AI Technical Summary

Problems solved by technology

However, this situation has not been optimized at present. The pictures stored in HDFS may be duplicated, or the pictures with the closest feature similarity are stored in different large files. When these similar pictures need to be extracted after retrieval, multiple files need to be opened. A large file is read, which reduces the efficiency

Images