Small file storing method and system

A technology of small files and files, which is applied in the field of computer distributed storage, can solve the problems of not considering the relationship between files, and the reading efficiency of small files without giving a merge strategy, so as to achieve the effect of improving the reading efficiency

Inactive Publication Date: 2016-08-10
QINGDAO TECHNOLOGICAL UNIVERSITY
View PDF5 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the current merging of small files generally does not consider the relationship between files. File merging is only a...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Small file storing method and system
  • Small file storing method and system
  • Small file storing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0053] Embodiment 1 of the present invention discloses a small file storage method, refer to figure 1 , the small file storage method may include the following steps:

[0054] S101: Obtain semantic description information of multiple small files to be stored.

[0055] Most of the small files stored in the cloud platform are related to each other and are logically related to each other. In view of this connection, this application proposes a semantic-based file merging strategy to merge small files that are more closely related. , and at the same time control the size of the large file formed after merging to ensure that the large file obtained after merging does not exceed the default storage object size of the distributed framework, so as to avoid cross-block (block refers to large file) storage of small files.

[0056] refer to figure 2 The typical application scenario of distributed file storage shown in this embodiment is specifically based on figure 2The method of th...

Embodiment 2

[0097] This embodiment continues to supplement the scheme of embodiment one, refer to image 3 , in this embodiment, the following steps may also be included before the step S104:

[0098] S105: Using the keywords of each of the small files, establish an inverted index for each of the small files; determine the mapping relationship between each of the small files and the corresponding large file and each The location information of the small file in the corresponding large file.

[0099] File retrieval is indispensable for any platform. While storing files, it is necessary to provide support for the file retrieval function at the same time, that is to say, it is necessary to be able to locate the desired file according to the search terms entered by the user. Based on this, in order to To provide support for file retrieval, the present embodiment utilizes the keywords of each small file to set up an inverted index for the small file, thereby forming the mapping of keywords→sm...

Embodiment 3

[0113] On the basis of embodiment two scheme, refer to Figure 4 , in this embodiment, the small file storage method may also include the following steps:

[0114] S106: Read the required small files based on the inverted index, the mapping relationship between the small files and the corresponding large files, and the position information of each of the small files in the corresponding large files.

[0115] On the basis of the above embodiments, this embodiment provides a solution for reading small files. The reading of small files can be divided into two steps: file retrieval and file download. Specifically, the file retrieval can be performed after receiving the search words input by the user. , use Lucene to search the created inverted index, so as to obtain a list of results that match the search terms, the list includes one or more small file IDs, and then continue to use the small file IDs obtained after retrieval to query the database to obtain small files The merged ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a small file storing method and system. According to the invention, association relations among all small files are determined based on semantic description information of each to-be-stored small file. On this basis, merging and storing of each to-be-stored small file are achieved based on the determined association relations among all small files. Thus, the invention provides a small file merging and storing strategy based on a semantic association angle and by applying the small file storing method and system, closely associated small files can be merged, thereby effectively increasing reading efficiency of the small files.

Description

technical field [0001] The invention belongs to the technical field of computer distributed storage, and in particular relates to a small file storage method and system. Background technique [0002] With the rapid development of the Internet, cloud storage has been widely used in the storage of massive Internet data. Cloud storage integrates a large number of different types of storage devices in the network to form a system that provides external storage and business access. Currently, it can provide There are many distributed systems for cloud storage, such as HDFS (Hadoop Distributed File System, distributed file system), Google's GFS (Google File System, Google File System), etc. [0003] In the current Internet environment, small files account for a large proportion. The research on cloud storage of small files is basically based on the strategy of file merging. By merging a large number of small files, the number of files on the platform is reduced, and the distribute...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/134G06F16/182G06F16/285
Inventor 王金龙段良涛
Owner QINGDAO TECHNOLOGICAL UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap