Acceleration method for mass small document IO operation transmission in distribution type document system

A technology of distributed files and small files, applied in transmission systems, electrical components, special data processing applications, etc., can solve problems such as small file transmission bottlenecks, achieve the effect of reducing the number of network interactions and increasing transmission speed

Active Publication Date: 2014-04-09
上海浪潮云计算服务有限公司
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Due to the design principle of the distributed file system, it has inherent defects in solving the problem of massive small file transfer, which makes th

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Acceleration method for mass small document IO operation transmission in distribution type document system
  • Acceleration method for mass small document IO operation transmission in distribution type document system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0013] Refer to the attached below figure 1 A detailed description of the method for increasing the transmission speed of a large number of small files in a distributed file system proposed by the present invention. The method involves the following main steps

[0014] 1. Interception of client IO calls. This step implements the redirection of IO operations on the client operating system by installing special software on the client, distinguishes between small files and large files by acquiring file volumes, and uses the system default IO path for transmission for large files. Small files are intercepted.

[0015] 2. Consolidation of traffic of massive small files. After the client intercepts the small file IO operation, the file name, file length, file content, and file attributes of the small file are read and encoded, and written into the preset network path. At this time, a large number of small file IOs are merged into a single large file IO. For the case where a large numb...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an acceleration method for mass small document IO operation transmission in a distribution type document system. The method comprises the steps of buffering a large amount of small document operation at a sending end, executing flow aggregation, packing small documents into a single large document, transmitting the single large document to a receiving end in one time, analyzing the large document at the receiving end, obtaining the IO operation of each small document, and executing the writing operation of relevant document. The method greatly reduces the network interaction times and improves the transmission speed of mass small document IO operation.

Description

technical field [0001] The invention relates to the field of computer storage, in particular to a method for accelerating transmission of massive small files in cluster storage. Background technique [0002] With the rapid development of network applications, the amount of network information data is increasing, and PB-level mass data storage is becoming more and more important. The traditional file system can no longer meet the requirements of large capacity, high reliability, and high performance of existing applications. In order to meet these new requirements, the cluster NAS system developed based on the distributed file system has received extensive attention. [0003] The existing distributed file system research is mainly divided into separate management of metadata and actual data storage. Through the isolation of metadata and actual data, the dynamic increase and decrease of capacity and the current expansion of performance of distributed file system are realized. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F17/30
Inventor 孟圣智
Owner 上海浪潮云计算服务有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap