Method and system concurrently accessing a large amount of small documents in cluster storage

A technology for cluster storage and small files, applied in transmission systems, electrical components, special data processing applications, etc., can solve problems such as slow orders, lagging picture display, affecting normal use of users, etc., to improve response time and speed, and improve performance Effect

Active Publication Date: 2010-10-06
SUZHOU LANGCHAO INTELLIGENT TECH CO LTD
View PDF7 Cites 57 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] With the continuous development of network applications and e-commerce in finance, petroleum, telecommunications, production, etc., the number of visits to various sites is increasing, and the scale of data is also continuously expanding. File access, especially a large numbe

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system concurrently accessing a large amount of small documents in cluster storage
  • Method and system concurrently accessing a large amount of small documents in cluster storage
  • Method and system concurrently accessing a large amount of small documents in cluster storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0044] see figure 1 , which shows a system for parallel access to a large number of small files in cluster storage according to an embodiment of the present invention, including application host clusters, metadata server clusters, data server clusters, back-end storage and high-speed interconnection devices, wherein:

[0045] The application host cluster is used to provide a file system interface for the client. When a small file is detected to be written, the small file is cached, and multiple cached small files are merged into a temporary file. The metadata and data objects are pushed to each metadata server node and each data server node in the cluster through the high-speed interconnection device.

[0046] The metadata server cluster is used to manage the backend storage of each metadata server node in the cluster.

[0047] The data server cluste...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and system concurrently accessing a large amount of small documents in cluster storage. The method comprises the following steps: the written small documents are buffered, a plurality of buffered small documents are merged into one temporary document, the metadata and the data object of the temporary document are stored into the backend storage of a metadata server node and a data server node. The method can effectively improve the response time and speed of the cluster document system service and the whole data read and write time and transaction capacity per unit time of the data.

Description

technical field [0001] The invention relates to a method and system for parallel accessing a large number of small files in cluster storage, which is usually used in large-scale mass data storage and read-write applications. Background technique [0002] With the continuous development of network applications and e-commerce in finance, petroleum, telecommunications, production, etc., the number of visits to various sites is increasing, and the scale of data is also continuously expanding. File access, especially a large number of small files in network applications The performance problems of parallel access to files are becoming more and more prominent, such as processing user orders too slowly, and a series of problems such as image display lag, which will seriously affect the normal use of users. [0003] Common enterprise data applications are built on disk arrays. Therefore, how to provide a method for organizing and accessing these data so that they can be applied in t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F17/30
Inventor 陈云松
Owner SUZHOU LANGCHAO INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap