Distributed storage method based on KEY-VALUE pair

A KEY-VALUE, distributed storage technology, applied in special data processing applications, instruments, electrical digital data processing, etc., can solve the problems of low performance, reduced performance, low efficiency during writing, etc. The effect of quickly and avoiding IO loss

Inactive Publication Date: 2015-05-27
INSPUR GROUP CO LTD
View PDF4 Cites 30 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The traditional local disk storage method can no longer meet the requirements of large capacity, high reliability, high performance, and horizontal expansion of existing applications. In order to meet these new requirements, network-based distributed storage systems have received extensive attention.
[0003] The data generated by various applications such as social networks, mobile communications, network video and audio, e-commerce, sensor networks, and scientific experiments can often generate tens of millions, billions, or even billions or tens of billions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage method based on KEY-VALUE pair
  • Distributed storage method based on KEY-VALUE pair

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The present invention is described in detail with reference to the accompanying drawings.

[0020] The distributed storage method based on KEY-VALUE key-value pairs includes an operation receiving layer, a cache search layer and an operation processing layer. The operation receiving layer receives the read and write operations sent by the client through the network, Form a transaction, and then issue it uniformly to ensure strong data consistency; the cache lookup layer will respond to the transaction mentioned in the previous section, execute it after parsing, and send it to the system's cache in memory according to the key value The area is queried, and if it is found in the cache, the next operation is performed according to the operation type; the operation processing layer receives the operation of the unprocessed transaction in the cache, depending on the operation type.

[0021] When it is currently a read operation, the operation receiving layer specifies the da...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention particularly relates to a distributed storage method based on a KEY-VALUE pair. The distributed storage method based on the KEY-VALUE pair comprises an operation receiving layer, a cache searching layer and an operation processing layer, wherein after receiving user data, the operation receiving layer of a server completes splicing of the key-value pair through internal data organization; a bottom layer is divided into different data levels for disk landing and is meanwhile responsible for analyzing and reading following data; since the access of the key value pair belongs to lightweight database type operation without metadata, and is appended to write in, time consumption brought by disk landing of random small files can be avoided. According to the distributed storage method based on the KEY-VALUE pair, the bottom layer storage is modified to be a key-value pair mode, so that the whole storage cluster can more quickly and efficiently process the small files, and IO loss caused by frequent addressing along with small file random write-in in a traditional distributed storage or local storage mode is avoided, and the storage efficiency under such scene is greatly improved.

Description

technical field [0001] The invention relates to the technical field of distributed storage, in particular to a distributed storage method based on KEY-VALUE key-value pairs. Background technique [0002] With the rapid development of network applications, the amount of network information data is increasing, and PB-level mass data storage is becoming more and more important. The traditional local disk storage method can no longer meet the requirements of large capacity, high reliability, high performance, and horizontal expansion of existing applications. In order to meet these new requirements, network-based distributed storage systems have received extensive attention. [0003] The data generated by various applications such as social networks, mobile communications, network video and audio, e-commerce, sensor networks, and scientific experiments can often generate tens of millions, billions, or even billions or tens of billions of massive small files. In many scenarios, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 魏盟张在贵
Owner INSPUR GROUP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap