A method for caching read and write data hierarchically in a storage cluster

A technology for storing clusters and reading and writing data, which is applied in the field of cloud computing storage. It can solve problems such as low reading and writing speeds and affect the performance of cloud management operations, and achieve the effect of reducing delay and improving cloud management performance experience

Active Publication Date: 2022-05-31
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF27 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In a cloud computing environment, storage clusters usually provide all storage services for computing clusters and control nodes. The data in these storage clusters is sometimes very disordered and massive. When a management or tenant-side program initiates a data request (such as creating Specify the virtual machine of the image specification or call the monitoring data of the specified node or request the file stream service), each time it will interact with the back-end main storage node, the back-end main storage node usually uses SATA Cluster, in the case of high concurrent I / O , the read and write speeds of these SATA Clusters are not high, which will affect the performance of the entire cloud management operation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method for caching read and write data hierarchically in a storage cluster
  • A method for caching read and write data hierarchically in a storage cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

The second-level cache L2 adopts a NAND flash memory array, and the pool data of the first-level cache L1 is extracted into segments according to the index

Segment, and then save it in the form of Segment. The extracted segment Segment only stores the index and address of the pool data.

Each segment segment has a dynamically variable length, and the length of each segment segment is based on the flash space of the second level cache L2

Make free scheduling.

[0039] The api gateway, as the traffic entry of all requests, handles requests uniformly and merges the same request data.

[0040] The journaling file system stores data as write requests occur.

In the present embodiment, the first level cache is used as cache, and the size of the data block and the recall probability are determined according to the size of the data block.

The formula depends on performance goals, which include maximizing cache hit rate, minimizing average response time, and minimizing

The amount...

Embodiment 2

Segment, and then save it in the form of Segment. The extracted segment Segment only stores the index and address of the pool data.

Each segment segment has a dynamically variable length, and the length of each segment segment is based on the flash space of the second level cache L2

Make free scheduling. A second background thread is defined in the second level cache L2, and the second background thread is used to periodically clean up the second level

Cache dirty data for L2.

[0056] The api gateway is used as the traffic entry for all requests, and processes requests uniformly and merges the same request data.

[0057] The journaling file system stores data as write requests occur.

In this embodiment, the first-level cache is used as cache, and the size of the data block and the recall probability are determined according to the size of the data block.

The formula depends on performance goals, which include maximizing cache hit rate, minimizing average response time,...

Embodiment 3

[0069] On the basis of the structure of the first embodiment or the second embodiment, the GlusterFS cluster can also be used to replace the ceph cluster.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for caching and reading and writing data hierarchically in a storage cluster, and relates to the technical field of cloud computing. Its realization is based on a back-end storage cluster, a first-level cache, a second-level cache, an API gateway, a log file system, and an application program. The back-end storage cluster manages the original data. The first-level cache stores hot data and divides the hot data into different pools. The second-level cache extracts the pool data into segments according to the index and saves them. The API gateway processes requests uniformly. When the application initiates a read request, the api gateway processes the request and publishes it to the second-level cache. The second-level cache searches for the relevant segment and locates it in the pool. Or, if no relevant segment is found, it further initiates a missing segment request to the first-level cache. If the relevant information still cannot be found, continue to search in the back-end storage cluster; when the application initiates a write request, the api gateway processes the request and writes it to the log file system, and then flushes it into the back-end storage cluster when the transaction is completed. The invention can greatly reduce the time delay.

Description

A method for hierarchical caching of read and write data in a storage cluster technical field The present invention relates to the technical field of cloud computing storage, specifically a kind of hierarchical cache read and write data in a storage cluster Methods. Background technique [0002] In a cloud computing environment, usually a storage cluster will provide all storage services for the computing cluster and control nodes, The data in these storage clusters is sometimes very disordered and massive. When the management or tenant-side program initiates a data request (For example, creating a virtual machine with a specified image specification, or calling the monitoring data of a specified node, or requesting a file stream service), each time They all interact with the back-end main storage nodes. The back-end main storage nodes usually use SATA Cluster. In the case of high concurrent I / O However, the read and write speed of these SATA clusters is not high, wh...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F12/0897
Inventor 齐光鹏窦洋张明哲
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap