Method for hierarchically caching read-write data in storage cluster

A technology for storing clusters and reading and writing data, which is applied in the field of cloud computing storage. It can solve problems such as low reading and writing speeds and affect the performance of cloud management operations, and achieve the effect of reducing delay and improving cloud management performance experience

Active Publication Date: 2021-04-20
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF27 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In a cloud computing environment, storage clusters usually provide all storage services for computing clusters and control nodes. The data in these storage clusters is sometimes very disordered and massive. When a management or tenant-side program initiates a data request (such as creating Specify the virtual machine of the image specification or call the monitoring data of the specified node or request the file stream service), each time it will interact with the back-end main storage node, the back-end main storage node usually uses SATA Cluster, in the case of high concurrent I / O , the read and write speeds of these SATA Clusters are not high, which will affect the performance of the entire cloud management operation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for hierarchically caching read-write data in storage cluster
  • Method for hierarchically caching read-write data in storage cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0035] Reference attached figure 1 , this embodiment proposes a method for caching read and write data hierarchically in a storage cluster, which is implemented based on a backend storage cluster, a first-level cache L1, a second-level cache L2, an api gateway, a log file system, and an application.

[0036] The back-end storage cluster uses SATA Cluster as the storage medium, and manages the original data through the ceph cluster. The original data is not limited to include the data initialized during environment deployment, metadata written through the storage medium, production data and monitoring data generated during operation.

[0037] The first-level cache L1 uses SSD Cluster to store dynamically changing hot data, and divides hot data into different pools according to process pids and business attributes. Among them, the pool is stored in units of objects, and the hotspot data is not limited to including data related to virtual machine images and monitoring alarm data ...

Embodiment 2

[0052] Reference attached figure 1 , this embodiment proposes a method for caching read and write data hierarchically in a storage cluster, which is implemented based on a backend storage cluster, a first-level cache L1, a second-level cache L2, an api gateway, a log file system, and an application.

[0053] The back-end storage cluster uses SATA Cluster as the storage medium, and manages the original data through the ceph cluster. The original data is not limited to include the data initialized during environment deployment, metadata written through the storage medium, production data and monitoring data generated during operation.

[0054] The first-level cache L1 uses SSD Cluster to store dynamically changing hot data, and divides hot data into different pools according to process pids and business attributes. Among them, the pool is stored in units of objects, and the hotspot data is not limited to including data related to virtual machine images and monitoring alarm data ...

Embodiment 3

[0069] Based on the structure of Embodiment 1 or Embodiment 2, a GlusterFS cluster can also be used instead of a ceph cluster.

[0070] To sum up, it can be seen that by adopting a method for caching read and write data hierarchically in a storage cluster according to the present invention, the delay can be greatly reduced by adding two levels of caching, and the cloud management performance experience can be improved.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for hierarchically caching read-write data in a storage cluster, relates to the technical field of cloud computing, and is implemented on the basis of a back-end storage cluster, a first-stage cache, a second-stage cache, an api gateway, a log file system and an application program. The back-end storage cluster manages the original data while the first-stage cache stores the hotspot data. Hot spot data is divided into different pools, the second-stage cache extracts the poll data into segments according to indexes and stores the segments, and the api gateway processes the requests in a unified mode. When the application program initiates a read request, the api gateway processes the request and publishes the request to the second-stage cache, the second-stage cache searches related segments and locates the related segments to the pool, or, if the related segments are not found, a segment missing request is further initiated to the first-stage cache, and after related information still cannot be found, the second-stage cache continues to search in a back-end storage cluster; when the application program initiates a write request, the api gateway processes the request and writes the request into a log file system, and Flush enters a back-end storage cluster after a transaction is completed. According to the invention, time delay can be greatly reduced.

Description

technical field [0001] The invention relates to the technical field of cloud computing storage, in particular to a method for hierarchically caching read and write data in a storage cluster. Background technique [0002] In a cloud computing environment, storage clusters usually provide all storage services for computing clusters and control nodes. The data in these storage clusters is sometimes very disordered and massive. When a management or tenant-side program initiates a data request (such as creating Specify the virtual machine of the image specification or call the monitoring data of the specified node or request the file stream service), each time it will interact with the back-end main storage node, the back-end main storage node usually uses SATA Cluster, in the case of high concurrent I / O , these SATA Clusters are not high in read and write speeds, which will affect the performance of the entire cloud management operation. Contents of the invention [0003] Aim...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F12/0897
Inventor 齐光鹏窦洋张明哲
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap