Hybrid storage system based on multidimensional data similarity and data management method

Abstract

The invention relates to a hybrid storage system based on multidimensional data similarity and a data management method thereof. The hybrid storage system is composed of an upper layer memory unit (3), a middle layer cache unit (2), a lower layer storage unit (1) and a control unit (4). The upper layer memory unit (3) is used for storing system data and data index tables which are frequently accessed in all storage layers. The middle layer cache unit (2) is used for storing part of data which are frequently accessed, and the lower layer storage unit (1) is used for storing all data sets. The control unit (4) is used for being connected with the upper layer memory unit (3), the middle layer cache unit (2) and the lower layer storage unit (1) through I / O ports to constitute the hybrid storage system of a multi-layer storage structure. The invention further discloses the data management method of the hybrid storage system. According to the hybrid storage system based on multidimensional data similarity and the data management method thereof, the accessed speed can be improved, corresponding data can be quickly searched and sought in the accessing process, data access delay is reduced, the data processing efficiency can further be improved, and a request throughput capacity is improved.

Description

[0001] Field [0002] The invention relates to a hybrid storage system and a data management method based on multidimensional data similarity, and a content caching scheme of the hybrid storage system oriented to multi-user real-time data query. The invention belongs to the technical field of distributed computing and computer storage. Background technique [0003] In the prior art, the storage structure of a computer system generally includes two storage structures of a memory unit and a hard disk storage unit. The storage structure of some computer systems is a combined storage layer structure of memory storage and hard disk storage unit storage, for example: ordinary HDD storage hard disk As a first-level data storage layer, it forms a two-tier data storage architecture with the memory cache. Although this storage structure can use the characteristics of the workflow to store the data requested by the data-reading-intensive workflow in the storage hard disk, it can effectiv...

AI Technical Summary

Problems solved by technology

[0006] Solution 1: Independent management of data at different levels has the following disadvantages: it does not consider the time correlation and user correlation of data closely related to applications, and is not practical when processing smart grid data queries

The disadvantage is that this scheme can only effectively identify the key data blocks, and the data management scheme in the hybrid storage system is given, but in practical applications, determining the key data blocks through the I / O access history involves Complex calculations are not suitable for real-time query processing services with high latency requirements

The disadvantage is that this scheme provides a scheme for SSD data cache management under the condition of given data block access mode characteristics, and the long-tail distribution property is not suitable for all applications and services, and the known data access mode is used to The design of the cache management scheme has great limitations in the actual system implementation, because the data access mode has time instability

The disadvantage is that this solution only proposes data storage management between different devices in the same data storage layer, and does not provide a data management solution between different storage layers, and does not involve the design of a data cache solution. At the same time, the solution It also simply classifies workflow types for reading and writing, and does not optimize data management according to the characteristics of workflows for data access or writing. Storage performance depends on application types and characteristics

Images