Storage system data management method and device, computing device and storage medium

A storage system and data management technology, applied in the Internet field, can solve the problems of difficulty in ensuring data reliability, high data storage cost, low data reliability, etc., to achieve dynamic migration and management, reduce CPU consumption, and improve data reliability. sexual effect

Inactive Publication Date: 2017-11-24
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since HDFS needs to store 3 copies, its data storage cost is high
In addition, when a hard disk in the storage system fails, the data in the failed hard disk is easily

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage system data management method and device, computing device and storage medium
  • Storage system data management method and device, computing device and storage medium
  • Storage system data management method and device, computing device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0031] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0032] In the present invention, the storage system operates based on an EC (Erasure Coding) data storage mechanism, and using this storage mechanism can effectively reduce the data storage cost. Among them, erasure coding is a data protection coding method. It first divides the original data into k blocks of data, and then obtains m blocks of redundant data based on k blocks of data redundancy calculation, a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a storage system data management method and device, a computing device and a computer storage medium. A storage system runs based on an erasure code data storage mechanism and integrates an extended instruction set. The storage system data management method includes the steps: acquiring hard disk running parameters corresponding to a hard disk in the storage system; analyzing the hard disk running parameters to obtain hard disk failure prediction results; transferring data of the hard disk with predicted failure according to the hard disk failure prediction results; computing the access frequency of the data stored in the storage system; determining data types according to the access frequency of the data, and transferring the data into storage areas corresponding to the data types. According to the technical scheme, data storage cost is reduced, CPU (central processing unit) consumption is effectively reduced, loss of the data in the hard disk with predicted failure is avoided, and data reliability is improved.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a data management method, device, computing device and computer storage medium of a storage system. Background technique [0002] With the continuous development of Internet technology, data storage technology has also developed rapidly, and many storage systems based on different methods have emerged, for example, HDFS (Hadoop Distributed File System, Hadoop Distributed File System) and GFS (Google File System, Google File System) System) and so on. Taking HDFS as an example, in most cases, the replication factor of HDFS is 3. Specifically, one copy is stored on the local rack node, one copy is stored on another node of the same rack, and the last copy is placed on the on nodes in different racks. Since HDFS needs to store three copies, its data storage cost is high. In addition, when a hard disk in the storage system fails, data in the failed hard disk is easily lost, m...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F11/10G06F11/30G06F3/06
CPCG06F3/0614G06F3/0647G06F3/0683G06F11/10G06F11/3037
Inventor 王锋庄泽超陈毅白泉
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap