HBase (Hadoop database) data usability and durability method based on remote log backup

A usability and persistence technology, which is applied in the direction of data error detection, special data processing application, electrical digital data processing, etc., can solve the problems of reducing system write operation time performance and user experience, and achieve reduction The effect of persistence frequency, guaranteed availability, and improved data writing speed

Active Publication Date: 2014-06-18
上海艾讯云计算有限公司
View PDF4 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although this availability and persistence solution can guarantee data persistence and basic availability, it introduces a large number of persistence processes in the data processing process. These persistence processes are disk write operations, which will greatly reduce the time for system write operations Performance and User Experience

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • HBase (Hadoop database) data usability and durability method based on remote log backup
  • HBase (Hadoop database) data usability and durability method based on remote log backup
  • HBase (Hadoop database) data usability and durability method based on remote log backup

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0029] Such as figure 1 A method of HBase data availability and persistence based on remote log backup is shown. When HBase data nodes are written, the log records are first encapsulated through the distributed system, and the logs are backed up to the pre-designated remote nodes through the network to ensure that the data The availability and durability of HBase, and based on this, a large amount of user data and log records are temporarily stored in the memory to reduce the data persistence process in the data processing process; when the data nodes of HBase are idle, the Data is persisted to the file system to reduce the pressure on memory storage, reduce the frequency of persistence processes during write operations, and improve the time performance of data writing. The data node processing of described HBase is divided into two phases; ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an HBase (hadoop database) data usability and durability method based on remote log backup. The method is characterized in that when a data node of the HBase is written, the log record is first encapsulated through a distributive system, the log is backed up to a preliminarily-designated remote node through a network, the usability and durability of the data can be guaranteed, on the basis of the data, a great amount of user data and log record are temporarily stored in a memory, and the data persisting process in the data processing process is reduced; when the data node of the HBase is idle, the data in the memory is persisted to a file system, the storage pressure of the memory can be alleviated, the frequency of the persistence process in the writing process can be reduced, and the data writing time performance can be improved. By adopting the method, the usability and durability of the data can be guaranteed, the data writing speed can be greatly increased, and the system performance is improved.

Description

technical field [0001] The invention relates to a solution for the availability and persistence of non-relational database data based on remote log backup, in particular to a method for the availability and persistence of HBase data based on remote log backup. Background technique [0002] Non-relational database refers to a new type of database that is different from traditional relational databases. It shows good performance in terms of massive data storage and high concurrent access support. HBase, also known as Hadoop Database, is a non-relational database based on column storage. HBase is a sub-project of Apache Hadoop. It is in the structured storage layer in the Hadoop architecture: the lower layer needs = relying on the distributed file system HDFS; it provides high-performance, high reliability, high scalability, and column-based storage for the upper-layer MapReduce computing module. Distributed storage system. HBase can store structured data, as well as semi-str...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F11/14G06F11/34
Inventor 杨峰陈宁昕孙晓燕周学海唐长城谢飞赵伟李政
Owner 上海艾讯云计算有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap