A data management method and system for hbase database

A data management system and data management technology, applied in electrical digital data processing, special data processing applications, instruments, etc., can solve the problem that large field data cannot be stored and read well, and achieve high reliability and high throughput The effect of high volume and high concurrency
CN103838830BActive Publication Date: 2017-03-29GUANGDONG ESHORE TECH
4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
GUANGDONG ESHORE TECH
Publication Date
2017-03-29

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a data management method and a system of an HBase database. The management method comprises the steps of writing response data into a request, generating HBase records to be stored in the HBase database; using fields with the content sizes larger preset field threshold values and included in the HBase records to be stored as large fields and extracting contents of the large fields to generate large field data blocks; writing the large field data blocks into container files stored in an HDFS file system; using positions of the large field data blocks in the HDFS file system as references and using the references to take the place of the contents of the large fields in the HBase records to be stored; storing the HBase records to be stored in the HBase. By adopting the data management method and the system of the HBase database, the advantages of an HBase and an HDFS can be well played, the large-field processing defect of the HBase is overcome, the shortcoming of overlarge resource consumption of the HDFS during small file storage is overcome, the business demand is met well, and good effect is obtained in the actual application.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of databases, in particular to a data management method and system of an HBase database. Background technique

[0002] With the surge in business volume, the amount of data and access that the IT operation system needs to process also soars. The traditional database solutions represented by Oracle have been far from satisfying users due to their low efficiency and poor scalability. The storage and query requirements have gradually given way to a new generation of distributed databases represented by HBase.

[0003] HBase-Hadoop Database is a highly reliable, high-performance, column-oriented, and scalable distributed storage system. Using HBase technology, large-scale structured storage clusters can be built on inexpensive PC Servers. HBase can well support records with a size in the kilobyte level, but due to the limitation of its implementation principle, the common records and data files in business (suc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More