Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Distributed database for big data and constructing method thereof

A construction method and database technology, applied in the big data-oriented database and its construction, distributed field, can solve the problems of poor HBase read and write ability, Redis read and write performance degradation, data loss, etc., to achieve the effect of improving efficiency

Inactive Publication Date: 2018-09-14
CHENGDU UNIV OF INFORMATION TECH
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (1) HBase has poor read and write capabilities; Redis itself has the possibility of data loss in its design
[0005] (2) When all the data cannot be stored in memory in some cases, the read and write performance of Redis will also drop sharply
Redis is used as a K-V-based cache database to give full play to its advantages in read and write performance, so as to overcome the inefficiency of HBase read and write

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed database for big data and constructing method thereof
  • Distributed database for big data and constructing method thereof
  • Distributed database for big data and constructing method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0042] The present invention supports high concurrency performance on the basis of supporting mass data storage. HBase is used as the core of the main data storage to give full play to its almost unlimited expansion capacity, and at the same time overcome the shortcomings of Redis in terms of data storage security; secondly, use Redis as a K-V-based cache database to give full play to its read and write performance In order to overcome the low efficiency of HBase read and write. Therefore, the combination of the two can not only realize massive data storage, but also take into account the high concurrency, high real-time p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of information retrieval and database structures thereof, and discloses a distributed database for big data and a constructing method thereof. A file storage layer is a distributed file system HDFS specifically and used for saving HBase data in the system and data needed by Hadoop. A database main body layer is divided into an HBase database core layerand a Redis high-speed buffering layer. An assisting tool layer is divided into an Sqoop data import export tool and a Tephra affair assisting processing framework. When the HBase and the Redis are selected to be combined and used, the saving of a large amount of data can be achieved while high concurrence and high timeliness of the system can be taken into account. For the ensuring of the consistency of data in the database, three types of replications of snapshot replication, affair replication and combining replication are mainly adopted.

Description

technical field [0001] The invention belongs to the technical field of information retrieval and its database structure, and in particular relates to a distributed, big data-oriented database and its construction method. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: Faced with the problem of growing data, various new technologies are emerging, including cloud storage, distributed storage, columnar storage, and search engines. Under normal circumstances, the above technologies can solve the processing of massive data, but each has its own advantages and disadvantages. Each technology cannot completely solve all the problems of massive data, and often needs to be combined with each other to solve practical problems. Based on the distributed architecture, it is particularly important to support big data, high performance, and strong security at the same time. The full name of HBase is Hadoop Database. It is a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
Inventor 唐聃于瀛周金全高燕袁炜
Owner CHENGDU UNIV OF INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com