Big data-oriented relational database hybrid heterogeneous query model and method

A database and relational technology, applied in the field of information technology processing, can solve the problems of inability to maintain the independent integrity of the database, high cost of database data preprocessing, and achieve the effect of improving preprocessing efficiency and improving response time.

Inactive Publication Date: 2019-11-08
NANJING UNIV OF POSTS & TELECOMM
View PDF3 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The cost of database data preprocessing for large-scale data is too high. At the same time, the data is evenly distributed to each node, which cannot maintain the independent integrity of the original databases.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data-oriented relational database hybrid heterogeneous query model and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further explained below in conjunction with the accompanying drawings.

[0025] The data layer establishes local indexes and global indexes. For the directories established by certain fields in the database table, the specific information in the table can be quickly accessed by using the index. Generally, there are multiple copies of database data, and data copies can be used to improve the memory access performance of tasks and increase the speedup ratio of memory access. On the basis of analyzing and filtering out the most commonly used query fields by users, and then using the characteristic that each piece of data in the system has multiple copies, a global index is established for the data table. Through the index generator of the middleware, generate index leaf nodes, intermediate nodes and root nodes to build index trees to access the global index file to obtain global index records that meet user query conditions. The global index i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a big data-oriented relational database hybrid heterogeneous query model, which is structurally characterized in that the bottommost layer is a central database and is used forstoring data to be queried; the middle layer is a Hadoop distributed file system HDFS and is used for storing metadata and middle results, and meanwhile, a data cache layer and a global index layer are added to store all data dictionary tables, result buffers of the data dictionary tables and global index information, index directories and index buffers of all the data tables; a MapReduce programming model is adopted in the top layer, and parallel processing is provided for data in the HDFS, and the fault tolerance is guaranteed; and the middleware is provided with four functional modules including a database connector, a data loader, an index generator and a query engine. The big data-oriented relational database hybrid heterogeneous query model supports dynamic division of original data, improves the preprocessing efficiency, and prolongs the response time of query requests under large-scale users on the basis of keeping the integrity of the original data.

Description

technical field [0001] The invention relates to a hybrid heterogeneous query model and method for a big data-oriented relational database, belonging to the field of information technology processing. Background technique [0002] In the field of big data analysis, with the rapid growth of data volume, high scalability and high performance are essential features of big data analysis platforms. Parallel databases include advanced technical means and algorithms, such as indexing, data compression, materialized views, results Buffering, I / O sharing, optimized data connection, etc.; however, they are lacking in scalability, and most of them only support limited expansion, that is, the scale of hundreds of nodes. MapReduce was proposed by Googel and is a non-structural Designed for one-time processing of data. MapReduce has high scalability, that is, you can add or remove nodes in the cluster arbitrarily without affecting the execution of existing tasks; but under the same hardwa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/28G06F16/24G06F16/22G06F16/182
CPCG06F16/182G06F16/2228G06F16/24G06F16/284
Inventor 张玉杰王汝传樊卫北李鹏韩志杰季一木贺帅帅
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap