Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Stored data warehouse row and column storage transformation implementation method for database all-in-one machine

A technology of in-memory data and implementation method, applied in the field of in-memory data warehouse storage conversion, can solve problems such as dictionary table update, complex data conversion, etc., and achieve the effects of optimizing transaction processing, optimizing data storage efficiency, and improving data update performance

Active Publication Date: 2017-06-20
RENMIN UNIVERSITY OF CHINA
View PDF3 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Current representative technologies, such as SAP HANA, use L1 row storage engine, L2 non-compressed column storage engine, and main storage column engine with data compression to support real-time OLAP analysis and processing. Need to solve issues such as column compression and dictionary table update, and need to deal with complex data conversion issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stored data warehouse row and column storage transformation implementation method for database all-in-one machine
  • Stored data warehouse row and column storage transformation implementation method for database all-in-one machine
  • Stored data warehouse row and column storage transformation implementation method for database all-in-one machine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0056] Example: such as Figure 4 As shown, the row-column conversion and multi-level cache mechanism proposed by the present invention are used to support the real-time OLAP query processing of the in-memory data warehouse platform, and the real-time OLAP query processing process is divided into four processing stages.

[0057] The first stage is the dimension table processing stage of the OLAP query. The OLAP query command is decomposed into subqueries oriented to the relevant dimension tables, and the selection and projection operations are performed on the relevant dimension tables, and the dimension table operation results are converted into dimension vectors of the column storage structure. , dynamically map the dimension table into columns, and complete the subsequent OLAP query processing tasks with the fact table records.

[0058] The high-performance server node of the database all-in-one machine is the main node of the memory data warehouse. The row storage database...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a stored data warehouse row and column storage transformation implementation method for a database all-in-one machine. The method comprises the steps of constructing a stored data warehouse all-in-one machine storage model, wherein a dimension table concentrated storage and fact table distributed storage strategy is adopted on a database all-in-one machine framework; arranging a storage engine; arranging a row and column storage model conversion strategy; breaking up real-time OLAP search processing tasks into distributed OLAP search processing tasks on main node fact table row caching, shadow data column caching and storage server node fact table main sheets on the database all-in-one machine, and achieving OLAP analysis treatment based on real-time data. Row storage records of a transaction processing engine are efficiently converted into column storage records of an analysis engine, and the method supports OLAP analysis treatment on real-time data. The method is applicable to row and column storage structure conversion application scenarios in the updating operation for the stored data warehouse all-in-one machine.

Description

technical field [0001] The invention relates to a memory data warehouse storage conversion method, in particular to a database integrated machine-oriented memory data warehouse row and column storage conversion realization method. Background technique [0002] With the development of hardware technologies such as large memory and multi-core processors and memory database technology, the performance of memory OLAP processing has been continuously improved, and real-time analysis and processing of big data memory has become the mainstream technology. Traditional transaction processing engines usually use a row storage model to optimize update operation performance, while analysis processing usually uses a column storage engine to optimize data access and processing performance. The current mainstream databases begin to support the integration of transaction processing engines and analysis processing engines, such as Oracle Database in-memory, SQL server 2016, etc., which suppo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
Inventor 张延松王珊杜小勇
Owner RENMIN UNIVERSITY OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com