Dimension modeling method, dimensional modeling apparatus, device and storage medium

Abstract

The invention discloses a dimension modeling method, a dimension modeling apparatus, a device and a storage medium. The method comprises the following steps: according to service needs, constructing alogic model of a fact table and a dimension table; according to the logic model, constructing a physical model of the fact table and the dimension table, wherein specified dimension data is denormalized to the fact table through a set composite field when the physical model of the fact table is constructed. According to the method, the apparatus, the device and the storage medium provided by theinvention, the dimension data is denormalized to the fact table, which is convenient for directly inquiring the fact table without performing associated inquiry while performing data inquiry. Moreover, when the dimension data is denormalized to the fact table, storage and recording are executed according to the set composite field, thus, field expansion caused after redundancy is avoided.

Description

technical field [0001] The present invention relates to the field of data modeling, in particular to a dimensional modeling method, device, equipment and storage medium. Background technique [0002] Data warehouse modeling methods are usually divided into four types: 3NF (Third Normal Form, third normal form), dimensional modeling, data-vault, and anchor. In the traditional RDBMS (Relational Database Management System, relational database management system) scenario, these four model design methods can cover most application scenarios, and different modeling methods provide convenience from different angles for data storage and analysis. [0003] However, after the amount of data rises to the tb / pb level, changes in the amount of data and changes in the computing and storage methods based on the Hadoop system have magnified the shortcomings of these modeling methods that require too many joins (associated queries). Specifically, take the join of table A and table B through...

AI Technical Summary

Problems solved by technology

[0003] However, after the amount of data rises to the tb / pb level, changes in the amount of data and changes in the computing and storage methods based on the Hadoop system magnify the shortcomings of these modeling methods that require too many joins (associated queries)

Specifically, take the join of table A and table B through the a1 field of table A and the b1 field of table B on hive as an example. This query will be converted into a hadoop map-reduce job. At the same time, every time one more table is joined For example, if the a2 field of table A is joined with the c1 field of table C, there will be one more map-reduce job, and the increase in the number of map-reduce jobs will directly lead to a decrease in query efficiency

Images