Unlock instant, AI-driven research and patent intelligence for your innovation.

Data table generation method and device

A data table and data recording technology, which is applied in the field of data warehouse construction, can solve problems such as low data processing efficiency, large storage space requirements, and inability to record the historical state of data changes.

Active Publication Date: 2021-05-18
内蒙古蒙商消费金融股份有限公司
View PDF9 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In view of the above problems, an embodiment of the present invention is proposed to provide a data table generation method, device, electronic equipment, and computer storage medium to solve or partially solve the historical state that cannot record data changes during data processing in the prior art, and store Large space requirements, increased computing resource requirements, and low data processing efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data table generation method and device
  • Data table generation method and device
  • Data table generation method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0066] In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0067] The data warehouse can include full data tables, incremental data tables, snapshot tables, and zipper data tables. Among them, the full-scale table records all the latest state data in the data warehouse every day. For the full-scale table, the data reported each time is all data, including the changed data and the unchanged data; the incremental data table records New data is added, and the incremental data is the new data after the last export. For the incremental data table, the data that has changed is reported each time. If there is no change, it will not be reported; the snapshot table can be partitioned by time period, and the record deadline The full amount of data on the date of the data, for the snapshot table, th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a data table generation method and device, and the method comprises the steps of obtaining a basic zipper data table and an incremental data table corresponding to a target business data table, comparing the stored data records of the basic zipper data table and the incremental data table, and if a second designated field corresponding to a first designated field in the basic zipper data table exists in the incremental data table, judging that corresponding incremental data exists in the incremental data table; establishing a data mapping relation between zipper data records in the basic zipper data table and incremental data records in the incremental data table according to zipper time information of the basic zipper data table and incremental updating time in the incremental data table; updating the basic zipper data table according to the data mapping relation, forming the full-amount zipper data table for the business data table, and building the complex zipper data table, so that the data storage space is reduced, the data can be recorded from multiple dimensions, and the data processing efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of data warehouse construction, in particular to a method for generating a data table and a device for generating a data table. Background technique [0002] In the data warehouse construction architecture, the data source system will have new or updated data every day. If every record change is stored in the data warehouse, it will easily bring a large amount of redundant data, resulting in storage The space increases dramatically. In addition, during data processing such as ETL (Extract-Transform-Load, Extract-Transform-Load), due to the increase in the amount of data to be processed, the data resources to be calculated will also increase, thus affecting the efficiency of data processing . However, in the field of data warehouse construction technology, the way of data warehouse architecture construction based on Hive is still the mainstream, because the bottom layer of Hive is based on HDFS (Hadoop Dist...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/23G06F16/28
CPCG06F16/2282G06F16/23G06F16/283G06F16/284
Inventor 杨春雨卫军付金赛
Owner 内蒙古蒙商消费金融股份有限公司