Method and equipment for analyzing data

Abstract

The embodiment of the invention discloses a method and equipment for analyzing data. The non-repetitiveness of data storage is realized by storing the data into data subregions according to the life cycle of the data, so the total capacity of accommodating the same number of snapshots is reduced, the data pre-processing time is saved, and the data pre-processing time is controlled in a normal gain combined time range; the acquired data subregions corresponding to a service date are combined so as to acquire the corresponding data snapshot; therefore, the efficiency of taking the snapshots is improved and the complexity of taking the snapshots is reduced.

Description

technical field [0001] The embodiments of the present application relate to the technical field of data storage, and in particular, to a data analysis method and device. Background technique [0002] Slowly Changing Dimensions (SCD) are dimensions that store and manage current and historical data over time in a data warehouse. It is considered and implemented as one of the most critical data extraction, transformation and loading (Extraction Transformation Loading, ETL) tasks in the process of tracking dimension record history. [0003] SCD is divided into three types, you can use Warehouse Builder to define, deploy and load these three types of SCD, respectively: [0004] Type 1 SCD - Coverage [0005] In Type 1 SCD, new data overwrites existing data. So existing data is lost and not stored anywhere else. This is the default type of dimension created and you do not need to specify any additional information to create a Type 1 SCD. [0006] Type 2 SCD - Creates another ...

AI Technical Summary

Problems solved by technology

[0012] The conventional slow-changing dimension method is positioned on the life cycle management of a single business record, and records the start time and end time of a single business data, so that the snapshot of the business data of the business date can be taken out through the specified business date, and it is necessary to The input-output (IO) cost of scanning is much larger than the actual size of the business date, which affects the efficiency of the application in taking snapshots. It can only solve the integrity of snapshots, but not the convenience and high cost of using snapshots. performance

[0013] The disadvantage of the existing conventional slow-changing dimension technology is that each preprocessing and business snapshot needs to scan IO much larger than the specified snapshot, which limits the efficiency of preprocessing and the efficiency of using snapshots

Images