A data processing method and system for a data warehouse

Abstract

The invention provides a data processing method for a data warehouse, which comprises: extracting original data from service systems and generating source system data tables; eliminating redundancy of the source system data tables, recording data histories and establishing historical data tables; establishing fact data tables to classify the data of the historical data tables; summarizing data of the fact data tables and establishing summarized data tables; counting index data among two or more summarized data tables, and establishing a statistic data table; based on the data of the fact data tables, the summarized data tables and the statistic data table, establishing an application data table. According to the data processing method for the data warehouse, which is provided by the invention, mutual dependency of tasks inside each level in a universal dimensional model layer is avoided, the parallel number of the tasks is maximized and computer resources can be effectively utilized, so that the data processing efficiency of the data warehouse is improved. The invention also provides a corresponding data processing system for the data warehouse, which can realize the data processing method, which is disclosed by the invention, for the data warehouse.

Description

Technical field

[0001] The invention relates to the field of databases, in particular to a data processing method and system for a data warehouse. Background technique

[0002] Database is a warehouse that organizes, stores and manages data according to data structure. The processing of database data can be roughly divided into two categories: one is operational processing, which is usually used to query and modify a few records in the database; the other is analytical processing, which is generally used for The historical data is analyzed so that the data can be applied to decision-making. The subject-oriented, integrated, time-related, and unmodifiable data collection obtained after the analysis-type processing can be called a data warehouse. The main task of the data warehouse is to organize and reorganize information, and provide it to decision-makers in a timely manner. At present, data processing in a data warehouse is usually implemented through a distributed system, whi...

Examples

Embodiment Construction

[0066] In order to enable those skilled in the art to better understand the technical solutions in this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described The embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of the present invention.

[0067] The following describes the first embodiment of the data processing system of the data warehouse of this application. figure 1 It is the composition structure diagram of the embodiment of the data warehouse data processing system of this application. Such as figure 1 As shown, the data processing system of the data warehouse of this applicat...

Technical field

[0001] The invention relates to the field of databases, in particular to a data processing method and system for a data warehouse. Background technique

[0002] Database is a warehouse that organizes, stores and manages data according to data structure. The processing of database data can be roughly divided into two categories: one is operational processing, which is usually used to query and modify a few records in the database; the other is analytical processing, which is generally used for The historical data is analyzed so that the data can be applied to decision-making. The subject-oriented, integrated, time-related, and unmodifiable data collection obtained after the analysis-type processing can be called a data warehouse. The main task of the data warehouse is to organize and reorganize information, and provide it to decision-makers in a timely manner. At present, data processing in a data warehouse is usually implemented through a distributed system, whi...

Examples

Embodiment Construction

[0066] In order to enable those skilled in the art to better understand the technical solutions in this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described The embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of the present invention.

[0067] The following describes the first embodiment of the data processing system of the data warehouse of this application. figure 1 It is the composition structure diagram of the embodiment of the data warehouse data processing system of this application. Such as figure 1 As shown, the data processing system of the data warehouse of this applicat...

Images