Data processing method and system of data library

A data warehouse and data processing technology, applied in the direction of electrical digital data processing, special data processing applications, multi-programming devices, etc., can solve the problems of unreasonable utilization of network and server resources, inability to process ETL process in time, low processing efficiency, etc. , to achieve the effect of solving timely refreshing problems, saving synchronization and refreshing time, and improving efficiency

Inactive Publication Date: 2008-01-16
ALIBABA GRP HLDG LTD
View PDF0 Cites 56 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0014] The technical problem to be solved by the present invention is to provide a data warehouse OLAP system data ETL process processing method and system to solve the problems of low processing efficiency and unreasonable utilization of network and server resources, as well as the inconvenience of program modification and the inability to process ETL in time process exception

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system of data library
  • Data processing method and system of data library
  • Data processing method and system of data library

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0051] The core idea of ​​the present invention is: by setting the data synchronization configuration table and the data refresh configuration table, the ETL process of the data warehouse OLAP system adopts a multi-process parallel synchronous refresh mode, and after the synchronous processing waits for the data source to recover, multiple data sources are started to be synchronized Process; refresh processing According to the completion of the synchronization process of the same data object and the refresh scheduling relationship of the data object, multiple refresh processes are started for multiple data objects; synchronization and refresh processing are started at the same time, and the same data object is refreshed in time afte...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data handling method and a system in database to solve the problem that the single course treatment requires great amount of time and manpower cost and can not handle abnormities in ETL process. The method includes: arrange data synchronous configuration table and data refreshing configuration table; configuration information of the corresponding data source, start up a plurality of synchronous courses in parallel; refreshing grades, and start up a plurality of refreshing courses to make sequential synchronous and refreshing treatment on a plurality of data sources. The invention also includes parallel monitoring treatment; periodically checking synchronous refreshing information table; any abnormity discovered should be recorded in the alarm information table; the invention also includes parallel alarm treatment: Periodically checking alarm information table; alarm information should be sent immediately for new abnormities. The invention enhances the efficiency of synchronization and refreshing and saves a great deal of time and manpower cost. Real-time monitoring ensures timely handling of problems and safe, reasonable, and stable operation of database.

Description

technical field [0001] The invention relates to data processing of a data warehouse, in particular to a processing method and system for the process of data extraction, transformation and transfer (ETL) in a data warehouse OLAP system. Background technique [0002] A data warehouse is a subject-oriented, integrated, time-related, and unmodifiable collection of data in enterprise management and decision-making, that is, for all application systems, such as customer relationship management (CRM, Customer Relationship Management) systems, financial systems, etc. , integrate by topic, and record the entire history of changes. With the continuous improvement of enterprise informatization, a large amount of business data has been accumulated within the enterprise, and the data warehouse system is used to uniformly process these mutually independent and scattered data to meet the needs of high-level decision-making and analysis of the enterprise. [0003] Referring to Figure 1, it...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F9/46
Inventor 史健樊亚飞张金银
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap