An etl process execution system and method
An execution system and process technology, applied in the field of ETL process execution system, can solve problems such as unfixed conditions and inapplicable data processing scenarios, and achieve the effect of improving flexibility and increasing the scope of application
Active Publication Date: 2018-12-14
方正国际软件(北京)有限公司
View PDF3 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] (1) The source of data to be processed is flexible and not fixed in advance. It is likely that the location and specific format of the data source will be determined by the external conditions determined in real time;
[0005] (2) The conditions to execute the ETL process are not fixed, it may also be determined dynamically
[0013] In addition, the above-mentioned dynamically determined specific rules are flexible and diverse, and it is difficult to implement a set of fixed patterns in specific data processing requirements. Therefore, a traditional ETL tool is not suitable for the above data processing scenarios
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0050] Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.
[0051] Such as figure 1 As shown, the ETL process execution system in this embodiment includes an ETL process execution device 1 , an execution policy device 2 , a display device 3 , a selection device 4 and a data container device 5 . The ETL process execution device 1 includes a window unit 11 .
[0052]The ETL process executing device 1 is used for executing the ETL process. The execution policy device 2 is used to monitor the start event of the ETL process execution device 1, and notify the ETL process execution device 1 of starting in the form of a message. The display device 3 is used to display the basic information of all execution policy devices 2 and data container devices 5 in the system, including names, function information and so on. The selection means 4 is used for selecting the execution policy means 2 and the data containe...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an ETL process execution system and method, and belongs to the technical field of data processing. According to the method, an ETL process start event is firstly monitored, when the ETL process start event occurs, a notification message is sent to an ETL process, and the ETL process is started; an ETL process execution device determines data source parameters and determines the positions wherein data sources are located according to the data source parameters, the data sources are opened, the ETL process is executed, and process execution results are written in a target data source until all the data sources are executed. An execution strategy device is added, so that operation time points of the ETL process can be flexibly determined according to site needs, and the application range of the ETL process execution system is greatly increased. A data container device is added, so that the function of determining the positions of the data sources in the operating time is achieved, the ETL process execution system is no longer limited to processing of fixed data sources, and flexibility is greatly improved.
Description
technical field [0001] The invention belongs to the technical field of data processing, and in particular relates to an ETL process execution system and method, and is especially suitable for the field of data mining. Background technique [0002] ETL (Extraction-Transformation-Loading, data extraction, conversion, and loading) is responsible for extracting data from distributed and heterogeneous data sources, such as relational data, flat data files, etc., to the temporary middle layer for cleaning, transformation, integration, and finally loading In the data warehouse or data mart, it becomes the basis of online analytical processing and data mining. The process of ETL extracting, processing, and loading data is usually completed by specialized ETL software. ETL software usually provides several data source objects to support different types of data source formats, and provides several different types of data processing units, using any combination of these data processin...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 周大良李欣王生李强曹五丰
Owner 方正国际软件(北京)有限公司