Apparatus and method for extracting, converting and loading total source data

A source data and data technology, applied in the field of database source data processing, can solve the problem that the standard historical zipper algorithm cannot meet the requirements of full source data processing, and achieve the effect of meeting flexibility requirements, achieving unity and achieving transparency.

Active Publication Date: 2009-08-12
INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF0 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] In view of this, the main purpose of the present invention is to provide a device and method for extracting, transforming and loading full source data, so as to meet the database’s requirement for keeping historical traces of full source data, and to solve the problem of standard history problems encountered in enterprise-level database projects. The problem that the zipper algorithm cannot meet the processing requirements of the full amount of source data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Apparatus and method for extracting, converting and loading total source data
  • Apparatus and method for extracting, converting and loading total source data
  • Apparatus and method for extracting, converting and loading total source data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0051] The present invention processes the deleted records in the source system by "closing the chain", that is, sets the end date (End_Dt) of the corresponding record as the current batch date, indicating that the validity period of the record ends on that day, and realizes the full source of the current day. The data is loaded into the data warehouse and saved in the form of historical zippers to meet the data warehouse's demand for full source data to retain historical traces.

[0052] Such as figure 2 as shown, figure 2 It is a structural schematic diagram of the device for extracting, transforming and loading the full amount of source data provided by the present invention. The device includes a data receiving uni...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a device and a method for extracting, converting and loading gross source data. The device comprises a data receiving unit, a data processing unit, a data export unit and a database, wherein the database is provided with a temporary working area and a foundation area; the foundation area of the database is provided with an object list inside; the data receiving unit is connected with the database and used for receiving gross source data files from a source system and loading the gross source data files into the temporary working area of the database; the data processing unit is connected with the database and used for extracting, converting and loading the gross source data files which are loaded into the temporary working area of the database and storing the processed gross source data into the database in the mode of historical chains; and the data export unit is connected with the database and used for exporting the gross source data which is stored into the database for other systems to use. The invention meets the requirement of the database on the reservation of historical track of the gross source data.

Description

technical field [0001] The invention relates to the technical field of database source data processing, in particular to a device and method for extracting, converting and loading full source data. Background technique [0002] For the enterprise data warehouse (EDW), its most basic function is to extract, transform and load the data from the business source system according to certain rules (ie Extract, Transform and Load, ETL). Due to the diversity of source systems, the data from source systems are also varied, and cannot be extracted and transformed with a single rule. Enterprise-level data warehouse is to transform data with different attributes and changing rules into standardized data information that meets certain rules and conforms to database-related paradigms. It is very important for enterprises. Especially for large state-owned commercial banks, due to the massive amount of data, how to extract and organize this massive data into a complete set of orderly and ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 陈慧俞斌赵亚华张海军
Owner INDUSTRIAL AND COMMERCIAL BANK OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap