Database conversion and cleaning information processing method

An information processing method and database technology, which is applied in the field of database conversion and cleaning information processing, can solve problems such as high cost and complicated use, and achieve the effect of ensuring consistency and integrity and avoiding data duplication and omission

Inactive Publication Date: 2012-04-11
上海众融信息技术有限公司
View PDF6 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Although there are many professional tools for data cleaning on the market, such as Ascential’s Datastage, Informatica’s Powercenter, NCR Teradata’s ETL Automation, etc., most of these tools are powerful, but at the same time their use is

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Database conversion and cleaning information processing method
  • Database conversion and cleaning information processing method
  • Database conversion and cleaning information processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0031] Such as figure 1 , figure 2 As shown, a database conversion and cleaning information processing method includes the following steps:

[0032] 1) The target database 1 is connected to the data source 2;

[0033] 2) Select the target data table that needs to be cleaned in the target database 1;

[0034] 3) Select the update method, if it is an incremental update, then perform step 4); if it is a full update, then perform step 10);

[0035] 4) Obtain the maximum update time last_update in the target data table. If the target data table is empty, last_update defaults to the set time;

[0036] 5) Filter all records in data source 2 whose update time is greater than last_update to a temporary table temp_table;

[0037] 6) Use the constraint fields in the target data table to remove duplicate records in the temporary table temp_table;

[0038] 7) By comparing the target data table with the temporary table temp_table, obtain the records existing in the target data table i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a database conversion and cleaning information processing method, which comprises the following steps of: 1) connecting a target database to a data source; 2) selecting a target data table to be cleaned in the target database; 3) selecting an update mode, executing the fourth step if the incremental update is adopted, and executing the tenth step if the total update is adopted; 4) obtaining the maximum update time last_update in the target data table, and defaulting the last_update as the set time if the target data table is null; 5) screening all records with the update time greater than the last_update in the data source to a temporary table temp_table; 6) deleting repeated records in the temp_table by restraining fields in the target data table; 7) comparing the target data table and the temp_table and obtaining the records of the temp_table in the target data table, and the like. Compared with the prior art, the method has the advantages that the problems of data repetitiveness and omission in the data cleaning process are effectively avoided, the data consistency and the completeness are ensured, and the like.

Description

technical field [0001] The invention relates to a database-related technology, in particular to a database conversion and cleaning information processing method. Background technique [0002] Data cleaning and transformation, also known as ETL (Extract, Transform, Load), is a problem that often needs to be solved in the field of databases, especially in the field of data warehouses. ETL is responsible for extracting data from distributed and heterogeneous data sources such as relational data, flat data files, etc. to the temporary middle layer for cleaning, conversion, integration, and finally loading into the target database (data warehouse, data mart, etc.), Become the foundation of online analytical processing and data mining. [0003] Although there are many professional tools for data cleaning on the market, such as Datastage of Ascential, Powercenter of Informatica, ETL Automation of NCR Teradata, etc., most of these tools are powerful, but their use is also relativel...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 雷发晶
Owner 上海众融信息技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap