Unlock instant, AI-driven research and patent intelligence for your innovation.

ETL data processing method and device, computer equipment and storage medium

A data processing and data technology, applied in the direction of electronic digital data processing, special data processing applications, database management systems, etc., can solve the problems of unsatisfactory program automation and insufficient automatic and effective collection management process, so as to improve data processing efficiency, The effect of fast processing

Inactive Publication Date: 2020-08-11
杭州城市大数据运营有限公司 +2
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the embodiments of the present invention is to provide an ETL data processing method, device, computer equipment, and storage medium, aiming to solve the problem that the ETL tools in the prior art are not automatic enough for the collection and management process of data, and cannot meet the needs of program automation The problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • ETL data processing method and device, computer equipment and storage medium
  • ETL data processing method and device, computer equipment and storage medium
  • ETL data processing method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0028] Such as figure 1 As shown, in one embodiment, a flow chart of an ETL data processing method is given, and this embodiment is mainly described by taking the method applied to a terminal or a server as an example. A kind of ETL data processing method, specifically can comprise the following steps:

[0029] Step S101, connect to the source database, and obtain the first source data table from the source database;

[0030] Step S102, identifying and matching the first source data table through a preset standard source data table, and synchronizing the data in the first source data table to the standard source data table;

[0031] Step S103, associating the standard source data table with a preset standard target table, synchronizing the data of the standard source data table into the standard target table, so as to synchronize the data of the standard target table to the target database, Complete ETL data processing.

[0032] In the embodiment of the present invention, t...

Embodiment 2

[0059] Such as Figure 6 As shown, in one embodiment, an ETL data processing device is proposed, and in this embodiment, the device can be integrated in a terminal or a server. An ETL data processing device may specifically include:

[0060] A data acquisition unit 610, configured to connect to the source database, and acquire the first source data table from the source database;

[0061] A source data standard processing unit 620, configured to identify and match the first source data table through a preset standard source data table, and synchronize data in the first source data table to the standard source data table;

[0062] The target table synchronization processing unit 630 is configured to associate the standard source data table with a preset standard target table, and synchronize the data of the standard source data table into the standard target table, so that the standard target table The data is synchronized to the target database to complete the ETL data proce...

Embodiment 3

[0090] In one embodiment, a computer device is proposed. The computer device includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, the following steps are implemented:

[0091] Connect with the source database, and obtain the first source data table from the source database;

[0092] Identifying and matching the first source data table through a preset standard source data table, and synchronizing data in the first source data table to the standard source data table;

[0093] Associating the standard source data table with a preset standard target table, synchronizing the data of the standard source data table into the standard target table, so as to synchronize the data of the standard target table to the target database, and completing the ETL data deal with.

[0094] Figure 8 An internal block diagram of a computer device in one embodiment is shown. Specifically, the c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of computer data processing, and discloses an ETL data processing method and device, computer equipment and a storage medium. The ETL data processing method comprises the following steps: connecting with a source database, and obtaining a first source data table from the source database; identifying and matching the first source data table through a preset standard source data table, and synchronizing data of the first source data table to the standard source data table; and associating the standard source data table with a preset standard target table, and synchronizing data of the standard source data table to the standard target table. According to the ETL data processing method provided by the embodiment of the invention, other source data tables are configured to the standard source data table, and the standard source data table is synchronized to the target database, so that data processing can be quickly carried out, and the data processing efficiency is improved.

Description

technical field [0001] The invention belongs to the technical field of computer data processing, and in particular relates to an ETL data processing method, device, computer equipment and storage medium. Background technique [0002] Information is an important resource of modern enterprises and the basis for enterprises to use scientific management and decision-making analysis. Enterprises use ETL (Extract-Transform-Load, extract, transform, load) technical means to extract heterogeneous source data from different data sources, clean, transform, and load the extracted source data, and store the processed data into the data warehouse. [0003] At present, the main research direction of the cutting-edge ETL technology is to improve development efficiency and make data management more orderly through component packaging, natural language, metadata management and other technologies. However, the current inventions on the market can only support some simple sql operation state...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/242G06F16/25G06F16/28
CPCG06F16/2433G06F16/254G06F16/285
Inventor 陈肖雅陈小勇金睿颖
Owner 杭州城市大数据运营有限公司