Unlock instant, AI-driven research and patent intelligence for your innovation.

Data updating method based on kettle tool

A data update and tool technology, applied in the database field, can solve the problems of insufficient use of ETL tools, high storage space redundancy, low update efficiency, etc., to improve data update efficiency, save database resources, and reduce development steps. Effect

Active Publication Date: 2020-02-21
北京首汽智行科技有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The data migration process takes a long time, has low parallelism, and has high storage space redundancy. Only incremental data is migrated through kettles, and data is updated through SQL databases. ETL tools are not fully utilized, resulting in low update efficiency and wasting a lot of time and databases. Storage space, resulting in low utilization of database resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data updating method based on kettle tool

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The present invention will be specifically introduced below in conjunction with the accompanying drawings and specific embodiments.

[0020] Such as figure 1 As shown, the kettle tool-based data update method provided by the embodiment of the present invention includes the following steps:

[0021] S101, using the kettle tool to obtain the historical cycle of data update from the source data table, and generate a historical cycle set.

[0022] S102, using the kettle tool to obtain the longest cycle from the historical cycle set as the current data extraction cycle.

[0023] As a specific embodiment, if the historical period set is 1 day, 5 days and 10 days, then 10 days is selected as the current data extraction period.

[0024] S103, input the data extraction cycle as a parameter into the kettle tool, and use the kettle tool to obtain incremental data;

[0025] S104, copy three incremental data to obtain the first incremental data, the second incremental data and th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data updating method based on a kettle tool, and relates to the technical field of databases. According to the method, the historical period of data updating and the data extraction period of this time are obtained through the kettle tool, the kettle tool is fully utilized, a temporary table does not need to be established in a database, development steps are reduced, theparallelism degree of program execution is increased, and the data updating efficiency and the utilization rate of database resources are improved.

Description

technical field [0001] The invention belongs to the technical field of databases, and in particular relates to a data updating method based on a kettle tool. Background technique [0002] The current data update method mainly uses the key fields of the incremental data as parameters to obtain the incremental data transferred to the temporary table in the target database through kettle, and then associates the temporary table with the target table through the SQL database, and transfers the data required in the target table The updated data is deleted, and finally the incremental data is inserted into the target table. [0003] There is following defective in prior art: [0004] The data migration process takes a long time, has low parallelism, and has high storage space redundancy. Only incremental data is migrated through kettles, and data is updated through SQL databases. ETL tools are not fully utilized, resulting in low update efficiency and wasting a lot of time and da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23
CPCG06F16/23
Inventor 付秉毅
Owner 北京首汽智行科技有限公司