Supercharge Your Innovation With Domain-Expert AI Agents!

Method for dynamically incrementally loading data based on relational database

An incremental loading and database technology, applied in the database field, can solve problems such as complex management, low data conversion efficiency, and workload reduction, and achieve the effects of reducing workload, reducing the difficulty of learning and using, and improving efficiency

Pending Publication Date: 2020-04-28
UNICLOUD (NANJING) DIGITAL TECH CO LTD
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The object of the present invention is to provide a kind of method based on relational database dynamic incremental loading data, by setting up data source configuration template table and dynamic incremental extraction configuration template; Reduce the workload of ODS table creation ETL process and task, ETL configuration All ODS tables are managed in a standardized way, which reduces the difficulty of maintenance and solves the existing problems of low data conversion efficiency and complex management

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for dynamically incrementally loading data based on relational database
  • Method for dynamically incrementally loading data based on relational database
  • Method for dynamically incrementally loading data based on relational database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0038] see figure 1 As shown, the present invention is a method for dynamically incrementally loading data based on a relational database, comprising the following steps:

[0039] A00: Establish a data source configuration template table to solve the problem of establishing a data warehouse in a multi-data source scenario;

[0040] The data source configuration template table structure is as follows:

[0041] field name Field Type field description ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for dynamically and incrementally loading data based on a relational database, and relates to the technical field of databases. The method comprises the steps of establishing a data source configuration template table; establishing a dynamic increment extraction configuration template table, and fixing all ETL configuration assemblies into a plurality of items; creating a storage process; creating a timing scheduling task in a target database through a database SQL language; inputting the data source configuration of the ODS table into a data source configuration template table according to a configuration template; inputting ETL configuration of the ODS table into a dynamic increment extraction configuration template table according to a configuration template; and starting a timing scheduling task, and extracting data in a timing dynamic increment manner. The method comprises the following steps: establishing a data source configuration template tableand a dynamic increment extraction configuration template. The workload of creating the ETL process and tasks of the ODS table is reduced, all ODS tables are managed in an ETL configuration mode, andthe maintenance difficulty is reduced.

Description

technical field [0001] The invention belongs to the technical field of databases, in particular to a method for dynamically incrementally loading data based on a relational database. Background technique [0002] A data warehouse is a central repository for decision analysis. The way to build a data warehouse is to regularly extract and convert data from other sources such as transaction systems and relational databases according to different topics, and finally load the cleaned data into the central repository to provide decision-makers with a basis for decision-making . [0003] At present, the establishment process of the data warehouse needs to rely on the combined development of C, Java, Python, SQL and other languages; and the data warehouse needs to split the data extraction process of multiple data sources into multiple data extraction processes of single data sources, resulting in The process of building a warehouse is very cumbersome; in addition, after the struc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/25G06F16/242G06F16/28
CPCG06F16/254G06F16/2433G06F16/284
Inventor 史叶强
Owner UNICLOUD (NANJING) DIGITAL TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com