Unlock instant, AI-driven research and patent intelligence for your innovation.

Data extraction method and apparatus adaptive to multiple data sources

A data extraction and data source technology, applied in the field of information processing, can solve problems such as high hard coding costs and poor stability, and achieve the effect of reducing maintenance costs and improving stability

Inactive Publication Date: 2017-06-23
DATANG SOFTWARE TECH
View PDF8 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to solve the problems of high cost and poor stability of hard coding in the data extraction process in the prior art, the embodiment of the present invention expects to provide a data extraction method and device adaptable to various data sources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data extraction method and apparatus adaptive to multiple data sources
  • Data extraction method and apparatus adaptive to multiple data sources
  • Data extraction method and apparatus adaptive to multiple data sources

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0041] refer to figure 1 , which shows a flow chart of the steps of Embodiment 1 of a data extraction method adapted to multiple data sources according to the present invention, which may specifically include:

[0042] Step 101, determine the corresponding data extraction rules according to the target data source type;

[0043] Specifically, the target data source adapted by the embodiment of the present invention may be a relational database, a non-relational database NoSQL, and a data file. Wherein, relational database can be any type of relational database such as Oracle, MySQL, DB2, SQL Server; Data file can be TXT, XML, Excel, CSV etc., the present invention is to the type of relational database, the type of non-relational database And the type of data file is not limited.

[0044] Among them, the data extraction rules describe how to extract data from the corresponding target data source. During specific implementation, the corresponding data extraction rules are deter...

Embodiment 2

[0062] refer to figure 2 , which shows a flow chart of steps in Embodiment 2 of a data extraction method adapted to multiple data sources according to the present invention, which may specifically include:

[0063] Step 201, pre-set data extraction rules in the configuration file;

[0064] Step 202, determine the corresponding data extraction rules from the configuration file according to the target data source type;

[0065] Step 203, generating a data extraction instruction according to the determined data extraction rule;

[0066] Specifically, the data extraction instruction may be generated according to the determined data extraction rule ID and actual required parameters. If the current data extraction instruction is a SQL statement or a stored procedure, it will be assigned and executed directly according to the parameters passed in; if the data extraction rule is defined in the form of a data extraction class, the parameters will be dynamically instantiated through ...

Embodiment 3

[0076] refer to image 3 , shows a structural block diagram of an embodiment of a data extraction device adapted to multiple data sources in the present invention, the device includes: an extraction rule determination module 301, an extraction instruction generation module 302, a data source adaptation module 303, and an instruction execution module Module 304; wherein,

[0077] An extraction rule determination module 301, configured to determine a corresponding data extraction rule according to the target data source type;

[0078] An extraction instruction generation module 302, configured to generate a data extraction instruction according to the data extraction rule;

[0079] A data source adaptation module 303, configured to perform adaptation processing on the target data source;

[0080] An instruction execution module 304, configured to execute the data extraction instruction to obtain target data from the target data source.

[0081] In an optional embodiment of th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Embodiments of the invention provide a data extraction method and apparatus adaptive to multiple data sources. The method comprises the steps of determining a corresponding data extraction rule according to a target data source type; generating a data extraction instruction according to the data extraction rule; performing adaptive processing on the target data source; and executing the data extraction instruction to obtain target data from the target data source. When an extraction demand is changed, only the extraction rule in a configuration file needs to be adjusted, and any program code does not need to be modified, so that the maintenance cost is effectively reduced and the system stability is greatly improved.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a data extraction method and device adapted to multiple data sources. Background technique [0002] At present, enterprises have greatly improved their production efficiency and management level by adopting information technology solutions. However, due to differences in data source types between different information systems, such as different relational databases Oracle, MySQL, DB2, SQL Server, etc., information exchange and sharing have become very difficult. [0003] In order to realize the data exchange between different information systems, the existing technology mostly adopts hard coding to solidify the data extraction process of different types of data sources in the program. However, the flexibility of the hard-coded method is very poor, and any change in the data source requires re-modification of the hard-coded part, which not only has high processing ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/254
Inventor 栗增光陈彬程之明
Owner DATANG SOFTWARE TECH