Method for realizing heterogeneous data source data aggregation in data lake environment

A heterogeneous data source and data aggregation technology, applied in structured data retrieval, database management systems, special data processing applications, etc., can solve problems such as database differences

Active Publication Date: 2020-10-02
IREADY INFORMATION TECH BEIJING CO LTD
View PDF6 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a method for realizing data aggregation of heterogeneous data sources in a data lake environment, so as to solve the phased, technical, and other economic and artif...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] The invention provides a technical solution:

[0032] A method for realizing data aggregation of heterogeneous data sources in a data lake environment, which is composed of the following method modules, the method modules include a query processing module, a transaction management module, an integrity constraint module, a communication processor module and an application module, and a query processing module It is composed of a retrieval sub-module, a terminal system module and a display module. The retrieval sub-module inside the query processing module is used for retrieval of different data, and the terminal system module inside the query processing module is used for operator operations. The display module inside the processing module is used for data display;

[0033] The transaction management module is composed of local transactions and global transactions. The local transactions inside the transaction management module have the role of transactions submitted by ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for realizing heterogeneous data source data aggregation in a data lake environment. Method module, the method module comprises a query processing module, a transaction management module, an integrity constraint module, a communication processor module and an application module. The query processing module is composed of a retrieval sub-module, a terminal system module and a display screen module. According to the invention, the complexity and feasibility of data access, integration and sharing are realized on the basis of the existing theory and technology; firstly, a unified middleware framework is provided; different local accesses are realized at the bottom layer; a middle layer realizes unified middleware operation, a high-level application can carry out transparent access, and a federated structure mode in a CSCW system structure, database metadata directory service, a three-layer cooperation model of an Agent, a unified middle data conversion format XML and the like are adopted to solve the problem that heterogeneous databases are more convenient to gather.

Description

technical field [0001] The invention belongs to the technical field related to heterogeneous data sources, and in particular relates to a method for realizing data aggregation of heterogeneous data sources in a data lake environment. Background technique [0002] A heterogeneous database system is a collection of related multiple database systems, which can realize data sharing and transparent access. Several database systems already existed before joining the heterogeneous database system, and have their own data array management system and external database. Each component of the database system has its own autonomy. While realizing data sharing, each database system still has its own application characteristics, integrity control and security control. [0003] The existing methods and techniques for data aggregation of heterogeneous data sources in the data lake environment have the following problems: In the process of enterprise informatization construction, due to the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/25
CPCG06F16/258G06F16/252Y02D10/00
Inventor 吴奇锋
Owner IREADY INFORMATION TECH BEIJING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap