A data lake system oriented to all data morphology open sharing

An all-data, data-based technology, applied in the field of open and shared data lake systems for all data forms, can solve problems such as lack of deep convergence of big data, lack of data innovation, and lack of data lake systems

Inactive Publication Date: 2019-01-25
GUANGDONG POLYTECHNIC NORMAL UNIV +1
View PDF5 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] During the research and practice of the existing technology, the inventors of the present invention found that there is a lack of a data lake system that supports cross-organization, cross-department, and cross-industry data aggregation and supports the sharing and opening of all data forms, which can solve the problems caused by big data. Lack of in-depth convergence leads to weak data innovation and lack of bright data applications

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data lake system oriented to all data morphology open sharing

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0035] see figure 1 .

[0036] Such as figure 1 As shown, this embodiment provides a data lake system oriented towards open sharing of all data forms, including a data producer platform layer 100, a data integration module 200, a data storage module 300, a data open module 400, and a consumer platform layer 500;

[0037] The data producer platform layer 100 is composed of one or more heterogeneous databases.

[0038] Specifically, various heterogeneous databases include relational databases, MPP databases, Hadoop databases, and file databases.

[0039] The data integration module 200 is configured to automatically acquire data from the data producer platform layer 100, and integrate the acquired data into the offline data lake and the real-time data lake of the data storage module 200 for storage.

[0040] Specifically, the data integration module 200 automatically acquires data to the data lake based on the IaaC cloud orchestration technology that integrates ETL and Kafka,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data lake system oriented to full data form open sharing, which comprises a data producer platform layer, an intermediate layer and a consumer platform layer. The middle layer includes a data integration module, a data storage module and a data opening module. The data producer platform layer consists of one or more heterogeneous databases; The data integration module isused for automatically acquiring data from the data producer platform layer, and integrating the acquired data into the offline data lake and the real-time data lake of the data storage module for storage; The data opening module is used for opening the data stored in the data storage module to the consumer platform layer according to the preset unified data directory, the standard open protocol and the SDN network optimization strategy. The consumer platform layer is used for data interaction with the consumer terminal. The invention can provide a data lake system oriented to full data form open sharing, which supports cross-organization, cross-department and cross-industry data convergence and supports full data form open sharing.

Description

technical field [0001] The present invention relates to the field of information technology, in particular to a data lake system open and shared for all data forms. Background technique [0002] With the rapid development of information technology, more and more enterprises have begun to deploy corresponding information systems. An enterprise's information system may have multiple subsystems at the same time, and different enterprises and different platforms have different forms of information systems. [0003] During the research and practice of the existing technology, the inventors of the present invention found that there is a lack of a data lake system that supports cross-organization, cross-department, and cross-industry data aggregation and supports the sharing and opening of all data forms, which can solve the problems caused by big data. The lack of in-depth convergence leads to weak data innovation and lack of bright data applications. Contents of the invention ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2458G06F16/25
Inventor 魏文国刘忻谢桂园蔡君
Owner GUANGDONG POLYTECHNIC NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap