Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Unified data resource management system and method for large data platform

A big data platform and data resource technology, applied in database management systems, other database retrieval based on metadata, electronic digital data processing, etc., can solve problems such as difficult management of data resources, and achieve the effect of facilitating application and authorization

Active Publication Date: 2016-12-07
FUDAN UNIV
View PDF2 Cites 35 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Aiming at the problem that data resources are difficult to manage due to the existence of various types of data resources in the big data platform, the present invention proposes a unified metadata description for different types of data resources and a unified adapter interface for different types of data management components On this basis, a unified data resource management method and system is designed to support unified data package upload and download, unified data discovery, unified data access application and authorization, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Unified data resource management system and method for large data platform
  • Unified data resource management system and method for large data platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] The technical implementation of the present invention will be described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the examples.

[0032] figure 1 The corresponding relationship between HDFS data management module and HBase data management module and four-layer meta-model is given. In an embodiment, each HDFS cluster or HBase cluster corresponds to a data container. Each HDFS cluster maintains a three-level directory " / username / directoryname / filename", where: " / username / directoryname" corresponds to a data package, and "username" indicates the owner of the data package The username of the ; and the "filename" corresponds to the data asset. Each namespace in each HBase cluster corresponds to a data package, the table in the namespace corresponds to the data asset in the data package, and the column family in the table corresponds to the data field in the data asset.

[0033] figu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of large data platforms, in particular to a unified data resource management system and method for a large data platform. In order to solve the problem that the data resources are difficult to manage due to the presence of different types of data resources in the large data platform, the unified metadata description for different types of data resources and the unified adapter interface criterion for different types of data management components are provided, and the unified data resource management method and system are designed on such basis to support unified data upload and download, unified data discovery, unified data access application and authorization, and other functions. The unified data resource management system and method provided by the invention can realize the dynamic extension, multi-tenancy management and uniform access control of the system, so that a user can manage and use different types of data resources in the large data platform.

Description

technical field [0001] The invention belongs to the technical field of big data platforms, and in particular relates to a unified data resource management system and method for big data platforms. Background technique [0002] With the development of information technology, data has penetrated into every industry and business field today and has become an important factor of production. Effective management, further mining and application of massive data has become the key for enterprises to improve their core competitiveness and seize market opportunities. Against this background, big data technology emerged as the times require, which has distinctive features such as large data volume, multiple data types, low value density, and fast processing speed. [0003] On existing big data platforms (typically represented by Hadoop), different types of data resources are often managed separately through different data management components in the big data ecosystem. For example: ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30G06F21/62
CPCG06F16/182G06F16/25G06F16/27G06F16/907G06F21/6227G06F2221/2117G06F2221/2141
Inventor 谢志鹏胡俊峰王鹏飞
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com