Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Unified data resource management system and method for big data platform

A big data platform and data resource technology, applied in database management systems, other database retrieval based on metadata, electronic digital data processing, etc., can solve the problem of difficult management of data resources, and achieve the effect of convenient application and authorization

Active Publication Date: 2020-05-26
FUDAN UNIV
View PDF2 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Aiming at the problem that data resources are difficult to manage due to the existence of various types of data resources in the big data platform, the present invention proposes a unified metadata description for different types of data resources and a unified adapter interface for different types of data management components On this basis, a unified data resource management method and system is designed to support unified data package upload and download, unified data discovery, unified data access application and authorization, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Unified data resource management system and method for big data platform
  • Unified data resource management system and method for big data platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] The technical implementation of the present invention will be described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0032] figure 1 The corresponding relationship between HDFS data management module and HBase data management module and the four-layer meta-model is given. In the embodiment, each HDFS cluster or HBase cluster corresponds to a data container. Each HDFS cluster maintains a three-level directory " / user name / directory name / file name", where: " / user name / directory name" corresponds to a data package, and "user name" represents the owner of the data package User name; and the "file name" corresponds to the data asset. Each namespace in each HBase cluster corresponds to a data package, the table in the namespace corresponds to the data asset in the data package, and the column family in the table corresponds to the data field in the data asset.

[0033] figure ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of large data platforms, in particular to a unified data resource management system and method for a large data platform. In order to solve the problem that the data resources are difficult to manage due to the presence of different types of data resources in the large data platform, the unified metadata description for different types of data resources and the unified adapter interface criterion for different types of data management components are provided, and the unified data resource management method and system are designed on such basis to support unified data upload and download, unified data discovery, unified data access application and authorization, and other functions. The unified data resource management system and method provided by the invention can realize the dynamic extension, multi-tenancy management and uniform access control of the system, so that a user can manage and use different types of data resources in the large data platform.

Description

Technical field [0001] The invention belongs to the technical field of big data platforms, and specifically relates to a unified data resource management system and method for big data platforms. Background technique [0002] With the development of information technology, data has penetrated into every industry and business area today, becoming an important production factor. Effective management of massive data and further mining and application have become the key to enterprises to improve their core competitiveness and seize market opportunities. In this context, big data technology emerged at the historic moment, which has distinctive features such as large data volume, multiple data types, low value density, and fast processing speed. [0003] On the existing big data platforms (typically represented by Hadoop), different types of data resources are often managed separately through different data management components in the big data ecosystem. For example: a distributed fi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/25G06F16/27G06F16/182G06F16/907G06F21/62
CPCG06F16/182G06F16/25G06F16/27G06F16/907G06F21/6227G06F2221/2117G06F2221/2141
Inventor 谢志鹏胡俊峰王鹏飞
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com