Dynamic heterogeneous metadata acquisition method and system

A dynamic heterogeneous and acquisition system technology, applied in the field of enterprise data resource management, can solve problems such as no explanation or report found, no data collected, etc., and achieve fast query speed and clear display effect

Inactive Publication Date: 2016-06-22
CETC CHINACLOUD INFORMATION TECH CO LTD +2
View PDF8 Cites 34 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At the same time, the real-time processing of massive data in enterprises, the diversity of data types, and the instantaneous update of data resource status pose challenges to the synchronous acquisition of dynamic metadata and metadata management methods
[0003] At present, there is no description or report of the similar technology of the present invention, and no similar data at home and abroad have been collected yet.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic heterogeneous metadata acquisition method and system
  • Dynamic heterogeneous metadata acquisition method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] This embodiment provides a dynamic heterogeneous metadata acquisition system, including a dynamic metadata acquisition module running on heterogeneous data sources, a dynamic metadata cache module running on metadata server nodes, a distributed message service module, Data synchronization service subscription module. Among them, the metadata dynamic acquisition module of the client provides a unified interface for dynamic metadata acquisition with any data storage (whether it is a relational database, a NoSQL database, or a spreadsheet or other format files). The cache module saves and changes metadata information, and issues metadata change requests through the distributed message service module; the metadata synchronization service subscription module subscribes to metadata change requests and maintains metadata dynamic partitions and metadata structure information.

Embodiment 2

[0057] This embodiment provides another structure of dynamic heterogeneous metadata acquisition system, the dynamic heterogeneous metadata acquisition system is a multi-layer architecture structure, such as figure 1 As shown, the functions of each layer are as follows:

[0058] The acquisition layer provides a unified metadata access interface for data sources, and provides different implementation methods for different metadata objects, including static metadata acquisition and dynamic metadata release;

[0059] The transport layer uses asynchronous communication and remote call protocol to realize the exchange between dynamic and static metadata;

[0060] The storage layer uses distributed storage technology to provide metadata storage and search;

[0061] The monitoring layer is used to complete the monitoring of metadata service nodes, and the data service nodes include: metadata collection adapters and metadata processing service nodes;

[0062]The service layer is a fu...

Embodiment 3

[0065] This embodiment provides a method for acquiring dynamic heterogeneous metadata, which can be executed by the system for acquiring dynamic heterogeneous metadata provided in Embodiment 1 and Embodiment 2, such as figure 2 shown, including the following steps:

[0066] Step S1: Register the metadata collection service to the monitoring center;

[0067] Step S2: Subscribe to the metadata status through the metadata collection service, where the metadata status includes any one or any of the following items: metadata MD5 check code, metadata acquisition method, and metadata average update frequency;

[0068] Step S3: The metadata collection service dynamically extracts metadata from structured data sources and unstructured data sources, and publishes the changed content as metadata. Through the definition of metadata format, that is, the XMLSchema form of metadata, publish to the network;

[0069] Step S4: The metadata synchronization service subscribes to the metadata r...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention provides a dynamic heterogeneous metadata acquisition method and system. A metadata dynamic acquisition module is provided with a unified interface used for dynamically acquiring metadata and is connected to a data storage device by means of the unified interface; a dynamic metadata cache module is used for storing and changing metadata information; a distributed message service module is used for publishing a metadata change request; and a metadata synchronization server subscription module is used for subscribing the metadata change request, and maintaining a metadata dynamic partition and metadata structural information. Moreover, provided is a second-type dynamic heterogeneous metadata acquisition system. According to the method and system provided by the present invention, a unified metadata system can be constructed based on a heterogeneous data source, and storage, management and usage abilities for the system are provided; and the method and system are applicable to providing a basic unified data standard for massive data mining and analysis, and lays a foundation for constructing an industry semantic library.

Description

technical field [0001] The present invention relates to heterogeneous metadata acquisition and management methods in the technical field of enterprise data resource management, in particular to a dynamic heterogeneous metadata acquisition method and system based on enterprise metadata acquisition and management. Background technique [0002] People use big data to describe and define the massive data generated in the era of information explosion, and name the related technological development and innovation. Data is rapidly expanding and becoming larger, and the types of data are abundant. How to manage these different types of data determines the future development of enterprises. Although enterprises may not realize the hidden dangers of problems caused by explosive data growth, as time goes by As time goes on, people will become more and more aware of the importance of data to enterprises. At the same time, the real-time processing of massive data in enterprises, the div...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/904G06F16/90335G06F16/907
Inventor 闵圣捷金日海靳永超漆杨
Owner CETC CHINACLOUD INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap