Multi-source data management mechanism and platform

Abstract

Platforms and mechanisms for multi-source data management are provided. In accordance with one aspect, a knowledge mining engine comprising an on-line knowledge mining component supporting multi-dimensional queries for on-line knowledge mining of multi-source data and an off-line knowledge mining component for running knowledge mining procedures for off-line knowledge mining of the multi-source data. The multi-source data management platform / mechanism may also include an adaptive data collector to collect data actively and passively, a data analysis cascade to cleanse and analyse the multi-source data concurrently and separately, a data integration cascade to consolidate and integrate the multi-source data according to multiple data integration dimensions, a centralized data repository implementing RESTful APIs for data CRUD operations and storing multi-source data generated in the platform, and a data sharing portal to share data and mined knowledge to users and up-layer applications.

Description

TECHNICAL FIELD[0001]The present invention generally relates to data management, and more particularly relates to a multi-source data management mechanism and platform.BACKGROUND OF THE DISCLOSURE[0002]With the rapid development of information and communication technology (ICT), diverse objects are connected to start upload of their unique data. And in the age of big data, tremendous data is generated through various sources. Such multi-source data (i.e., data generated from multiple objects) needs to be properly managed to improve efficiency and effectiveness of an application domain in Smart Cities, e.g., urban mobility, urban planning. However, most current data management systems only address issues related to big data storage and processing based on distributed computing technologies, instead of solving critical issues such as how to collect, integrate, analyze, mine and share multi-source data.[0003]For example, in order to implement a knowledge-based and responsive urban desi...

AI Technical Summary

Benefits of technology

The platform enables efficient and effective management of multi-source data by providing a unified framework for data collection, analysis, integration, knowledge mining, and sharing, supporting applications like urban mobility and planning by generating actionable insights from diverse data sources.

Problems solved by technology

However, most current data management systems only address issues related to big data storage and processing based on distributed computing technologies, instead of solving critical issues such as how to collect, integrate, analyze, mine and share multi-source data.

However, as required data are generated from different objects (e.g., sensors, social networks, mobile applications, mobile networks) and how to consistently and uniformly collect, analyze, integrate, mine, store and share these multi-source data becomes a challenging issue.

Images