Spatio-temporal stream data integration platform

Abstract

The invention provides a spatio-temporal stream data integration platform, which comprises a data transmission module for establishing control and management functions on the basis of distributed message-oriented middleware so that a user can establish a transmission channel between data source-connector-message-oriented middleware -connector-data destination; auser management module used for registering and auditing users, managing historical operation logs and distributing permissions to the users; atask management module used for providing new establishment, control and notification functions of tasks; wherein the task refers to a process of executing and finishing one-time data transmission through a transmission channel established by the data transmission module; anoperation and maintenance monitoring module used for managing and controlling all data transmission processes and progress conditions through a visual operation and maintenance board and providing diversified graphs; areal-time calculation module used for providing real-time calculation for filtering, aggregating and summarizing spatio-temporal stream data in combination with streaming distributed calculation middleware; and a data cleaning module. According to the invention, the space-time stream data transmission performance is greatly improved.

Description

technical field [0001] The invention relates to a data processing and transmission system, in particular to a space-time flow data integration platform. Background technique [0002] A variety of real-time streaming data information such as bus card swiping records, driving behavior recorded by driving recorders, GPS / Beidou positioning data, aircraft and ship situation information, equipment operating conditions, and satellite earth observation information constitute a smart city, The foundation of the Internet of Things, Industry 4.0. Since these real-time information have time and space attribute tags, they describe the state change information of an object in the historical time dimension and geographical latitude. The analysis of these data is the process of trying to grasp and control the law of its change. With the development of Internet of Things, big data and artificial intelligence technologies, spatio-temporal flow data has shown explosive growth. [0003] The i...

AI Technical Summary

Problems solved by technology

However, the existing data integration software systems and methods are mostly used for batch processing of offline files, such as patents: data integration system and data integration method (application number 201310392454.3), a data integration system and method (application number: 201811558696.4), based on aluminum The data integration platform of the electrolysis plant (application number: 201711481111.9), whose purpose is to integrate business data (operational data) files from the databases of multiple application systems into a unified data warehouse, so as to support delayed decision-making analysis, such The integrated system does not have high throughput and high concurrency capabilities, nor can it support real-time stream computing methods, and it is completely unable to meet the integration requirements of spatiotemporal stream data, making it difficult to integrate and utilize spatiotemporal stream data and benefit from it

Images