Real-time data middle station processing method, device and platform

Abstract

The invention relates to a real-time data middle station processing method, device and platform, and the method comprises the steps: constructing a data synchronization service library and a data conversion service library according to the corresponding relation between a customer type and a preset data synchronization service library deployment strategy, a data conversion service library deployment strategy and environment configuration management; receiving the service data, analyzing the service data to obtain data source feature information, calling a corresponding target data conversion strategy according to the data source feature information, and converting the service data into corresponding storage data; constructing a storage identifier of the storage data based on the customer type and a preset storage data identification strategy; and aggregating and storing the storage data to a corresponding data synchronization service library and a data conversion service library according to the storage identifier and a preset storage strategy, and sending the storage identifier to a data query directory for storage. The method supports incremental data synchronization of various target databases and data sources, and is convenient for users to understand data meanings.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a processing method, device and platform for a real-time data center. Background technique [0002] With the development of the Internet, the number of enterprise business systems has increased, and the data sources are various. The data requirements for business operations are high-frequency and diverse. However, the data system is complex and the data is not uniform. It is difficult to guarantee the speed of data analysis and the accuracy and consistency of data. Strategy Decision-making and data-based operations are hindered, so a data center is needed to provide stable and consistent data access services. [0003] At present, the pain points of each user in the field of data development include the following: [0004] The data sources are complex, and the data generated by various information systems cannot be efficiently extracted and used. There are a lot ...

AI Technical Summary

Problems solved by technology

[0004] The data sources are complicated, and the data generated by various information systems cannot be efficiently extracted and used

There are many data processing and management software, it is difficult to connect software packages from different suppliers, and there is a lack of consistent user experience

Data silos make it difficult to maximize the value of data

Difficult data management, limited by many technical issues such as development language, technical framework, data storage, etc., unable to conduct unified management

Repeated development, lack of standardization and sharing and planning mechanism, repeated development, serious waste

Images