Platform architecture and processing method based on multi-engine data modeling calculation analysis

Abstract

The invention discloses a platform architecture based on multi-engine data modeling calculation analysis, including a front-end module for calculating task configuration, providing a WEB page and a graphical interface for users to configure data processing models and calculation processing tasks; a unified standard data access service module, Receive external data access requests and provide external data services; multi-source data input and output service modules realize data interaction between modules within the platform, as well as data interaction logic functions between the platform and external data sources; computing task configuration service modules for The computing task configuration front end provides data processing model configuration and data processing task configuration service functions, and stores these configuration information in the platform metadata database; the platform metadata database module stores the platform data processing model configuration information and data processing task configuration information. The invention can solve the deficiencies of the prior art, and improves the reusability of data processing logic.

Description

technical field [0001] The invention belongs to the technical field of industrial big data, in particular to a platform architecture based on multi-engine data modeling, calculation and analysis, and a processing method thereof. Background technique [0002] With the advancement of information, electronics and mathematical technology in modern industrial production, and the rapid development of sensors, Internet of Things and other technologies, a number of intelligent, high-precision, long-life, cost-effective, micro-sensors are available. Networking is supported by mobile data communication, which can collect and transmit data at any time and anywhere. These technologies greatly reduce the technical threshold and expenditure cost of industrial data collection and processing. These factors have led to the explosive growth of industrial production data. [0003] The industrial big data platform has become an indispensable information management tool for modern industrial pr...

AI Technical Summary

Problems solved by technology

However, these exquisitely designed and powerful management platforms still have some shortcomings:

[0004] 1. At the data processing level, the computing framework of a single data computing engine is often not suitable for data processing in a certain data processing scenario

[0005] 2. At the data storage level, enterprises have a variety of production data types, and it is difficult for a single storage platform to meet the high concurrency, high throughput, and low-latency read and write access requirements of all types of data in an enterprise

Moreover, building a data processing task through a big data platform often requires a complicated process, which is time-consuming and labor-intensive.

Images