Stream computing platform and method based on visualization, equipment and storage medium

Abstract

The embodiment of the invention discloses a stream computing platform and method based on visualization, equipment and a storage medium. The stream computing platform comprises a data development component, a stream computing component and an intelligent scheduling component, wherein the data development component is used for visually developing a corresponding stream computing application througha dragging operator in stream computing, and determining a stream computing engine corresponding to the stream computing application; the stream computing component is used for analyzing scheduling parameters carried in the stream computing application and calling the stream computing engine to execute the stream computing application; and the intelligent scheduling component is used for scheduling execution of the stream computing application according to the scheduling parameters. The technical scheme provided by the embodiment of the invention does not need to be realized only by developers who deeply understand a stream computing application technology, so that the application range of the stream computing platform is expanded, and the expandability of different stream computing engines is supported, and the flexibility of stream computing development is improved.

Description

technical field [0001] The embodiments of the present invention relate to the field of data processing, and in particular to a visualization-based stream computing platform, method, device, and storage medium. Background technique [0002] In the traditional data processing process, the collected data is usually stored in the database in a unified manner, and when the user needs it, the data query is performed on the database to obtain the result, or to perform related data processing. With the rise of web applications, network monitoring, and sensor detection, a new data-intensive application has emerged: streaming data, that is, data is continuously generated in a large, fast, and time-varying stream. Traditional data processing methods, such as MapReduce offline processing, cannot handle these stream data well, so a new data processing method has emerged: stream computing, which can obtain massive data from different data sources in real time. Analyze and process to obta...

AI Technical Summary

Problems solved by technology

[0007] Both Spark Streaming and Flink are general-purpose open source large-scale streaming data processing engines. Currently, one system supports all data processing to improve performance. Both have a mature ecosystem, but both engines require experienced programs. Only staff can participate in the development, and current stream computing products usually use a fully managed method for stream computing, which does not support the processing interfaces of multiple complex applications, making stream computing development less flexible and unable to meet the needs of different applications for stream computing engines

Images