Stream computing platform and method based on visualization, equipment and storage medium

A stream computing and platform technology, applied in the field of data processing, can solve problems such as low flexibility, no support for processing interfaces, and inability to meet requirements

Pending Publication Date: 2020-03-24
CHINA CONSTRUCTION BANK
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Both Spark Streaming and Flink are general-purpose open source large-scale streaming data processing engines. Currently, one system supports all data processing to improve performance. Both have a mature ecosystem, but both engines require experienced programs. Only staff can participate in the developmen

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stream computing platform and method based on visualization, equipment and storage medium
  • Stream computing platform and method based on visualization, equipment and storage medium
  • Stream computing platform and method based on visualization, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] Figure 1A The schematic diagram of a visualization-based stream computing platform provided by Embodiment 1 of the present invention is applicable to real-time processing of stream data. refer to Figure 1A , the stream computing platform 10 in this embodiment may include: a data development component 110 , a stream computing component 120 and an intelligent scheduling component 130 .

[0031] Among them, the data development component 110 is used to visually develop the corresponding stream computing application through the drag-and-drop operator in the stream computing, and determine the stream computing engine corresponding to the stream computing application; the stream computing component 120 is used to analyze the scheduling information carried in the stream computing application parameter, and invoke the stream computing engine to execute the stream computing application; the intelligent scheduling component 130 is used to schedule the execution of the stream com...

Embodiment 2

[0058] figure 2 A schematic diagram of a visualization-based stream computing platform provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. refer to figure 2 In this embodiment, the visualization-based flow computing platform 20 may include, in addition to the data development component 210 , the flow computing component 220 and the intelligent scheduling component 230 , a data collection component 240 , a data management component 250 and an operation component 260 .

[0059] Among them, the data collection component 240 is used to collect the stream data corresponding to the stream computing application in real time, and send it to the stream computing component 220, so that the stream computing component invokes the stream computing engine to process the stream data corresponding to the stream computing application; the data management component 250 is used to Manage the metadata and data permissions c...

Embodiment 3

[0064] image 3 It is a flowchart of a stream data processing method provided by Embodiment 3 of the present invention, and this embodiment is applicable to the case of real-time processing of stream data. The stream data processing method provided in this embodiment can be applied to the visualization-based stream computing platform provided in the embodiment of the present invention. The stream data processing method provided in this embodiment can be executed by the device provided in the embodiment of the present invention. The device executing the method in this embodiment may be a terminal device configured with a stream computing platform.

[0065] Specifically, refer to image 3 , the method may include the following steps:

[0066] S310. Obtain in real time the stream data corresponding to the stream computing application developed through the drag operator visualization in the stream computing, and determine the corresponding stream computing engine.

[0067] Spec...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a stream computing platform and method based on visualization, equipment and a storage medium. The stream computing platform comprises a data development component, a stream computing component and an intelligent scheduling component, wherein the data development component is used for visually developing a corresponding stream computing application througha dragging operator in stream computing, and determining a stream computing engine corresponding to the stream computing application; the stream computing component is used for analyzing scheduling parameters carried in the stream computing application and calling the stream computing engine to execute the stream computing application; and the intelligent scheduling component is used for scheduling execution of the stream computing application according to the scheduling parameters. The technical scheme provided by the embodiment of the invention does not need to be realized only by developers who deeply understand a stream computing application technology, so that the application range of the stream computing platform is expanded, and the expandability of different stream computing engines is supported, and the flexibility of stream computing development is improved.

Description

technical field [0001] The embodiments of the present invention relate to the field of data processing, and in particular to a visualization-based stream computing platform, method, device, and storage medium. Background technique [0002] In the traditional data processing process, the collected data is usually stored in the database in a unified manner, and when the user needs it, the data query is performed on the database to obtain the result, or to perform related data processing. With the rise of web applications, network monitoring, and sensor detection, a new data-intensive application has emerged: streaming data, that is, data is continuously generated in a large, fast, and time-varying stream. Traditional data processing methods, such as MapReduce offline processing, cannot handle these stream data well, so a new data processing method has emerged: stream computing, which can obtain massive data from different data sources in real time. Analyze and process to obta...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F8/20G06F8/34
CPCG06F8/20G06F8/34
Inventor 彭海平唐景峰苏毓仁林华邦陈铭新李晓敦赵世辉
Owner CHINA CONSTRUCTION BANK
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap