Data processing method and device based on flink data engine

A data engine and data processing technology, which is used in electrical digital data processing, multi-programming devices, inter-program communication, etc. It can solve the problem of inability to view data engine data throughput information in real time, and achieve the effect of improving the degree of transparency.

Active Publication Date: 2021-02-05
HANGZHOU ANHENG INFORMATION TECH CO LTD
View PDF5 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present application provides a data processing method, device, electronic device and storage medium based on the flink data engine, so as to at least solve the problem in the related art that the data throughput information of the job of the data engine cannot be viewed in real time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device based on flink data engine
  • Data processing method and device based on flink data engine
  • Data processing method and device based on flink data engine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. In addition, it can also be understood that although such development efforts may be complex and lengthy, for those of ordinary skill in the art relevant to the content disclosed in this application, the technology disclosed in this application Some design, manufacturing or production changes based on the content are just conventional technical means, and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a data processing method and device based on an flink data engine, an electronic device and a storage medium. The method comprises the steps: receiving instruction informationof data monitoring, and enabling the instruction information to carry first operation information of an flink operation requesting the data monitoring; obtaining first application information associated with all the flink jobs according to the instruction information; and calling a plurality of candidate flink jobs in a preset deployment mode according to the first application information. According to the method, the target flink operation is detected in the plurality of candidate flink operations according to the first operation information, and the data throughput information of the targetflink operation is generated according to the state information of the target flink operation, so that the problem that the data throughput information of the flink operation based on the flink dataengine cannot be checked in real time is solved; therefore, transparency degree of the data throughput information of the flink operation based on the flink data engine is improved.

Description

technical field [0001] The present application relates to the technical field of data auditing, in particular to a data processing method, device, electronic device and storage medium based on a flink data engine. Background technique [0002] At present, when testing the performance of the product, if you want to obtain performance data, some job (job) running status, back pressure, etc., you need to manually visit the web interface of flink (distributed stream data engine) to view, because the job is running On yarn (yet another resource negotiator, another resource coordinator), we need to visit the web (World Wide Web, World Wide Web) interface of yarn first, and get the application (the application of the job running on yarn) ID (identity identification) Information) corresponding to the running job, and then enter the flink web interface, and then view the corresponding metric (data throughput indicator). However, the web interface also has imperfections. For example,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/32G06F9/54
CPCG06F11/324G06F9/547Y02D10/00
Inventor 魏燕子范渊刘博
Owner HANGZHOU ANHENG INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap