Implementation method and device for submitting flink job to yarn cluster in application program

A technology of application programs and implementation methods, applied in the field of big data computing, can solve the problems of space occupied by public resources, achieve the effects of reducing dependence, avoiding mutual interference, and improving deployment flexibility
CN114489833AActive Publication Date: 2022-05-13WUHAN DAMENG DATABASE
4 Cites 4 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
WUHAN DAMENG DATABASE
Publication Date
2022-05-13

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention relates to the technical field of big data computing, discloses an implementation method and device for submitting an flink job to a Yarn cluster in an application program, solves the deployment problem of a Hadoop configuration file and a third-party dependent jar package, and realizes self-defined deployment of the Hadoop configuration file and the third-party dependent jar package by rewriting a part of functions in an flink system class. The deployment flexibility, convenience and operation efficiency of the application programs are improved, the degree of dependence on the environment is reduced, and mutual interference possibly existing between different application programs is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

【Technical field】

[0001] The invention relates to the technical field of big data computing, in particular to an implementation method and device for submitting a flink job to a yarn cluster in an application program. 【Background technique】

[0002] Flink is a high-performance, high-throughput, low-latency stream processing framework. It is not only a stream processing framework, but also unifies batch processing (in Flink, batch processing is a special case of stream processing). This architecture of Flink also better solves the cumbersome component accumulation of the traditional big data architecture, allowing batch flow to perform batch or stream processing without changing the original code, thus becoming a more efficient An increasingly popular big data processing framework.

[0003] Flink supports a variety of deployment methods, such as Local, Standalone, Yarn, K8s, etc. Most enterprises now use yarn as a resource manager because of their big data platforms, so for...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More