Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Resource adjustment method and device for Flink task processing, equipment and storage medium

A technology of task processing and adjustment methods, applied in the field of data processing, can solve problems such as inaccurate evaluation and unreasonable adjustment, and achieve the effect of accurate acquisition and reasonable and accurate adjustment of resources

Pending Publication Date: 2022-03-01
南京苏宁电子信息技术有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In the traditional technical solution, when the Flink framework performs task processing, the resource utilization is generally evaluated by the highest value and the average value of QPS (throughput), and resources are adjusted according to the evaluation results, which easily leads to inaccurate evaluation. Adjust unreasonable problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Resource adjustment method and device for Flink task processing, equipment and storage medium
  • Resource adjustment method and device for Flink task processing, equipment and storage medium
  • Resource adjustment method and device for Flink task processing, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

[0053] Please refer to figure 1 , figure 1 It is a schematic diagram of an application environment of a resource adjustment method for Flink task processing provided by an exemplary embodiment of the present application. Such as figure 1 As shown, the application environment includes a server 100 and a terminal 101, and a communicable connection between the server 100 and the terminal 101 can be realized through the network 102, so as to realize the resource adjustment method for Flink task processing of this application.

[0054] The server 100 is used to obtain the actu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a resource adjustment method and device for Flink task processing, equipment and a storage medium. The method comprises the following steps: acquiring the actual usage amount of each resource usage index in the execution process of a current Flink task; according to each actual usage amount and the index parameter of each resource usage index, generating a resource usage total score of the current Flink task; obtaining an adjustment strategy of the current Flink task according to the total resource use score and a mapping relation between the total resource use score and the adjustment strategy; and adjusting the resource allocation of the current Flink task according to the adjustment strategy. According to the method and the device, the resource use of the current Flink task is scored from multiple dimensions by setting a plurality of resource use indexes, and the adjustment strategy matched with the current Flink task is accurately positioned based on the mapping relationship between the total score of the resource use and the adjustment strategy, so that the adjustment strategy is more accurately obtained, the resources are adjusted based on the adjustment strategy, and the resource utilization rate is improved. And the resource adjustment is more accurate.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a resource adjustment method, device, device and storage medium for Flink task processing. Background technique [0002] At present, each TaskManager (task manager) of the underlying Flink framework is set to 1C and 2G by default. Users can adjust the number of Slots (task slots) and memory of the TaskManager through advanced parameters. For example, the current user sets concurrency mainly by setting multiple TaskManagers or setting multiple slots through advanced parameters. The main problem is that when users configure concurrency, they lack experience in resource allocation. They often solve the problem with a small amount of concurrency. As a result, the configuration is too large and the utilization rate of CPU and memory is low, resulting in waste of resources. Therefore, how to rationally adjust resources has become a problem to be solved. [0003] In ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5016G06F9/505
Inventor 范东汪金忠孙迁方伍元李成
Owner 南京苏宁电子信息技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com