Adaptive task scheduling method and system and retrieval method comprising adaptive task scheduling method

A task scheduling and self-adaptive technology, applied in the field of big data processing, to achieve the effect of realizing performance and solving relative imbalance problems

Pending Publication Date: 2020-12-11
BEIJING XUEZHITU NETWORK TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present application provides an adaptive task scheduling method, system and distributed retrieval method to at least solve the problem of task scheduling in the related art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Adaptive task scheduling method and system and retrieval method comprising adaptive task scheduling method
  • Adaptive task scheduling method and system and retrieval method comprising adaptive task scheduling method
  • Adaptive task scheduling method and system and retrieval method comprising adaptive task scheduling method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0052]Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present application, and those skilled in the art can also apply the present application to other similar scenarios. In addition, it can also be understood that although such development efforts may be complex and lengthy, for those of o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of big data processing, in particular to an adaptive task scheduling method and system and a retrieval method comprising the adaptive task scheduling method. The adaptive task scheduling method comprises the steps that according to the resource utilization condition and the load change condition of each working node, the working nodes adjust the weights of the nodes every certain period; and a task scheduling node reads the weight condition of each working node, ranks the weights, and performs task scheduling according to the rank of the weights. According tothe method, the cluster heterogeneity and the current resource remaining situation of the working nodes are considered, the problem of relative imbalance of calculation task allocation of the workingnodes is solved, the overall calculation capability of the system is improved, the overall task completion time is shortened, and the performance is improved.

Description

technical field [0001] The present application relates to the field of big data processing, in particular to an adaptive task scheduling method, system and retrieval method including the method. Background technique [0002] With the upgrading of equipment in major data centers, supercomputing centers, and Internet companies and the introduction of high-performance components (such as GPUs, etc.), each node in the cluster has gradually become heterogeneous. Different aspects of performance lead to differences in their processing capabilities. As a result, there are large differences in the comprehensive computing capabilities of each node, and the entire cluster is in a state of unbalanced resources. Due to the different capabilities of each node in the cluster, assigning the same task to different nodes will have different impacts on the node load. [0003] Spark's default task scheduling is that the spark task scheduler randomly selects some nodes from the registered wor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50G06F16/2458G06F16/31G06F16/33G06F40/284
CPCG06F9/5038G06F9/5044G06F9/505G06F9/5083G06F16/2471G06F16/325G06F16/3344G06F40/284
Inventor 汪月刘建辉乔智孙军锋李亚飞
Owner BEIJING XUEZHITU NETWORK TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap