Cluster computing method and device, and computer-readable storage medium

A computing method and technology for computing devices, which are applied in computing, multi-programming devices, program control design, etc., can solve problems such as competition for resources, increase or decrease of computing load, increase or decrease of available resources, etc., and achieve data expansion. Throughput efficiency, improve computing efficiency, and reduce the effect of resource occupancy

Inactive Publication Date: 2019-02-12
YI TAI FEI LIU INFORMATION TECH LLC
View PDF4 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The inventors have found that at least the following problems exist in the prior art: for a computing task, after the parallelism setting is completed, the parallelism remains constant until the task is executed (batch type) or the task is canceled (streaming type).
However, during the running of a computing task, the state of the cluster is not static. The computing load may rise or fall, and the available resources may increase or decrease. At this time, if the parallelism of the computing task remains unchanged from the initial state, there will be It may cause waste of resources, or compete for resources with other applications, resulting in unstable running status of computing tasks
Therefore, there is room for improvement in the operating efficiency of the existing system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cluster computing method and device, and computer-readable storage medium
  • Cluster computing method and device, and computer-readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, various implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

[0018] The first embodiment of the present invention relates to a cluster computing method, which can be applied to the processing of batch or streaming computing tasks of clusters such as SQL. This embodiment does not specifically limit the type of cluster. The cluster computing method includes: monitoring ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention relates to the distributed data processing technical field, and discloses a cluster computing method and device and a computer-readable storage medium. The cluster computing method comprises the following steps of: monitoring the running status of computing tasks on the cluster; determining whether the running state satisfies the condition of parallelism degree update; if the condition of parallelism degree update is satisfied, calculating the new parallelism degree according to the running state of the monitored computing task on the cluster; computing tasks onthe cluster with the new degree of parallelism. The embodiment of the invention improves the overall computing efficiency of the cluster by dynamically adjusting the parallelism degree.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of distributed data processing, and in particular to a cluster computing method, device, and computer-readable storage medium. Background technique [0002] With the rapid development of information technology, the amount of data generated and processed is increasing rapidly, and big data technology based on distributed cluster computing is increasingly being applied to various fields. Among them, SQL (Structured Query Language, Structured Query Language), as a data manipulation language with high popularity and low learning cost, has been integrated into more and more big data processing systems and used as a tool for computing logic construction. [0003] In the existing distributed big data computing system, whether it is a streaming or batch computing task, a certain degree of parallelism is usually set for the computing task during the construction of the computing logic, and the s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5022G06F9/5083
Inventor 唐剑飞
Owner YI TAI FEI LIU INFORMATION TECH LLC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap