Distributed storage system load balancing scheduling method and device and storage medium

A distributed storage and load balancing technology, applied in transmission systems, electrical components, etc., can solve the problems of insufficient resources, unbalanced load, and low accuracy of balanced scheduling, and achieve the effect of improving overall performance and accuracy.

Pending Publication Date: 2022-05-13
CHINA TELECOM CLOUD TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the existing method of using the weighted connection algorithm to achieve balance, only the actual number of connections is used as an important indicator for balanced scheduling, and the actual resources occupied by the connection are not considered, resulting in relatively low accuracy of balanced scheduling
[0005] The weight value of the weighted least connection algorithm used is often manually set by the administrator based on experience, and cannot be automatically adjusted dynamically according to business performance and cluster load
When the performance of some servers deteriorates or is replaced by servers with large performance gaps, the load will be unbalanced, which will lead to waste of resources or insufficient resources when the business traffic is high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage system load balancing scheduling method and device and storage medium
  • Distributed storage system load balancing scheduling method and device and storage medium
  • Distributed storage system load balancing scheduling method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0063] see figure 1 As shown, the embodiment of the present invention provides a distributed storage system load balancing scheduling method, the method includes the following steps:

[0064] Step S101, obtaining connection information of all storage nodes, and determining the actual number of connections of all storage nodes;

[0065] Step S102, using the obtained connection information to determine the duration of each connection on each storage node;

[0066] Step S103, for each storage node, optimize using the duration of each connection and the actual number of connections to obtain the optimized connection number of each storage node, wherein the optimized connection number is the same as the corresponding storage node's related to the current load situation;

[0067] Step S104, based on the number of optimized connections of each node and the current weight value corresponding to each node, select M storage nodes from all storage nodes as the target storage cluster to...

Embodiment 2

[0115] see image 3 As shown, the embodiment of the present invention provides a distributed storage system load balancing scheduling device, the device includes the following modules:

[0116] An acquisition module 31, configured to acquire connection information of all storage nodes, and determine the actual number of connections of all storage nodes;

[0117] A determining module 32, configured to determine the duration of each connection on each storage node by using the obtained connection information;

[0118] An optimization module 33, configured to optimize each storage node using the duration of each connection and the actual number of connections to obtain the optimized number of connections for each storage node, wherein the optimized number of connections is the same as the corresponding Related to the current load of the storage node;

[0119] The selection module 34 is used to select M storage nodes from all storage nodes as the target storage cluster to be sch...

Embodiment 3

[0132] The embodiment of the present invention also provides a computer device, which may be a computing device such as a desktop computer, a notebook computer, a palmtop computer, or a cloud server. Such as Figure 4 As shown, the device may include, but is not limited to, a processor and a memory, wherein the processor and the memory may be connected through a bus or in other ways.

[0133] The processor can be a central processing unit (Central Processing Unit, CPU) or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), graphics processing units (Graphics Processing Unit, GPU), embedded neural network processors (Neural-network Processing Unit, NPU) or other dedicated deep learning coprocessor, Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices , discrete gate or transistor logic devices, discrete hardware components and other chips,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a distributed storage system load balancing scheduling method, device and equipment and a storage medium, and the method comprises the steps: optimizing a calculation method of the number of connections in a load balancing scheduling algorithm, distinguishing long connections from short connections, improving a calculation algorithm of the number of long connections with large resource consumption according to the actually occupied resources, and calculating the number of short connections in the load balancing scheduling algorithm; the connection number in the weighted connection algorithm better fits the actual resource occupation condition; according to the method, the comprehensive weight in a load balancing algorithm is optimized, and the weighted value is adaptively learned and optimized through regression training, so that adaptive dynamic adjustment can be realized according to the service flow and the load condition of the node, and the problem of load unbalance caused by service flow jitter and node load fluctuation or fault is solved.

Description

technical field [0001] The present invention relates to the field of load technology, and in particular to a distributed storage system load balance scheduling method, device, equipment and storage medium. Background technique [0002] Load balancing scheduling is widely used in distributed storage systems. There are many nodes in a distributed system. When a node failure occurs in the system or the node load is too high, this will affect the performance of the entire distributed system. For example, when a node is seriously overloaded and the node goes down, it will affect the distributed system. overall performance of the system. Therefore, balanced scheduling is usually required to ensure the stability of the distributed system. [0003] The inventors have found that the weighted least connection algorithm is a widely used distributed load balancing method at present. But it has the following problems: [0004] There are long connections and short connections in the b...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L67/1097H04L67/1008H04L67/1029H04L67/1042H04L41/14H04L43/16
CPCH04L67/1097H04L67/1008H04L67/1046H04L67/1029H04L41/145H04L43/16
Inventor 吴文峰肖露林洁琬黄鹄刘汪洋颜文强
Owner CHINA TELECOM CLOUD TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap