Message queue load balancing method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a message queue load balancing method and device, electronic equipment and a medium, and relates to the technical field of computers, the method comprises the steps that the number of messages to be consumed and the consumption speed of each message queue are collected, and the consumption speed is the consumption speed of the messages in the message queues; according to the number of the to-be-consumed messages in each message queue and the consumption speed, determining the average duration required for consuming all the to-be-consumed messages in each current message queue; under the condition that the to-be-consumed messages exceeding the first preset proportion are located in the message queues smaller than a second preset proportion and the average duration is larger than a preset duration threshold value, load balancing is carried out on all the message queues; the difference value between the first preset proportion and the second preset proportion is greater than a specified threshold value. Through load balancing of the message queues, the situation that a large number of backlog messages appear in the message queues subscribed by consumers is avoided. The overstocked messages are processed in time, and the service timeliness is ensured.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a message queue load balancing method, device, electronic equipment and media. Background technique [0002] RocketMQ is an open source distributed messaging system, based on highly available distributed cluster technology, providing low-latency, highly reliable message publishing and subscription services. Each topic (topic) of RocketMQ includes multiple message queues. The name server (NameServer) in RocketMQ can configure the subscription relationship between consumers included in the same consumer group and message queues included in a topic based on the principle of load balancing. That is, different consumers can consume messages in different message queues, and the number of message queues corresponding to each consumer is relatively balanced. [0003] In related technologies, only when the number of consumers in the same consumer group changes, the load balancing opera...

AI Technical Summary

Problems solved by technology

However, in some business scenarios, there may be unbalanced message loads in different message queues

For example, each message queue includes messages with device lists. Some messages carry only one piece of device information, and some messages carry multiple pieces of device information. As a result, although the number of message queues subscribed by different consumers is the same, the actual processing speed is different. There is a big difference, some consumers have very low load, and there is a large backlog of messages in the message queue subscribed by other consumers

The backlog of messages cannot be processed in time, affecting business timeliness

Images