Unlock instant, AI-driven research and patent intelligence for your innovation.

Consumption method and device of message queue

A message queue and message technology, applied in the field of communication, can solve the problems of dynamic adjustment, consumption failure, affecting the normal operation of the data processing system, etc., to improve the accuracy and avoid consumption failure.

Active Publication Date: 2020-07-10
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the inventor found in the process of implementing the present invention that the above-mentioned method in the prior art has at least the following defects: the constant consumption speed cannot be dynamically adjusted according to the remaining consumable amount in the message partition, so when a certain message When there is less message data in the partition, it is likely to lead to consumption failure, resulting in wrong data, and even affecting the normal operation of the entire data processing system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Consumption method and device of message queue
  • Consumption method and device of message queue
  • Consumption method and device of message queue

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0017] figure 1 A schematic flowchart of a message queue consumption method provided by Embodiment 1 of the present invention is shown. like figure 1 As shown, the method includes the following steps:

[0018] Step S110: Assign corresponding consumer terminals to each message partition in the message queue for consumption, and regularly obtain the current consumption offset and maximum effective offset of the message partition corresponding to each consumer terminal.

[0019] Among them, the number of consumers can be set according to the number of message partitions in the message queue. In one setting method, the number of consumers is equal to the number of message partitions in the message queue. At this time, the consumer and message partitions are One-to-one correspondence, correspondingly, one consumer is fixedly responsible for consuming messages in a corresponding message partition. This method can minimize mutual interference between multiple consumers and consumpt...

Embodiment 2

[0027] figure 2 A schematic flowchart of a message queue consumption method provided by Embodiment 2 of the present invention is shown. like figure 2 As shown, the method includes the following steps:

[0028] Step S210: Assign corresponding consumer terminals to each message partition in the message queue for consumption.

[0029] Specifically, the number of consumers can be set according to the number of message partitions in the message queue. In one setting method, the number of consumers is equal to the number of message partitions in the message queue. At this time, the consumer and the message partition Correspondingly, a consumer is fixedly responsible for consuming messages in a corresponding message partition. This method can avoid mutual interference between multiple consumers and consumption confusion to the greatest extent. In another setting method, the number of consumers is greater than the number of message partitions in the message queue. At this time, t...

example 1

[0041] Assuming that there are three message partitions and three consumer ends, the remaining consumable volume of message partition 1 corresponding to consumer end 1 is 100 (pieces), and the remaining consumption volume of message partition 2 corresponding to consumer end 2 is 200 (pieces). The remaining consumption volume of message partition 3 corresponding to consumer end 3 is 800 (messages), and the current consumption speed of the three consumers is 200 (messages / second).

[0042] Correspondingly, the minimum value of the remaining consumable volume of the message partition corresponding to each consumer is 100, that is, the minimum consumable volume is 100. In addition, since the consumption terminal corresponding to the minimum consumable quantity is consumption terminal 1, its consumption speed is 200 (pieces / second), therefore, the reference consumption speed is 200 (pieces / second), which is adjusted according to the minimum consumable quantity (100) When the refere...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a message queue consumption method and apparatus, which at least can solve the problem of a consumption failure phenomenon easily caused by a way of performing consumption at a constant consumption speed in the prior art. The message queue consumption method comprises the steps of allocating corresponding consumption ends to message partitions in a message queue for performing consumption, and regularly obtaining current consumption offsets and maximum effective offsets of the message partitions corresponding to the consumption ends; determining residual consumption quantities of the message partitions corresponding to the consumption ends according to the current consumption offsets and the maximum effective offsets of the message partitions corresponding to the consumption ends; and dynamically adjusting the consumption speeds of the consumption ends according to the residual consumption quantities of the message partitions corresponding to the consumption ends.

Description

technical field [0001] The invention relates to the field of communication technology, in particular to a message queue consumption method and device. Background technique [0002] At present, after a business front-end producer generates a message, it usually stores the message to be processed in the message partition of the message queue, and then sends it to the back-end consumer for processing. In the prior art, when each consumer end consumes the messages in the message partition, the consumption speed is often constant. [0003] However, the inventor found in the process of implementing the present invention that the above-mentioned method in the prior art has at least the following defects: the constant consumption speed cannot be dynamically adjusted according to the remaining consumable amount in the message partition, so when a certain message When there is less message data in the partition, it is likely to lead to consumption failure, resulting in wrong data, an...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/54
CPCG06F9/546G06F2209/548
Inventor 邓怡豪
Owner BEIJING QIHOO TECH CO LTD