Unlock instant, AI-driven research and patent intelligence for your innovation.

Message theme processing method and device, computer equipment and readable storage medium

A processing method and topic technology, applied in the processing method of message topics, computer equipment and readable storage media, and the field of devices, can solve problems such as unavailability of topic1 and topic2, unavailability of message topic consumption, and affecting normal consumption of other topics, etc. To achieve the effect of ensuring stable consumption and avoiding consumption

Pending Publication Date: 2021-04-06
平安消费金融有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the inventor found that when kafka currently provides consumer groups to enable multiple threads to consume one or more message topics at the same time, when one of the message topics has a problem and hangs up, it will trigger a balance and cause thread reallocation. In some unstable In the scenario, it is necessary to ensure the stable consumption of the program, and it is necessary to restart the suspended thread, so that another thread reassignment will occur, which will affect the normal consumption of other topics
If the suspended message topic is pulled up infinitely, the current consumption group will always be in balance, and the consumption of other message topics in the same group will be completely unavailable
For example, when a consumer group subscribes to three message topics topic1, topic2, and topic3 ​​at the same time, if topic3 ​​hangs up, threads will be reassigned to topic1 and topic2. When topic3 ​​is infinitely restarted, if the continuous restart fails, topic1 and topic2 will be caused while in an unavailable state

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Message theme processing method and device, computer equipment and readable storage medium
  • Message theme processing method and device, computer equipment and readable storage medium
  • Message theme processing method and device, computer equipment and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0027] Embodiment 1 of the present application provides a method for processing message topics. Through this method, interrupted message topics can be restarted infinitely in a new message group, avoiding repeated restarts in the original working consumption group and other message topics in the same consumption group. , and it can continue to consume from the breakpoint according to the offset in the offset group after restarting, specifically, figure 1 The flow chart of the message subject processing method provided by Embodiment 1 of this application, such as figure 1 As shown, the processing method of the message topic includes the following steps S101 to S103:

[0028] Step S101: Set an offset group corresponding to the message queue and at least two consumption groups.

[0029] Among them, the offset group is used to record the offset of each message topic in the message queue, and the consumption group includes several threads, which are used to consume the message top...

Embodiment 2

[0092] Corresponding to Embodiment 1 above, Embodiment 2 of the present application provides a device for processing message topics. For corresponding technical features and corresponding technical effects, refer to Embodiment 1 above, which will not be repeated here. figure 2 The block diagram of the device for processing the message subject provided by Embodiment 2 of this application, such as figure 2 As shown, the device includes: a first setting module 201 , a first distribution module 202 and a second distribution module 203 .

[0093] The first setting module 201 is configured to set an offset group corresponding to a message queue and at least two consumption groups, wherein the offset group is used to record the offset of each message topic in the message queue, and the The consumption group includes several threads for consuming the message topic and updating the offset; the first allocation module 202 is used for allocating the message topic in the message queue t...

Embodiment 3

[0102] Embodiment 3 also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including an independent server, Or a server cluster composed of multiple servers), etc. Such as image 3 As shown, the computer device 01 in this embodiment at least includes but is not limited to: a memory 012 and a processor 011 that can communicate with each other through a system bus, such as image 3 shown. It should be pointed out that, image 3 Only the computer device 01 is shown with components memory 012 and processor 011, but it should be understood that implementing all of the illustrated components is not a requirement and that more or fewer components may instead be implemented.

[0103] In this embodiment, the memory 012 (that is, a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of data processing, and provides a message theme processing method and device, computer equipment and a readable storage medium. The method comprises the following steps: setting an offset group and at least two consumption groups corresponding to a message queue; distributing the message topics in the message queue to a consumption group, wherein if the consumption group is distributed with the message topics, the consumption group is a working consumption group, and if the consumption group is not distributed with the message topics, the consumption group is a standby consumption group; when at least two message topics are allocated to the working consumption group and the message topics in the working consumption group are interrupted, allocating the interrupted message topics to the standby consumption group, so that the standby consumption group restarts the interrupted message topics according to the corresponding offsets in the offset group. Through the method and the device, the suspended message theme can be pulled up infinitely, so that the consumption of other message themes in the same message group is prevented from being influenced while the stable consumption of a program is ensured.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a message subject processing method, device, computer equipment and readable storage medium. Background technique [0002] Kafka is a distributed publish-subscribe messaging system, based on its high-throughput data processing capabilities, it is currently widely used. However, the inventor found that when kafka currently provides consumer groups to enable multiple threads to consume one or more message topics at the same time, when one of the message topics has a problem and hangs up, it will trigger a balance and cause thread reallocation. In some unstable In the scenario, it is necessary to ensure the stable consumption of the program, and it is necessary to restart the suspended thread, so that another thread reassignment will occur, which will affect the normal consumption of other topics. If the suspended message topic is pulled up infinitely, the cur...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50G06F9/54
CPCG06F9/5027G06F9/546
Inventor 王晓初赵宏军
Owner 平安消费金融有限公司