Message processing method and device, and electronic device

Abstract

Embodiments of the present invention provide a message processing method and device, and electronic equipment, wherein the message processing method includes: performing a consumption operation on messages in the first message queue to be processed, and forming a consumption progress of the first message queue Information: broadcasting the consumption progress information in the network, so that other consumers in the network can locally save the consumption progress of the messages in the first message queue. The solutions of the embodiments of the present invention enable each consumer in the network to save the consumption progress information of other consumers, thereby effectively avoiding repeated consumption of the same message.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a message processing method and device and electronic equipment. Background technique [0002] Most of today's network service platforms (such as shopping, payment, etc.) use a distributed message system (such as Kafka, a high-throughput distributed publish-subscribe message system) as the basic message facility of the platform to support the platform The development of many online businesses in China and the processing of hundreds of millions of messages. In an existing distributed message system, it usually includes a message producer (Producer) responsible for generating messages and sending them to the message server, a message server (Broker) responsible for organizing and coordinating messages, and a message server (Broker) responsible for receiving messages from the message server. The message consumer (Consumer) that consumes the message. [0003] In the proce...

AI Technical Summary

Problems solved by technology

[0003] In the process of the existing message consumer consuming the message, usually the message consumer first records the information of the consumer offset (Consumer Offset) of the message queue, and then sends the consumer offset information to the message server for further processing. Save (also called asynchronous save), in the process, if the message consumer encounters a sudden failure (for example, downtime, etc.) or the network is not smooth when sending the consumption site information to the message server, Some consumption site information may not be sent and saved to the message server in time, which may lead to repeated consumption of messages by other message consumers

In practical applications, the repeated consumption of messages is likely to cause situations such as repeated notifications corresponding to a certain message and repeated deductions for orders corresponding to a certain message, which not only affects user experience, but may also cause economic losses to users.

[0004] Although in the prior art there is also the use of setting up the database to avoid repeated consumption of messages by querying the database, etc., there is still a problem that the implementation process depends on the database, and frequent query operations will cause system resource loss and low efficiency. In addition, due to It is necessary to set up an additional database, which will also cause a more complex system structure, which will increase the cost of the system

Images