Message processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention discloses a message processing method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: in response to determining that a message queue meets a message receiving condition, storing the message queue into a persistent queue storage space according to a mapping relation between the message queue and the persistent queue storage space; when a message consumption instruction is received and it is determined that the target consumption message cannot be read from the memory, reading the target consumption message from the persistent queue storage space according to the message consumption instruction; and sending the target consumption message to the message consumer. According to the technical scheme, the message accumulation capacity is improved, consumption backtracking, off-line message reading, consumption mark deletion confirmation, delay message processing, a message ACK mechanism, stream computing system access and storage and computing separation are achieved, the system throughput is improved while the real-time performance of messages is not affected, and the processing requirement of the user on the message and the ever-increasing requirement on the storage expansibility of the system are met.

Description

technical field [0001] Embodiments of the present disclosure relate to the technical field of message processing, and in particular, to a message processing method, device, electronic equipment, storage medium, and program product. Background technique [0002] AMQP (Advanced Message Queuing Protocol, Advanced Message Queuing Protocol) is an application-level standard advanced message queuing protocol that provides unified message services and is designed for message-oriented middleware. Based on the AMQP protocol, point-to-point subscription of messages can be realized, such as RabbitMQ The solution is to send, route and receive messages through the AMQP protocol. However, the RabbitMQ solution has the following technical obstacles: when there are many node messages accumulated, the accumulated messages will quickly sink into a deep queue, which will increase the average cost of processing each message, thereby reducing the data processing capacity of the entire system. At...

AI Technical Summary

Problems solved by technology

However, the RabbitMQ solution has the following technical obstacles: when there are many node messages accumulated, the accumulated messages will quickly sink into a deep queue, which will increase the average cost of processing each message, thereby reducing the data processing capacity of the entire system. At the same time, because it takes more time and resources to process the accumulated messages, the incoming messages will also be accumulated and squeezed, resulting in a vicious cycle of system load and serious memory accumulation; since only the message queue processing mechanism is supported, it cannot be realized Confirmation of consumption backtracking and deletion of consumption marks, as well as processing of delayed messages, cannot meet the growing needs of users for processing messages themselves; due to its own accumulation capacity and architecture, it is difficult to support streams with high real-time requirements and high throughput The access of the computing system, therefore, cannot realize stream computing; although message persistence is supported, the persistence of messages depends more on the ability of local disks, which cannot meet the growing needs of users for system storage scalability

Images