Local message queue implementation system and method and storage medium

Abstract

The invention belongs to the technical field of message queues, and particularly discloses a local message queue implementation system and method and a storage medium, message production and message consumption capabilities are provided for an upper-layer application through a user layer, a producer and a consumer are enabled to interact with a bus, the producer sends a message to the bus, the bus transmits the message to a Broker, and the Broker transmits the message to the Broker. The Broker processes the message and then distributes the message consumption event to the consumer through the bus, the Broker can be responsible for receiving, distributing and storing management of the message, the storage layer provides the storage capacity of the message, the bus bears the transmission work of all the message events of the message queue, the producer, the consumer and the Broker are decoupled, the asynchronous execution capacity is provided, and the message queue can be used for receiving and distributing the message. Therefore, the whole message queue architecture is loosely coupled, and the expansibility and robustness are improved. According to the method, the functions of traffic peak clipping, asynchronous processing, module decoupling and the like can be realized by a single service of a single application or a distributed application through the message queue.

Description

Technical Field

[0001] The present invention belongs to the technical field of message queues, and particularly relates to a local message queue implementation system, method, and storage medium. Background Art

[0002] With the continuous development of message queue technology, message queues have been increasingly widely used in distributed systems. In distributed systems, message queues are usually applied to scenarios such as traffic peak shaving, service decoupling, and system communication. However, in a single application or a single service of a distributed application, capabilities such as traffic peak shaving, asynchronous processing, and module decoupling are often also required. If an ordinary message queue is used to implement the above capabilities, the complexity of the program will be greatly increased, and the corresponding network IO overhead will also reduce the program performance. Summary of the Invention

[0003] The purpose of the present invention is to provide a local message queue implementation system, method, and storage medium to solve the above problems existing in the prior art.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] In a first aspect, a local message queue implementation system is provided, including a user layer, a bus, and a Broker layer deployed locally. The user layer includes a producer and a consumer, where:

[0006] The producer is used to send a message to the bus and send a message sending event to the bus after the message is successfully sent;

[0007] The consumer is used to subscribe to receive a message consumption event sent by the bus and consume the message according to the message consumption event;

[0008] The bus is used to send a message to the Broker layer, synchronously send a message sending event to the Broker layer, receive a message consumption event sent by the Broker layer, and send the message consumption event to the consumer;

[0009] The Broker layer is used to subscribe to receive a message sending event sent by the bus, receive the message sent by the bus, save the message, and then send a message consumption event to the bus.

[0010] In a possible design, the Broker layer is further used to send a message receipt event to the bus after receiving the message, the bus is further used to send a message receipt event to the producer, and the producer is further used to receive the message receipt event sent by the bus.

[0011] In a possible design, the consumer is also used to send a consumption completion event to the bus after consuming a message, and the bus is also used to send the consumption completion event to the Broker layer. The Broker layer is also used to subscribe to and receive the consumption completion event sent by the bus, and update the consumption status of the saved message according to the consumption completion event.

[0012] In a possible design, the Broker layer includes a message receiver and a message dispatcher. The message receiver is used to receive messages and message sending events from the bus, and the message dispatcher is used to send message consumption events to the bus.

[0013] In a possible design, the system further includes a storage layer, and the Broker layer is used to save messages to the storage layer.

[0014] In a possible design, the storage layer includes a logic layer and a physical layer. The logic layer is used to provide a storage API, and the physical layer is used to provide storage space. The Broker layer is used to save messages to the storage space by calling the storage API.

[0015] In a possible design, the producer is provided with a message sending application API, and the consumer is provided with a message consumption application API.

[0016] In a second aspect, a method for implementing a local message queue is provided, which is applied to the Broker layer in the first aspect and includes:

[0017] Receiving a message sending event transmitted by the producer via the bus, and receiving the message transmitted by the producer via the bus;

[0018] Saving the message, generating a message consumption event, and transmitting the message consumption event to the consumer via the bus, so that the consumer consumes the message after receiving the message consumption event.

[0019] In a possible design, the method further includes:

[0020] Receiving a consumption completion event transmitted by the consumer via the bus, and updating the consumption status of the saved message according to the consumption completion event.

[0021] In a third aspect, a computer-readable storage medium is provided. Instructions are stored on the computer-readable storage medium. When the instructions run on a computer, the computer is made to execute the method for implementing a local message queue in the second aspect. At the same time, a computer program product containing the instructions is also provided. When the instructions run on a computer, the computer is made to execute the method for implementing a local message queue in the second aspect.

[0022] Beneficial effects: The present invention provides the capabilities of message production and message consumption for upper-layer applications through the user layer, enabling producers and consumers to interact with the bus. Producers send messages to the bus, the bus delivers the messages to the Broker, and after the Broker processes them, it distributes message consumption events to consumers through the bus. The Broker is responsible for message reception, distribution, and storage management. The storage layer provides the storage capacity for messages. The bus undertakes the transfer work of all message events in the message queue, decouples the producer, consumer, and Broker, and provides the ability for asynchronous execution, making the entire message queue architecture loosely coupled and improving scalability and robustness. The present invention enables a single application or a single service of a distributed application to implement functions such as traffic peak shaving, asynchronous processing, and module decoupling through the message queue. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0024] Figure 1 It is a schematic diagram of the architecture of the system in Embodiment 1 of the present invention. Detailed Embodiments

[0025] It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation to the present invention. The specific structural and functional details disclosed herein are only used to describe the exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms and should not be construed as limited to the embodiments described herein.

[0026] It should be understood that unless otherwise clearly defined and limited, the corresponding terms should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be an electrical connection, a direct connection, or an indirect connection through an intermediate medium, and can be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments can be understood according to specific situations.

[0027] Specific details are provided in the following description to facilitate a complete understanding of the exemplary embodiments. However, those of ordinary skill in the art should understand that the exemplary embodiments can be implemented without these specific details. For example, the apparatus may be shown in block diagrams to avoid obscuring the examples with unnecessary details. In other embodiments, well-known processes, structures, and techniques may not be shown with unnecessary details to avoid obscuring the embodiments.

[0028] Embodiment 1:

[0029] This embodiment provides a local message queue implementation system, as Figure 1 shown. The system includes a user layer, a bus, and a Broker (middleware) layer deployed locally. The user layer includes producers and consumers, where:

[0030] The producer is used to send messages to the bus and send a message sending event to the bus after the message is successfully sent.

[0031] The consumer is used to subscribe to receive the message consumption event sent by the bus and consume messages according to the message consumption event.

[0032] The bus is used to send messages to the Broker layer, synchronously send a message sending event to the Broker layer, receive the message consumption event sent by the Broker layer, and send the message consumption event to the consumer.

[0033] The Broker layer is used to subscribe to receive the message sending event sent by the bus, receive the message sent by the bus, save the message, and then send a message consumption event to the bus.

[0034] Furthermore, the Broker layer is also used to send a message receipt event to the bus after receiving a message. The bus is also used to send the message receipt event to the producer, and the producer is also used to receive the message receipt event sent by the bus. The consumer is also used to send a consumption completion event to the bus after consuming a message. The bus is also used to send the consumption completion event to the Broker layer, and the Broker layer is also used to subscribe to and receive the consumption completion event sent by the bus, and update the consumption status of the saved message according to the consumption completion event. The bus stores the subscription information of event subscribers such as consumers and Brokers. All event subscribers need to subscribe to events from the bus to receive message events in the bus. The bus provides synchronous delivery and asynchronous delivery capabilities. When delivering messages, the bus will find the subscribers who have subscribed to the message event from the subscription information and deliver the message to them. The Broker layer includes a message receiver and a message dispatcher. The message receiver is used to receive messages and message sending events from the bus, and the message dispatcher is used to send message consumption events to the bus. That is, the message receiver will subscribe to the message sending event from the bus to receive messages sent by the producer. When it receives a message, it will first save the message, and then use the message dispatcher to deliver a message consumption event to notify the consumer to consume the message. When receiving a message consumption receipt event, it will update the consumption status of the message.

[0035] The producer is provided with a message sending application API (Application Programming Interface), and the consumer is provided with a message consumption application API. Through the message sending application API and the message consumption application API, the upper-layer application can be provided with the capabilities of message production and message consumption. The upper-layer application can use the API to complete message sending and consumption. The consumer will subscribe to the message consumption event from the bus. The upper-layer application needs to register a message listener with the consumer to consume messages. When the consumer receives a message consumption event from the bus, it will determine whether the message is the concerned topic or tag. If so, it can call the message listener for asynchronous consumption, and after consumption is completed, it will deliver a message consumption receipt event to the bus.

[0036] Furthermore, the system also includes a storage layer, and the Broker layer is used to save messages to the storage layer. The storage layer includes a logical layer and a physical layer. The logical layer is used to provide a storage API, and the physical layer is used to provide storage space. The Broker layer is used to save messages to the storage space by calling the storage API. The physical layer can have multiple implementations, which can be implemented by memory, files, databases, etc., depending on the application scenario. The storage layer shields the storage implementation details upward, and can flexibly select and switch the storage media used. After the storage layer completes message storage, it can feedback the storage result to the Broker through the storage API.

[0037] After the local deployment of this system, a single service of a monomer application or a distributed application can implement functions such as traffic peak shaving, asynchronous processing, and module decoupling through a message queue.

[0038] Embodiment 2:

[0039] This embodiment provides a method for implementing a local message queue, which is applied to the Broker layer in Embodiment 1. The method includes:

[0040] Receiving a message sending event transmitted by a producer via a bus, and receiving a message transmitted by the producer via the bus;

[0041] Saving the message, generating a message consumption event, and transmitting the message consumption event to a consumer via the bus, so that the consumer consumes the message after receiving the message consumption event.

[0042] Further, the method further includes:

[0043] Receiving a consumption completion event transmitted by a consumer via the bus, and updating the consumption status of the saved message according to the consumption completion event.

[0044] Embodiment 3:

[0045] This embodiment provides a local message queue implementation device, which locally deploys the user layer, bus, Broker layer, and storage layer in Embodiment 1. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. The storage layer includes a memory, which can but is not limited to include a random access memory (RAM), a read only memory (ROM), a flash memory, a first input first output (FIFO) memory, and / or a first in last out (FILO) memory, etc. The Broker layer is deployed in a corresponding processor, and the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0046] Embodiment 4:

[0047] This embodiment provides a computer-readable storage medium, on which instructions are stored. When the instructions run on a computer, the computer is enabled to execute the local message queue implementation method in Embodiment 2. Among them, the computer-readable storage medium refers to a carrier for storing data, which can but is not limited to include a floppy disk, an optical disc, a hard disk, a flash memory, a USB flash drive, and / or a memory stick, etc. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0048] This embodiment also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the local message queue implementation method in Embodiment 2. Among them, the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0049] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A local message queue implementation system, characterized in that: It includes a locally deployed user layer, bus and Broker layer, wherein the user layer includes producers and consumers, wherein: Producer, used to send messages to the bus, and send a message sending event to the bus after the message is successfully sent; Consumers are used to subscribe to message consumption events sent by the receiving bus and consume messages according to the message consumption events; The bus is used to send messages to the Broker layer, send message sending events to the Broker layer synchronously, receive message consumption events sent by the Broker layer, and send message consumption events to consumers; The Broker layer is used to subscribe to the message sending events sent by the bus, receive the messages sent by the bus, save the messages, and then send message consumption events to the bus.

2. A local message queue implementation system according to claim 1, characterized in that: The Broker layer is also used to send a message receipt event to the bus after receiving the message. The bus is also used to send a message receipt event to the producer. The producer is also used to receive the message receipt event sent by the bus.

3. A local message queue implementation system according to claim 1, characterized in that: The consumer is also used to send a consumption completion event to the bus after consuming the message. The bus is also used to send a consumption completion event to the Broker layer. The Broker layer is also used to subscribe to the consumption completion event sent by the bus and update the consumption status of the saved message according to the consumption completion event.

4. A local message queue implementation system according to claim 1, characterized in that: The Broker layer includes a message receiver and a message distributor. The message receiver is used to receive messages and message sending events from the bus, and the message distributor is used to send message consumption events to the bus.

5. A local message queue implementation system according to claim 1, characterized in that: The system further includes a storage layer, and the Broker layer is used to save the message to the storage layer.

6. A local message queue implementation system according to claim 5, characterized in that: The storage layer includes a logical layer and a physical layer, the logical layer is used to provide a storage API, the physical layer is used to provide a storage space, and the Broker layer is used to save messages to the storage space by calling the storage API.

7. A local message queue implementation system according to claim 1, characterized in that: The producer is provided with a message sending application API, and the consumer is provided with a message consuming application API.

8. A local message queue implementation method, applied to the Broker layer described in any one of claims 1-6, characterized in that: include: Receive message sending events transmitted by producers via the bus, and receive messages transmitted by producers via the bus; The message is saved and a message consumption event is generated, which is transmitted to the consumer through the bus so that the consumer consumes the message after receiving the message consumption event.

9. A local message queue implementation method according to claim 8, characterized in that: The method further comprises: Receive the consumption completion event transmitted by the consumer via the bus, and update the consumption status of the saved message according to the consumption completion event.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the local message queue implementation method of claim 8 or 9.

Images