High-availability message sending method and device based on RabbitMQ and medium
The RabbitMQ message forwarder is managed through the heartbeat detection and routing priority mechanism, which solves the problem of low message distribution efficiency in distributed systems, and achieves high availability and successful message delivery.
Patent Information
- Application Number
- CN202510394855.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-18
AI Technical Summary
The existing RabbitMQ static routing configuration is difficult to dynamically adjust in distributed systems, resulting in low message distribution efficiency and no message resend mechanism is set, which can easily lead to message loss or service blockage.
The health status of the message forwarder is detected regularly through the heartbeat detection mechanism, maintain the list of available message forwarders, combine the routing priority mechanism to select the optimal forwarder, and perform exception processing when message sending fails.
It realizes efficient message forwarding management, ensures that messages are successfully delivered to the receiving end, avoids message loss, meets the high availability requirements of the production environment, and has low development intrusion.
Smart Images
Figure CN120336042A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of message queues, and specifically relates to a high-availability message sending method, device, and medium based on RabbitMQ. Background Art
[0002] RabbitMQ is a message middleware written in the Erlang language, following the AMQP protocol, providing a stable and reliable message transmission service. It realizes message storage, forwarding, and routing through core components such as queues and exchanges, has high availability and scalability, supports clustering and failover, can handle a large number of concurrent connections, and is widely used in distributed systems.
[0003] When applying RabbitMQ in a distributed system, a fixed connection configuration is usually used. This static routing configuration is difficult to dynamically adjust the message routing path according to real-time load, network latency, or geographical distribution, resulting in low message distribution efficiency. Moreover, no message resending mechanism is set. After a message fails to be sent due to a fault, it is difficult to automatically switch to a faulty node and retry sending, easily leading to message loss or service blockage. Summary of the Invention
[0004] To solve the above problems, this application proposes a high-availability message sending method based on RabbitMQ, including:
[0005] Regularly detect the health status of each message forwarder in the message forwarder group within a preset time interval through a heartbeat detection mechanism;
[0006] Determine a list of available message forwarders in the message forwarder group according to the health status, store it in the memory cache, and update the list of available message forwarders based on the preset time period;
[0007] Receive the message sent by the application side, select the optimal message forwarder in the list of available message forwarders through a routing priority mechanism, and send the message to the receiving end through the optimal message forwarder;
[0008] Receive the message sending receipt and determine whether the message is sent successfully;
[0009] If not, determine that the optimal message forwarder is abnormal, and perform resending processing on the abnormal message through an exception handling mechanism.
[0010] On the other hand, this application also proposes a high-availability message sending device based on RabbitMQ, including:
[0011] At least one processor; and,
[0012] A memory communicatively connected to the at least one processor; wherein,
[0013] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a high-availability message sending method based on RabbitMQ as described in the above example.
[0014] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured as: a high-availability message sending method based on RabbitMQ as described in the above example.
[0015] By proposing a high-availability message sending method based on RabbitMQ in the present application, the following
[0016] beneficial effects can be brought:
[0017] By adding a RabbitMQ message forwarder group and scheduling and managing multiple RabbitMQ message forwarders, the rigid requirements for RabbitMQ high availability in the production environment are met, and the intrusion into development is relatively low. Development only needs to configure the RabbitMQ message forwarder group and can be implemented through the underlying framework, with low application transformation cost.
[0018] And by combining the message receipt mechanism, RabbitMQ can ensure that messages are successfully delivered to the receiving end. When message sending fails, the message can be resent through the exception handling mechanism, avoiding message loss. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:
[0020] Figure 1 is a schematic flowchart of a high-availability message sending method based on RabbitMQ in an embodiment of the present application;
[0021] Figure 2 is a schematic diagram of a high-availability message sending device based on RabbitMQ in an embodiment of the present application. DETAILED DESCRIPTION
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.
[0023] The following will, with reference to the drawings, elaborate on the technical solutions provided by each embodiment of this application.
[0024] As Figure 1 shown, an embodiment of this application provides a highly available message sending method based on RabbitMQ, including:
[0025] S101: Through a heartbeat detection mechanism, regularly detect the health status of each message forwarder in the message forwarder group at a preset time interval.
[0026] Specifically, based on the forwarder parameters, multiple RabbitMQ message forwarders are pre-configured to build a RabbitMQ message forwarder group, so that when a message is sent from the application system, a RabbitMQ message forwarder can be selected through the RabbitMQ message forwarder group, and the list of available RabbitMQ message forwarders is determined and cached. Through the heartbeat detection mechanism, regularly detect the health status of each message forwarder in the message forwarder group at a preset time interval to maintain the list of available message forwarders.
[0027] Among them, the core implementation method of the heartbeat mechanism is to regularly send a heartbeat signal to confirm whether the status of the message forwarder is normal.
[0028] Furthermore, based on preset heartbeat detection parameters, including detection interval, timeout duration, retry times, etc., a heartbeat detection request is generated and regularly sent to the message forwarder group.
[0029] In the embodiment of this application, the heartbeat detection request sending methods include HTTP requests, TCP connections, and internal message queues. Specifically, the HTTP request method sends a simple request to the HTTP endpoint of the message forwarder to detect whether a successful response can be obtained; the TCP connection method detects whether the message forwarder is still in a normal working state through a TCP connection; the internal message queue method enables the message forwarder to return a heartbeat response through a specific queue or message passing method.
[0030] Further, obtain the response results returned by each message forwarder in the message forwarder group, and determine the health status of each message forwarder according to the response results. For example, when a successful response is received, the HTTP 200 status code will be returned, or the TCP connection remains stable; when a failed response is received, a heartbeat timeout or an error (such as a 500 internal server error, etc.) is returned. The health status includes healthy, abnormal, and faulty.
[0031] Furthermore, respectively determine whether each message forwarder responds normally according to the response results. If there is a message forwarder that does not respond normally, such as a heartbeat request not being responded to or an error response being received, it is determined that the message forwarder is abnormal, and further determine whether the message forwarder has a fault. Obtain the corresponding abnormal message forwarder, based on the number of retry attempts, repeatedly send a heartbeat detection request to the abnormal message forwarder, and obtain the corresponding repeated response results. When the repeated response results do not include a normal response, it is determined that the abnormal message forwarder still has not returned to the normal state after multiple heartbeat detections, and it is determined that the abnormal message forwarder has a fault.
[0032] If the message forwarder responds normally, then determine whether the response duration is lower than the preset timeout duration. If it is lower, it is determined that the message forwarder is healthy; if it is not lower, it is determined that the message forwarder is abnormal, and further detect the abnormal message forwarder to determine whether there is a fault.
[0033] S102: Determine the list of available message forwarders in the message forwarder group according to the health status, store it in the memory cache, and update the list of available message forwarders based on the preset time period.
[0034] Specifically, add the healthy message forwarders to the list of available message forwarders, store them in the memory cache, and update the list of available message forwarders based on the preset time period. Automatically remove and warn the abnormal message forwarder nodes to achieve regular maintenance.
[0035] It should be noted that by regularly maintaining the list of available message forwarders, the fault tolerance of message forwarding is enhanced, ensuring that even if some forwarders fail, messages can still be normally forwarded to other healthy forwarders.
[0036] Moreover, to improve the response speed of message forwarding, the list of available message forwarders is stored in the memory cache. The memory cache can quickly access data, avoiding querying the database or other persistent storage every time a message needs to be forwarded, thereby reducing query latency and achieving efficient access.
[0037] S103: Receive the message sent by the application side, select the optimal message forwarder from the list of available message forwarders through the routing priority mechanism, and send the message to the receiving end through the optimal message forwarder.
[0038] Specifically, initialize the list of available message forwarders. When a message is sent from the application side, extract the attribute features of each available message forwarder in the available message forwarding list. Among them, the attribute features include distance feature, load feature, and latency feature. The distance feature is the distance between the message target region and the data center where the node is located; the load feature is the current load of the message forwarder; the latency feature is the network latency between the message forwarder and the receiving end.
[0039] Furthermore, based on the weight assignment corresponding to the attribute features, calculate the comprehensive weight of each available message forwarder, determine the priority of each available message forwarder according to the comprehensive weight, determine the optimal message forwarder based on the priority, and forward the message sent by the application side to the receiving end through the optimal message forwarder.
[0040] After the message forwarding is completed, continuously monitor the attribute status of the message forwarder. Based on the preset time interval, update the attribute features of each available message forwarder in the available message forwarding list. If the priority of the message forwarder changes (such as a higher load) in subsequent message forwarding requests, recalculate the priority of each available message forwarder according to the updated attribute features, and adjust the message forwarding strategy to select a new preferred forwarder to ensure that the message can reach the receiving end in the optimal way.
[0041] It should be noted that if the message is large, the message forwarder may need to fragment it and forward it through multiple network requests or protocols.
[0042] S104: Receive the message sending receipt and determine whether the message is sent successfully.
[0043] Specifically, after the message is sent, the receiving end returns a message sending receipt to the application side. Receive the message sending receipt and determine whether the message is sent successfully. When the message is sent successfully, a success receipt is returned, and when the message sending fails, a failure receipt is returned.
[0044] It should be noted that the receipt is a confirmation message, which contains the status of whether the message is successfully received and processed, as well as error information. The receipt can be synchronous or asynchronous. A synchronous receipt means that after sending the message, the sender waits for the receiving end to return the receipt before proceeding to the next step. An asynchronous receipt means that after sending the message, the sender does not need to wait for the receipt immediately and can continue to execute other tasks. The receipt will be returned later through a callback or event mechanism.
[0045] S105: If not, determine that the optimal message forwarder is abnormal, and re-send the abnormal message through the exception handling mechanism.
[0046] Specifically, when a message sending exception occurs, a new RabbitMQ message forwarder is replaced according to the exception handling mechanism to re-send the message.
[0047] Determine the exception type of message sending according to the message sending receipt. The exception types include connection - related exceptions, resource - related exceptions, and network interruptions.
[0048] Specifically, when the optimal message forwarder fails to connect successfully to the receiving end, or the connection duration exceeds the preset connection duration threshold, determine that the exception type is a connection - related exception. When the consumption of hardware resources exceeds the preset limit of the receiving end, determine that the exception type is a resource - related exception. When the optimal message forwarder and the receiving end are initially connected successfully and the message sending is interrupted, determine that the exception type is a network interruption.
[0049] It should be noted that connection - related exceptions usually involve problems that occur during the establishment, maintenance, and release of network connections, and may include database connections, message forwarder connections, client - server connections, etc. Resource - related exceptions are usually related to the exhaustion or insufficiency of the system's hardware resources (such as memory, CPU, disk, network bandwidth, etc.). Network interruptions usually indicate that there are problems with the network communication between servers, resulting in the inability to transmit messages normally.
[0050] When the exception type is a connection - related exception, trigger the exception rotation mechanism, re - send the abnormal message through the remaining available message forwarders in the available message forwarder list, and delete the optimal message forwarder with the exception. When the exception type is a resource - related exception, temporarily store the message in disk storage and delay the sending of the abnormal message. When the exception type is a network interruption, after determining network re - connection, re - send the abnormal message.
[0051] Among them, before passing through the remaining available message forwarders in the list of available message forwarders, it is judged whether the number of remaining available message forwarders in the list of available message forwarders is greater than 0, that is, it is judged whether there is an available RabbitMQ message forwarder in the cache. When there is an available RabbitMQ message forwarder in the cache, a RabbitMQ message forwarder is selected as the new RabbitMQ message forwarder, and the RabbitMQ message forwarder with an exception is deleted. The message is resent to the corresponding channel by using the new RabbitMQ message forwarder. When there is no available RabbitMQ message forwarder in the cache, a RabbitMQ message forwarder is selected from the multiple configured RabbitMQ message forwarders in the RabbitMQ message forwarder group as the new RabbitMQ message forwarder, and the message is tried to be resent to the corresponding channel by using the new RabbitMQ message forwarder until the message is successfully sent, and the new RabbitMQ message forwarder is added to the list of available RabbitMQ message forwarders and cached.
[0052] By adding a RabbitMQ message forwarder group, scheduling and management of multiple RabbitMQ message forwarders are carried out, which meets the rigid requirements for RabbitMQ high availability in the production environment, and the intrusion into development is relatively low. Development only needs to configure the RabbitMQ message forwarder group and is implemented in the way of the underlying framework, and the transformation cost of the application is low.
[0053] And by combining the message receipt mechanism, RabbitMQ can ensure that the message is successfully delivered to the receiving end. When the message sending fails, the message can be resent through the exception handling mechanism, avoiding message loss.
[0054] Such as Figure 2 As shown, the embodiment of the present application also proposes a high-availability message sending device based on RabbitMQ, including:
[0055] At least one processor; and,
[0056] A memory communicatively connected to the at least one processor; wherein,
[0057] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a high-availability message sending method based on RabbitMQ as described in any one of the above embodiments.
[0058] The embodiments of the present application also provide a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured as: a high-availability message sending method based on RabbitMQ according to any one of the above embodiments.
[0059] The various embodiments in the present application are all described in a progressive manner. For the same or similar parts among the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.
[0060] The devices and media provided by the embodiments of the present application correspond one by one to the methods. Therefore, the devices and media also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be elaborated here.
[0061] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0062] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be realized by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.
[0063] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including instruction means, and the instruction means realizes the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.
[0064] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0065] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0066] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0067] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0068] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0069] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.
Claims
1. A highly available message sending method based on RabbitMQ, characterized in that, Including: Through a heartbeat detection mechanism, regularly detect the health status of each message forwarder in the message forwarder group at a preset time interval; According to the health status, determine the list of available message forwarders in the message forwarder group, store it in the memory cache, and update the list of available message forwarders based on the preset time period; Receive a message sent by the application side, select the optimal message forwarder from the list of available message forwarders through a routing priority mechanism, and send the message to the receiving end through the optimal message forwarder; Receive a message sending receipt and determine whether the message is successfully sent; If not, determine that the optimal message forwarder is abnormal, and perform re-sending processing on the abnormal message through an exception handling mechanism.
2. The high-availability message sending method based on RabbitMQ according to claim 1, characterized in that, The step of detecting the health status of each message forwarder in the message forwarder group through the heartbeat detection mechanism at a preset time period specifically includes: Based on the forwarder parameters, pre-configure multiple message forwarders to construct a message forwarder group; Generate a heartbeat detection request based on preset heartbeat detection parameters and regularly send it to the message forwarder group at a preset time interval; Obtain the response results returned by each message forwarder in the message forwarder group, and determine the health status of each message forwarder according to the response results.
3. The high-availability message sending method based on RabbitMQ according to claim 2, wherein The step of determining the health status of each message forwarder according to the response results specifically includes: Respectively judge whether each message forwarder responds normally according to the response results; If not, determine that the message forwarder is abnormal, obtain the corresponding abnormal message forwarder, repeatedly send the heartbeat detection request to the abnormal message forwarder based on the number of retries, and obtain the corresponding repeated response results; When the repeated response results do not include a normal response, determine that the abnormal message forwarder has failed; If so, judge whether the response duration is lower than the preset timeout duration; If so, determine that the message forwarder is healthy; If not, determine that the message forwarder is abnormal.
4. A high-availability message sending method based on RabbitMQ according to claim 1, characterized in that The step of selecting the optimal message forwarder from the list of available message forwarders through the routing priority mechanism specifically includes: Extract the attribute features of each available message forwarder in the available message forwarding list; the attribute features include distance features, load features, and delay features; Based on the weight allocation corresponding to the attribute features, calculate the comprehensive weight of each available message forwarder, determine the priority of each available message forwarder according to the comprehensive weight, and determine the optimal message forwarder based on the priority.
5. A high-availability message sending method based on RabbitMQ according to claim 4, characterized in that, After determining the priority of each available message forwarder according to the comprehensive weight and determining the optimal message forwarder based on the priority, the method further includes: Update the attribute features of each available message forwarder in the available message forwarding list within the preset time interval; Update the priority of each available message forwarder according to the updated attribute features.
6. A high-availability message sending method based on RabbitMQ according to claim 1, characterized in that, The step of performing re-sending processing on the abnormal message through the exception handling mechanism specifically includes: Determine the abnormal type of message sending according to the message sending receipt; the abnormal types include connection - related abnormalities, resource - related abnormalities, and network interruption; When the abnormal type is a connection - related abnormality, trigger the abnormal rotation mechanism, resend the abnormal message through the remaining available message forwarders in the available message forwarder list, and delete the optimal message forwarder for the abnormality; When the abnormal type is a resource - related abnormality, temporarily store the message in disk storage and delay the sending of the abnormal message; When the abnormal type is network interruption, after determining network reconnection, resend the abnormal message.
7. A high-availability message sending method based on RabbitMQ according to claim 6, characterized in that Before resending through the remaining available message forwarders in the available message forwarder list, the method further includes: Judge whether the number of remaining available message forwarders in the available message forwarder list is greater than 0; If not, obtain the historical available message forwarders in the message forwarder group and resend the abnormal message through the historical available message forwarders.
8. A high-availability message sending method based on RabbitMQ according to claim 6, characterized in that, The determining the abnormal type of message sending according to the message sending receipt specifically includes: When the optimal message forwarder fails to connect successfully with the receiving end or the connection duration exceeds the preset connection duration threshold, determine that the abnormal type is a connection - related abnormality; When the consumption of hardware resources exceeds the preset limit of the receiving end, determine that the abnormal type is a resource - related abnormality; When the optimal message forwarder and the receiving end are initially connected successfully and the message sending is interrupted, determine that the abnormal type is network interruption.
9. A highly available message sending device based on RabbitMQ, characterized in that, It includes: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a high - availability message sending method based on RabbitMQ as described in any one of claims 1 to 8.
10. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer - executable instructions are set to a high - availability message sending method based on RabbitMQ as described in any one of claims 1 to 8.