Method for effectively solving containerized rabbitmq non-stop capacity expansion

Abstract

The invention discloses a method for effectively solving non-stop capacity expansion of containerized RabbitMQ. The method comprises the following steps: S1, starting a mirror image strategy and setting an existing queue as a mirror image queue; s2, starting a mirror image queue priority mode and a message priority mechanism; s3, verifying the state of the mirror image queue; and S4, carrying out capacity expansion on the RabbitMQ and rebalancing the distribution of the main node. According to the method, the RabbitMQ middleware in a single-point and cluster mode can be subjected to non-stop capacity expansion, the problem that restarting is needed when the RabbitMQ service deployed in a containerization mode modifies resource configuration and operating parameters is solved through a mirror image mode, and particularly for a production environment, the phenomenon that the service is influenced by stop is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of software operation and maintenance, and in particular to a method for effectively solving the non-stop capacity expansion of containerized RabbitMQ. Background Art

[0002] With the development of cloud native technology, RabbitMQ, as a widely used message middleware, has a growing demand for deployment in containerized environments. Containerized deployment can achieve flexible resource allocation, but it often faces service interruptions and performance degradation during capacity expansion. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a method for effectively solving the non-stop expansion of containerized RabbitMQ, comprising the following steps: S1. enabling the mirroring strategy and setting the existing queue as a mirrored queue; S2. enabling the mirrored queue priority mode and the message priority mechanism; S3. verifying the mirrored queue status; S4. expanding the capacity of RabbitMQ and rebalancing the master node allocation.

[0004] Furthermore, in the step S1, enabling the mirroring strategy specifically includes: mirroring the queue on multiple nodes by setting ha-mode, and setting queue synchronization by ha-sync-mode.

[0005] Furthermore, the S3 step specifically includes: verifying through a command whether the queue is successfully set as a mirror queue.

[0006] Furthermore, in the S3 step, there is another way to verify the state of the mirror queue, specifically: checking whether the mirror queue is successfully set up by enabling the manager plug-in of rabbitmq.

[0007] Furthermore, the method also includes a mirror cancellation step: when performing performance optimization or testing, closing or clearing the mirror strategy.

[0008] The present invention provides a method for effectively solving the problem of non-stop expansion of containerized RabbitMQ, which has the following beneficial effects: 1. High availability: Mirrored queues replicate the contents of the queue to multiple nodes in the cluster. Even if a node fails, other nodes can still seamlessly take over the message processing of the queue to ensure that the system runs without interruption. This high availability design is very suitable for production environments that require high service continuity.

[0009] 2. Fault tolerance: If the primary node where the queue is located fails, RabbitMQ will automatically promote the queue to another mirror node to ensure that messages are not lost or cannot be processed. This automatic failover mechanism can greatly reduce downtime and the need for manual intervention.

[0010] 3. Data persistence: Mirrored queues replicate messages to multiple nodes, providing data redundancy. When a node is lost or data is corrupted, data can be restored from copies on other nodes to avoid data loss. This redundancy ensures the data integrity of the system.

[0011] 4. Seamless rolling upgrade and expansion: When using mirrored queues, you can perform seamless rolling upgrades or expansions on the RabbitMQ cluster. When a node is upgraded or expanded, other mirrored nodes will automatically take over its message processing work, ensuring that the cluster will not be interrupted during the expansion and upgrade process.

[0012] 5. Cluster load balancing: By properly configuring mirror queues, the master copies of the queues can be distributed on different nodes to balance the cluster load and prevent some nodes from being overloaded. By rebalancing the master node allocation, cluster performance can be further optimized and the overall throughput of the system can be improved.

[0013] 6. Message synchronization mode selection: RabbitMQ mirror queue supports synchronous mode and asynchronous mode. Synchronous mode can ensure that all messages are copied to the mirror node before being processed by the primary node, providing higher data consistency. Asynchronous mode can reduce the performance overhead caused by synchronization and flexibly select the synchronization strategy according to application requirements.

[0014] 7. Fast fault recovery: The mirror node in the mirror queue already holds a copy of the queue. When the primary node fails, the mirror node can be immediately promoted to the primary node and continue to process client requests. This fast switching mechanism shortens the system recovery time and improves the system availability.

[0015] 8. Simplified operation and maintenance: Mirror queues make cluster node maintenance easier. Because node upgrades, restarts, and resource expansion can be performed without interrupting services, cluster operation and maintenance are more efficient and reduce the risk during maintenance. In the process of non-stop expansion, the message priority mechanism ensures that high-priority messages are processed first, preventing low-priority messages from blocking the execution of critical tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0017] Figure 1 A flow chart of the method provided by the present invention; Figure 2 A schematic diagram of the mirror queue structure provided by the present invention. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0019] The following is a detailed description of the implementation method of the present invention in conjunction with the accompanying drawings. Only some embodiments are described, not all embodiments. For the purpose of clarity, representations and descriptions that are not related to the present invention are omitted in the drawings and descriptions.

[0020] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of the embodiments, and cannot be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0021] like Figure 1 As shown, the present invention provides a method for effectively solving the non-stop expansion of containerized RabbitMQ, comprising the following steps: S1. enabling the mirroring strategy and setting the existing queue as the mirroring queue; S2. enabling the mirroring queue priority mode and the message priority mechanism; S3. verifying the mirroring queue status; S4. expanding the capacity of RabbitMQ and rebalancing the master node allocation.

[0022] The mirroring strategy in step S1 is to mirror the queue on multiple nodes by setting ha-mode, and set queue synchronization by ha-sync-mode. RabbitMQ supports multiple mirroring queue strategies. By setting ha-mode, the queue can be mirrored on multiple nodes. The "all" mode of ha-mode means that the queue will be mirrored to all nodes in the cluster. ha-sync-mode is used to set the synchronization mode of the queue. "automatic" means automatic synchronization. When a new node joins the cluster, the queue data will be automatically synchronized to the node.

[0023] The specific implementation commands are: rabbitmqctl set_policy -p / ha-all-queue "^" '{"ha-mode":"all","ha-sync-mode":"automatic"}'; or: rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}' --priority 0 --apply-to queues.

[0024] For some specific important queues, manually set them as mirror queues to enhance their availability and ensure that when a node fails, the messages in the queue can still be processed by other nodes: rabbitmqctl set_policy ha-queue "^your-queue-name$" '{"ha-mode":"all"}' --priority 0 --apply-to queues.

[0025] In step S2, when expanding capacity, the mirroring policy of some queues may consume a lot of system resources. You can enable the master node priority mode of the mirrored queue to ensure that the master node processes first and reduce the synchronization overhead during expansion: rabbitmqctlset_policy ha-mirror "^your-queue-name$" '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic","ha-promote-on-failure":true}' --priority 1 --apply-to queues.

[0026] During the process of non-stop expansion, the message priority mechanism is used to ensure that high-priority messages are processed first to prevent low-priority messages from blocking the execution of critical tasks: rabbitmqctl set_policy priority-queue "^your-queue-name$" '{"x-max-priority": 10}' --priority 1 --apply-to queues.

[0027] In step S3, specifically: verify whether the queue is successfully set as a mirror queue through the command: rabbitmqctllist_queues name arguments policy; there is another way to verify the status of the mirror queue, specifically: check whether the mirror queue is successfully set by enabling the rabbitmq manager plug-in: rabbitmq-pluginsenable rabbitmq_management.

[0028] Manually modify the cpu, memory limit and request parameters in the YAML of the containerized rabbitmq deployment and save and confirm.

[0029] At this time, the service is restarted in a rolling manner. Because the mirror mode is started, when one node is restarting, the other two nodes take over the service, the client connection will not be disconnected, and RabbitMQ can continue to provide services.

[0030] After starting the service, rebalance the master node allocation for all queues to ensure that the queue load is distributed on multiple nodes, avoid excessive pressure on a certain node, and optimize resource utilization. all refers to all queues, and --vhost-pattern / means that the operation applies to all virtual hosts: rabbitmq-queues rebalance all --vhost-pattern / .

[0031] It also includes mirroring cancellation steps: When performing performance optimization or testing, turn off or clear the mirroring policy: rabbitmqctl clear_policy -p / ha-all-queue.

[0032] In some scenarios, you may need to temporarily or permanently cancel the mirroring policy, for example, to reduce network or disk pressure. At this time, you can clear the mirror queue policy as needed. After cancellation, the queue will no longer be mirrored to multiple nodes.

[0033] In the above method, for the specific structure of the mirror queue, such as Figure 2 As shown in the figure, after the producer and consumer connect to rabbitmq, the corresponding Connection and Channel processes will be created inside rabbitmq. The Connecton process receives the message sent by the producer from the socket and delivers it to the Channel process.

[0034] In the Channel process, according to the exchange to which the message is sent and the routing-key of the message, the process PIDs of all matching Queues are found in the routing table of the internal database, and then the message is delivered to the Queue process. In the case of mirrored queues, the Channel process sends the message to all Follower processes of the queue in addition to the Leader process of the queue, and the Follower processes are all on remote nodes, so there is an additional network interaction between clusters.

[0035] After the leader process of the mirror queue receives the message, it needs to synchronize the message to all follower processes. RabbitMQ uses the GM (multicast) algorithm to implement it. The leader and all followers in the mirror queue will send and receive a message once, and will also send an ACK for the message once and receive an ACK for the message once.

[0036] The present invention can expand the capacity of RabbitMQ middleware in single-point and cluster modes without downtime, and solves the problem that the resource configuration and operating parameters of the containerized RabbitMQ service need to be restarted when modified through the mirror mode, especially for the production environment, to avoid downtime affecting the business.

[0037] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for effectively solving the problem of non-stop expansion of containerized RabbitMQ, characterized in that: The following steps are involved: S1. Enable the mirroring strategy and set the existing queue as a mirrored queue; S2. Enable mirror queue priority mode and message priority mechanism; S3. Verify the mirror queue status; S4. Expand RabbitMQ and rebalance the master node allocation.

2. The method for effectively solving the non-stop expansion of containerized RabbitMQ according to claim 1 is characterized in that: The mirroring strategy enabled in the step S1 is specifically as follows: the queue is mirrored on multiple nodes by setting ha-mode, and queue synchronization is set by ha-sync-mode.

3. The method for effectively solving the non-stop expansion of containerized RabbitMQ according to claim 1 is characterized in that: The S3 step specifically includes: verifying through a command whether the queue is successfully set as a mirror queue.

4. The method for effectively solving the non-stop expansion of containerized RabbitMQ according to claim 3 is characterized in that: There is another way to verify the state of the mirror queue in the S3 step, specifically: check whether the mirror queue is successfully set up by enabling the RabbitMQ manager plug-in.

5. The method for effectively solving the non-stop expansion of containerized RabbitMQ according to claim 1 is characterized in that: It also includes a mirror cancellation step: when performing performance optimization or testing, turn off or clear the mirroring strategy.

Images