Method and system for calling Rocket MQ in cluster outside cluster

Abstract

The invention discloses a method and system for calling a Rocket MQ in a cluster outside the cluster, and relates to the technical field of cluster deployment, and the method comprises the following steps: packaging an instruction for obtaining a podIP into a mirror image, embedding the mirror image into a containers field of a pod to be called of the Rocket MQ cluster, and storing the podIP and a port number as environment variables into a broker configuration file of the pod to be called; when the Rocket MQ cluster is started, establishing IP communication between the naming server and the broker node of the pod to be called based on the broker configuration file; and establishing a connection between the API interface of the K8s cluster and the naming server, and obtaining the pod IP and the port number of the pod to be called based on the API interface of the K8s cluster. According to the method and the device, the technical problem that the broker node of the old version Rocket MQ cannot be exposed and called outwards in the prior art is relieved.

Description

Technical Field

[0001] This invention relates to the field of cluster deployment technology, specifically to a method and system for calling RocketMQ within a cluster from outside the cluster. Background Technology

[0002] The official new architecture of RocketMQ inserts a broker layer between the original client and message service nodes (Broker nodes). The strategy is to move stateless functionality from the client side down to the broker layer as much as possible, while simultaneously moving stateless functionality from the Broker side up to the broker layer. The client's original load balancing mechanism, fault isolation, and push / pop consumption model are moved down to the broker layer, while the Broker's access control, multi-protocol adaptation, client governance, and NameServer's message routing capabilities are moved up to the broker layer.

[0003] Current technology utilizes newly added resources to obtain the mapping relationship between tags and virtual IPs when accepting requests. These new resources then monitor and update the podIP list in real time, updating the load balancing configuration file with the updated podIP list. However, this method only supports RocketMQ versions 5.0 and later; earlier versions are not supported, making it impossible to smoothly upgrade existing RocketMQ clusters. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for calling RocketMQ from outside the cluster in order to solve at least one of the above-mentioned technical problems.

[0005] In a first aspect, embodiments of the present invention provide a method for calling RocketMQ within a cluster from outside the cluster, comprising: packaging the instruction to obtain the pod IP into an image and embedding it into the containers field of the pod to be called in the RocketMQ cluster; saving the pod IP and port number as environment variables in the broker configuration file of the pod to be called; the RocketMQ cluster is a component deployed in a K8s cluster; when the RocketMQ cluster starts, establishing IP communication between the name server and the broker node of the pod to be called based on the broker configuration file; establishing a connection between the API interface of the K8s cluster and the name server, and obtaining the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

[0006] Secondly, embodiments of the present invention also provide a system for calling RocketMQ within a cluster from outside the cluster, comprising: an embedding module, a communication module, and a calling module; wherein, the embedding module is used to package the instruction to obtain the pod IP into an image and embed it into the containers field of the pod to be called in the RocketMQ cluster, and save the pod IP and port number as environment variables in the broker configuration file of the pod to be called; the RocketMQ cluster is a component deployed in a K8s cluster; the communication module is used to establish IP communication between the name server and the broker node of the pod to be called based on the broker configuration file when the RocketMQ cluster starts; the calling module is used to establish a connection between the API interface of the K8s cluster and the name server, and obtain the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method provided in the embodiments of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method provided in the embodiments of the present invention.

[0009] This invention provides a method and system for calling RocketMQ within a cluster from outside the cluster. It eliminates the need for creating new resources to continuously monitor pod IPs; simply package the code for obtaining the pod IP into a Docker image and place it in a container to ensure that the container executes first, thus obtaining the actual IP of the current pod. This invention supports exposing broker nodes in RocketMQ versions 5.0 and below, allowing services outside the Kubernetes cluster to use the RocketMQ cluster deployed within the Kubernetes cluster. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1A flowchart illustrating a method for calling RocketMQ from outside the cluster, as provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a system for calling RocketMQ from outside the cluster, provided as an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0013] Figure 1 This is a flowchart illustrating a method for calling RocketMQ from outside the cluster, according to an embodiment of the present invention. Figure 1 As shown, the method specifically includes the following steps: Step S102: Package the instruction to obtain the podIP into an image and embed it into the containers field of the pod to be called in the RocketMQ cluster. Save the podIP and port number as environment variables in the broker configuration file of the pod to be called. The RocketMQ cluster is a component deployed in a K8s cluster.

[0014] Step S104: When the RocketMQ cluster starts, establish a name server based on the broker configuration file to communicate with the IP address of the broker node of the pod to be invoked.

[0015] Step S106: Establish a connection between the API interface of the K8s cluster and the naming server, and obtain the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

[0016] Specifically, the method provided in this invention for calling RocketMQ from outside the cluster obtains the broker node IP and port number of the RocketMQ cluster through the official API (Application Programming Interface) of Kubernetes (an open-source container orchestration platform). In the RocketMQ operator, the communication between the broker and the nameserver is established through the broker configuration file during RocketMQ cluster startup. Since the broker node IP is only determined after the pod starts, the code for obtaining the pod IP only needs to be packaged into an image and placed in the pod's containers field, ensuring that this container executes first, and saving the pod IP and port number as environment variables. After the RocketMQ cluster starts, the communication with the nameserver node is via the broker node IP, not the broker's virtual IP.

[0017] Preferably, in the method of calling RocketMQ from outside the cluster provided in this embodiment of the invention, all components are deployed in a K8s cluster. When the broker node pod starts, it reads the broker's configuration file, replaces the IP and port number for communicating with the nameserver with the real IP and port number of the broker node, and then the service outside the cluster connects through the service IP and port number exposed by the nameserver. Finally, it obtains the IP and port number of the current pod through the K8s API interface, saves them as environment variables in the pod, and reads the broker's configuration file when the broker starts.

[0018] Compared with existing technologies, the method provided in this invention does not require creating new resources for continuous monitoring of pod IPs. It only requires encapsulating the code for obtaining the pod IP into a Docker image and placing it in a container to ensure that the container executes first, thus obtaining the actual IP of the current pod. This invention supports the external exposure of broker nodes below RocketMQ version 5.0, allowing services outside the Kubernetes cluster to also use the RocketMQ cluster deployed within the Kubernetes cluster.

[0019] Figure 2 This is a schematic diagram of a system for calling RocketMQ within a cluster from outside the cluster, according to an embodiment of the present invention. Figure 2 As shown, the system includes: an embedded module 10, a communication module 20, and a calling module 30.

[0020] Specifically, the embedded module 10 is used to package the instruction to obtain the podIP into an image and embed it into the containers field of the pod to be called in the RocketMQ cluster, and save the podIP and port number as environment variables in the broker configuration file of the pod to be called; the RocketMQ cluster is a component deployed in a K8s cluster; Communication module 20 is used to establish a name server and communicate with the IP address of the broker node of the pod to be invoked based on the broker configuration file when the RocketMQ cluster starts. Module 30 is called to establish a connection between the API interface of the K8s cluster and the name server, and to obtain the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

[0021] The present invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method provided in the embodiments of the present invention.

[0022] The present invention also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method provided in the embodiments of the present invention.

[0023] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0024] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for calling RocketMQ within a cluster from outside the cluster, characterized in that, include: The command to obtain the pod IP is packaged into an image and embedded into the containers field of the pod to be called in the RocketMQ cluster. The pod IP and port number are saved as environment variables in the broker configuration file of the pod to be called. The RocketMQ cluster is a component deployed in a K8s cluster. When the RocketMQ cluster starts, a naming server is established based on the broker configuration file to communicate with the IP address of the broker node of the pod to be invoked; Establish a connection between the API interface of the K8s cluster and the naming server, and obtain the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

2. A system for calling RocketMQ within a cluster from outside the cluster, characterized in that, include: The module consists of an embedded module, a communication module, and a calling module; among which, The embedding module is used to package the instruction to obtain the podIP into an image and embed it into the containers field of the pod to be called in the RocketMQ cluster, and save the podIP and port number as environment variables in the broker configuration file of the pod to be called; the RocketMQ cluster is a component deployed in a K8s cluster; The communication module is used to establish a name server and communicate with the IP address of the broker node of the pod to be invoked based on the broker configuration file when the RocketMQ cluster starts. The calling module is used to establish a connection between the API interface of the K8s cluster and the naming server, and to obtain the pod IP and port number of the pod to be called based on the API interface of the K8s cluster.

3. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in claim 1.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in claim 1.

Images