Directional communication method of instant messaging system during clustering based on MQTT protocol

Abstract

The invention relates to a directional communication method of an instant messaging system during clustering based on an MQTT protocol, which comprises the following steps of: when a plurality of back-end Broker service nodes communicate with each other, only sending communication data to a specified node, and storing a mapping relationship between each user and the back-end Broker service node connected with the user by using a uniform storage component, according to the method, a plurality of Broker service nodes in a cluster environment can conveniently obtain the serial numbers of the Broker nodes to be connected with all users, and meanwhile, the messages are directionally forwarded to the specified Broker service nodes, so that the network flow can be greatly reduced, the performance loss of non-target Broker service nodes is reduced, the possible avalanche effect is avoided, the processing efficiency of a system and the network throughput are improved, and the user experience is improved. The system stability is enhanced.

Description

Technical Field

[0001] The invention relates to a directional communication method of an instant messaging system based on the MQTT protocol in clustering, belonging to the technical field of instant messaging. Background Art

[0002] For instant messaging systems, if the MQTT-based instant messaging system only uses a single-node backend Broker service, because the underlying layer uses the TCP / IP protocol, the number of simultaneous online users (that is, the number of connections held by the backend Broker service) will be subject to many restrictions such as the CPU / memory / maximum number of connections of a single server. If the server configuration of the Broker service node is continuously increased to adapt to the growing number of enterprise or commercial application services and users, the cost is very expensive and not proportional to the benefits. If only multiple Broker service nodes are deployed, different users connect to different backend Broker services, that is, each backend Broker service only holds the user connection connected to this service node, so the service node cannot know the user session status of other services holding connections, and the service node cannot send messages to users through the connection.

[0003] When an instant messaging system based on the MQTT protocol deploys multiple Broker service nodes and forms a Broker service cluster, these Broker service nodes will communicate with each other. When there is a message that needs to be forwarded, a Broker service node will publish the message to the Redis topic, and all the remaining Broker service nodes will receive the message and try to process it. Especially when the instant messaging system is at its peak, a huge amount of messages will be forwarded in the network, which will not only cause the network traffic to increase exponentially, but also the Broker service will cause performance loss because it needs to receive and judge the messages to be discarded. At the same time, these two adverse consequences will further cause the instant messaging system to be unable to process continuous user messages in a timely manner, and even form an avalanche effect, causing the system to crash and fail to provide services to users normally.

[0004] A common method is to use the publish / subscribe function of Redis to let all Broker service nodes subscribe to the same topic. When a Broker service node sends a message to the topic, all other nodes will receive the message and push it to the user. This technology does not have too many disadvantages when the number of users and messages is not high, but once the number of users and messages increases exponentially, multiple backend Broker service nodes are usually deployed, and different users connect to different Broker service nodes, presenting a staggered form. At this time, when users send and receive messages, the messages will most likely be forwarded through the Redis publish / subscribe channel, which means that a huge amount of messages will reach Redis through the network, and then Redis will forward these huge messages to all Broker service nodes. After all nodes receive the data, they will process it. This process is undoubtedly under tremendous pressure during the peak period of messages, making the entire instant messaging system shaky, and also causing inconvenience to operations and maintenance personnel. Summary of the invention

[0005] In order to solve the above problems in the prior art, the present invention proposes a directional communication method of an instant messaging system based on the MQTT protocol in clustering.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, the present invention proposes a directional communication method of an instant messaging system based on the MQTT protocol in a cluster, comprising the following steps:

[0008] Set a unique number for each Broker service node in the Broker cluster of the instant messaging system;

[0009] Each Broker service node stores the mapping relationship between the connected user and its unique number in the storage component;

[0010] When a user sends a message to another user, the Broker service node corresponding to the message sender determines whether the message sender and the message receiver are connected to the same Broker service node. If they are connected to the same Broker service node, the message is directly sent to the message receiver.

[0011] If they are not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship in the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component. The Broker service node to which the message receiver is connected then sends the information to the message receiver.

[0012] As a preferred implementation mode of the present invention, each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in the publish / subscribe component with the unique number as the topic name.

[0013] As a preferred implementation mode of the present invention, each Broker service node stores its mapping relationship with all users in a local memory.

[0014] As a preferred implementation mode of the present invention, the Broker service node corresponding to the message sender determines whether the message receiver is connected to the same Broker service node through the mapping relationship stored in its local memory.

[0015] On the other hand, the present invention also provides a directional communication system of an instant messaging system based on the MQTT protocol in a cluster, comprising a Broker initialization module, a mapping relationship storage module, a communication connection judgment module and a communication module;

[0016] The Broker initialization module is used to set a unique number for each Broker service node in the Broker cluster in the instant messaging system;

[0017] The mapping relationship storage module is used to store the mapping relationship between the connected user and its unique number in the storage component by each Broker service node;

[0018] The communication connection judgment module is used to judge whether the message sender and the message receiver are connected to the same Broker service node;

[0019] The communication module is used to send messages based on the judgment result of the signal connection judgment module. Specifically, if the message sender and the message receiver are connected to the same Broker service node, the message is sent to the message receiver directly through the Broker service node. If the message sender and the message receiver are not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship in the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component. The Broker service node to which the message receiver is connected then sends the information to the message receiver.

[0020] As a preferred implementation mode of the present invention, each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in the publish / subscribe component with the unique number as the topic name.

[0021] As a preferred implementation mode of the present invention, each Broker service node stores its mapping relationship with all users in a local memory.

[0022] As a preferred implementation mode of the present invention, the Broker service node corresponding to the message sender determines whether the message receiver is connected to the same Broker service node through the mapping relationship stored in its local memory.

[0023] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any embodiment of the present invention is implemented.

[0024] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0025] The present invention has the following beneficial effects:

[0026] 1. The present invention stores the mapping relationship between users and Broker service nodes through a unified storage component, so that multiple Broker service nodes in a cluster environment can obtain the Broker service node numbers to which all users are connected. By forwarding messages to designated Broker service nodes in a targeted manner, network traffic can be greatly reduced, performance loss of non-target Broker service nodes can be reduced, possible avalanche effects can be avoided, system processing efficiency and network throughput can be improved, and system stability can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method of the present invention;

[0028] Figure 2 The subscription relationship between the Broker cluster node and Redis in the embodiment of the present invention;

[0029] Figure 3 This is a timing diagram of a user connecting to a cluster node according to an embodiment of the present invention;

[0030] Figure 4 This is a timing diagram of message sending to different users under the same Broker service node in an embodiment of the present invention;

[0031] Figure 5 This is a timing diagram of different user message sending under different Broker service nodes in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0034] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0036] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0037] Embodiment 1:

[0038] Terminology explanation:

[0039] MQTT protocol: MQTT (Message Queuing Telemetry Transport) is literally translated as message queue telemetry transmission. It is a lightweight, publish / subscribe-based message transmission application layer protocol built on the TCP / IP protocol. It is suitable for resource-constrained devices and low-bandwidth, high-latency or unstable network environments. It is widely used in the Internet of Things, small devices, mobile applications, etc.

[0040] Publish / Subscribe model: It is a messaging model that decouples the client that sends messages (publishers) from the client that receives messages (subscribers), so that the two do not need to establish a direct connection or specify the existence of each other.

[0041] Broker: In the MQTT protocol, as a proxy role, it is responsible for routing and distributing all messages, that is, receiving messages sent by publisher clients and forwarding them to eligible subscriber clients. In addition, the broker is also responsible for processing requests initiated by clients such as connection, disconnection, subscription, and unsubscription. The Broker server will hold a persistent connection with the client and communicate with the client through the persistent connection.

[0042] Instant Messaging (IM for short) is a real-time communication service that allows users to set up a private chat room on the Internet, allowing two or more people to use the Internet to instantly transmit text messages, files, voice and video communications, such as the common WeChat software. Instant messaging is actually shorter than sending emails and more convenient than making phone calls. It is undoubtedly the most convenient way of communication in the Internet age. Most instant messaging services provide status information features - displaying the list of contacts, whether the contacts are online, and whether you can talk to the contacts, such as the common QQ software.

[0043] Cluster: usually refers to a group of independent computers or application systems connected by high-speed networks. They form a group and are managed as a single system. When a client interacts with the cluster, the cluster acts like an independent server or application system. Cluster technology can achieve relatively high benefits in performance, reliability, and flexibility at a relatively low cost.

[0044] TCP / IP protocol: TCP / IP (Transmission Control Protocol / Internet Protocol) refers to a protocol suite that can realize information transmission between multiple different networks. TCP / IP protocol does not only refer to TCP and IP protocols, but also refers to a protocol suite composed of FTP, SMTP, TCP, UDP, IP and other protocols.

[0045] See also Figure 1 , a directional communication method of an instant messaging system based on the MQTT protocol in a cluster, comprising the following steps:

[0046] Set a unique number for each Broker service node in the Broker cluster of the instant messaging system;

[0047] Each Broker service node stores the mapping relationship between the connected user and its unique number in a storage component. In this embodiment, the storage component is Redis. Hazelcast, relational database, etc. can also be selected as a storage component;

[0048] When a user sends a message to another user, the Broker service node corresponding to the message sender determines whether the message sender and the message receiver are connected to the same Broker service node. If they are connected to the same Broker service node, the message is directly sent to the message receiver.

[0049] If it is not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship of the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component (in this embodiment, the publish / subscribe component is also implemented through Redis, and other optional publish / subscribe components are RocketMQ, RabbitMQ, etc.), and the Broker service node to which the message receiver is connected then sends the information to the message receiver.

[0050] As a preferred implementation of this embodiment, see Figure 2 Each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in Redis with the unique number as the topic name.

[0051] As a preferred implementation of this embodiment, see Figure 3 When each user uses the instant messaging system, he will first connect to a node in the Broker cluster under the instant messaging service, and the current Broker service node will receive the current user's connection. The node saves the user's connection descriptor object to the local memory so that it can subsequently read the message data sent by the user to the Broker service through the connection descriptor object and write the message data to the user client. Subsequently, the Broker service node stores the mapping relationship between the user and the Broker service node number in Redis.

[0052] When multiple users have successfully established connections with different Broker service nodes in the Broker cluster through the above process, these users can now send messages to each other.

[0053] When one user sends a message to another user, the message will be transmitted from the sender's client to the Broker service node to which it is connected through the MQTT protocol. The Broker service node will first parse out which target user the message needs to be delivered to, and then determine whether there is a connection descriptor object for the target user in the local memory.

[0054] If it exists, it means that the target user is also connected to the current Broker service node, see Figure 4, then the Broker service node can directly obtain the user's connection descriptor object from the user and its connection descriptor mapping relationship set stored in the local memory, and directly write the message data through the connection descriptor object to achieve the effect of pushing the message content to the target user;

[0055] If it does not exist, it means that the target user is connected to other Broker service nodes, see Figure 5 , then the Broker service node can obtain the Broker service node number of the target user from the mapping relationship between users and Broker service node numbers stored in Redis, and send the message information to be pushed to the "Broker service node number of the target user" topic in Redis. When the Broker service node to which the target user belongs receives the message information through the Redis publish / subscribe channel, it will obtain the connection descriptor object of the target user of the message from the local memory. After obtaining the connection descriptor object, the Broker service will write the message data through the object to achieve the effect of pushing the message content to the target user.

[0056] Optionally, when the Broker service is started, instead of subscribing to the Redis topic, the current Broker service node actively establishes MQTT protocol connections with all other existing Broker service nodes as a client, and stores the mapping relationship between all Broker service node numbers and their connection descriptor objects in the local memory of the current Broker service node;

[0057] In the message forwarding process, sending a message to the "specified Broker number" topic in Redis is replaced by obtaining the connection descriptor object of the specified Broker number from the local memory and forwarding the message to the target Broker service node through the object.

[0058] Embodiment 2:

[0059] A directional communication system of an instant messaging system based on the MQTT protocol in a cluster, comprising a Broker initialization module, a mapping relationship storage module, a communication connection judgment module and a communication module;

[0060] The Broker initialization module is used to set a unique number for each Broker service node in the Broker cluster in the instant messaging system;

[0061] The mapping relationship storage module is used to store the mapping relationship between the connected user and its unique number in the storage component by each Broker service node;

[0062] The communication connection judgment module is used to judge whether the message sender and the message receiver are connected to the same Broker service node;

[0063] The communication module is used to send messages based on the judgment result of the signal connection judgment module. Specifically, if the message sender and the message receiver are connected to the same Broker service node, the message is sent to the message receiver directly through the Broker service node. If the message sender and the message receiver are not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship in the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component. The Broker service node to which the message receiver is connected then sends the information to the message receiver.

[0064] As a preferred implementation of this embodiment, each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in the publish / subscribe component with the unique number as the topic name.

[0065] As a preferred implementation of this embodiment, each Broker service node stores its mapping relationship with all users in a local memory.

[0066] As a preferred implementation of this embodiment, the Broker service node corresponding to the message sender determines whether the message receiver is connected to the same Broker service node through the mapping relationship stored in its local memory.

[0067] Embodiment three:

[0068] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0069] Embodiment 4:

[0070] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present invention is implemented.

[0071] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0072] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0074] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0075] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A directional communication method for an instant messaging system based on the MQTT protocol in a cluster, characterized in that: The following steps are involved: Set a unique number for each Broker service node in the Broker cluster of the instant messaging system; Each Broker service node stores the mapping relationship between the connected user and its unique number in the storage component; When a user sends a message to another user, the Broker service node corresponding to the message sender determines whether the message sender and the message receiver are connected to the same Broker service node. If they are connected to the same Broker service node, the message is directly sent to the message receiver. If they are not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship in the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component. The Broker service node to which the message receiver is connected then sends the information to the message receiver.

2. According to claim 1, a directional communication method of an instant messaging system based on the MQTT protocol in a cluster, characterized in that: Each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in the publish / subscribe component with the unique number as the topic name.

3. According to claim 1, a directional communication method of an instant messaging system based on the MQTT protocol in a cluster, characterized in that: Each Broker service node stores its mapping relationship with all users in local memory.

4. The directional communication method of an instant messaging system based on the MQTT protocol in a cluster according to claim 3, characterized in that: The Broker service node corresponding to the message sender determines whether the message receiver is connected to the same Broker service node through the mapping relationship stored in its local memory.

5. A directional communication system for an instant messaging system based on the MQTT protocol in clustering, characterized in that: It includes Broker initialization module, mapping relationship storage module, communication connection judgment module and communication module; The Broker initialization module is used to set a unique number for each Broker service node in the Broker cluster in the instant messaging system; The mapping relationship storage module is used to store the mapping relationship between the connected user and its unique number in the storage component by each Broker service node; The communication connection judgment module is used to judge whether the message sender and the message receiver are connected to the same Broker service node; The communication module is used to send messages based on the judgment result of the signal connection judgment module. Specifically, if the message sender and the message receiver are connected to the same Broker service node, the message is sent to the message receiver directly through the Broker service node. If the message sender and the message receiver are not connected to the same Broker service node, the Broker service node to which the message receiver is connected is queried through the mapping relationship in the storage component, and the message is sent to the corresponding Broker service node through the publish / subscribe component. The Broker service node to which the message receiver is connected then sends the information to the message receiver.

6. The directional communication system of the instant messaging system based on the MQTT protocol in cluster according to claim 1, characterized in that: Each Broker service node generates a random string as a unique number during the startup process, and subscribes to a topic in the publish / subscribe component with the unique number as the topic name.

7. The directional communication system of the instant messaging system based on the MQTT protocol in cluster according to claim 1, characterized in that: Each Broker service node stores its mapping relationship with all users in local memory.

8. The directional communication system of the instant messaging system based on the MQTT protocol in cluster according to claim 7, characterized in that: The Broker service node corresponding to the message sender determines whether the message receiver is connected to the same Broker service node through the mapping relationship stored in its local memory.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images