A dynamic, aggregated message distribution method and system

Abstract

The application discloses a dynamic message aggregation and distribution method and system. The destination address of an original message in a received message is judged. If the destination address is local, the message is directly sent. If the destination address is not local, it is judged whether the message is repeated according to the message type and content. For the repeated message, message aggregation is performed. The message aggregation includes counting the message identifier of the repeated message and determining the forwarding server list of the aggregated message. The aggregated message is sent to the remote server closest to the local server in the server list. The aggregated message in the received message is dis-aggregated. The message with the local destination address is sent to the destination address. After the local address in the server list is deleted, the aggregated message is sent to other remote servers. Through accurate calculation of only receiving the forwarding server node set of the message, the forwarding of the group message to irrelevant servers is reduced, thereby greatly reducing the redundant network traffic and saving the valuable bandwidth resources.

Description

Technical Field

[0001] This invention relates to distributed information systems, specifically to a dynamic aggregation message distribution method and system. Background Technology

[0002] With the rapid development of internet technology, distributed systems, due to their advantages such as high availability, scalability, and fault tolerance, are widely used in various large-scale application scenarios. Message distribution systems are responsible for reliably and efficiently transmitting messages between different nodes and clients, and are a widely used application system.

[0003] In existing distributed message distribution systems, when a client needs to send a group message, a common practice is to send the message to its registered server, which then broadcasts the message to all other servers in the system, or forwards it according to some preset routing rules. However, this approach has significant drawbacks: First, it wastes bandwidth: if the member clients in the group are only distributed on a few servers, broadcasting the message to all servers or too many unnecessary servers will lead to a large amount of redundant message transmission, seriously wasting network bandwidth; second, it is inefficient: redundant message forwarding increases the possibility of network congestion, prolongs the end-to-end latency of messages, and reduces the overall processing efficiency of the system; third, it has poor adaptability: it is difficult to quickly adapt to topology adjustments such as server nodes joining or leaving, which may lead to message loss or duplicate transmission.

[0004] Therefore, how to achieve accurate and non-duplicated forwarding of group messages in a distributed environment with dynamically changing server network topology, in order to save bandwidth and improve message distribution efficiency, is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] Purpose of the invention: To address the above-mentioned shortcomings, this invention provides a dynamic aggregation message distribution method and system that reduces the repeated forwarding of messages between servers, thereby effectively saving network bandwidth and improving the timeliness of message delivery and the overall performance of the system.

[0006] Technical Solution: To solve the above problems, this invention employs a dynamic aggregated message distribution method, comprising the following steps:

[0007] Receive messages, including raw messages and aggregated messages;

[0008] The system determines the destination address of the original message. If the destination address is local, the message is sent directly to that address. If the destination address is not local, the system checks for duplicate messages based on the message type and the hash code of the message content. For duplicate messages, the system aggregates the messages. This aggregation includes identifying the message identifiers of the duplicate messages and determining a list of forwarding servers for the aggregated messages. The aggregated messages are then sent to the remote server in the server list that is closest to the local server. The server list includes servers associated with the destination address based on the message identifiers and a mapping table. This mapping table includes the correspondence between the client and the server.

[0009] De-aggregate the aggregated message, obtain the message whose destination address is local, and send the message to the destination address; after deleting the local server address from the server list in the aggregated message, send the aggregated message to other remote servers in the server list.

[0010] Furthermore, the original message includes a message ID, a flag block, a source address, a destination address, a message content, and a hash code; the aggregated message includes a message ID list, a flag block, a source address list, a destination address list, a forwarding path, a message content, and a hash code; the flag block is used to describe the aggregation type and message type of the message, the aggregation type includes unaggregated and aggregated, and the message type includes general messages, client registration messages, group management messages, and server registration messages.

[0011] The present invention discloses a dynamic aggregated message distribution system, comprising a client node and a server node, wherein the server node comprises a message receiving module, a message sending module, and a message aggregation module;

[0012] The message receiving module is used to receive messages sent by other server nodes or client nodes, including original messages and aggregated messages;

[0013] The message aggregation module is used to determine the destination address of the original message. If the destination address of the original message is local, it is directly sent to the destination address through the message sending module. If the destination address of the original message is not local, it determines whether the message is duplicated by using the message type and the hash code of the message content. For duplicate messages, message aggregation is performed. Message aggregation includes counting the message identifiers of duplicate messages and determining the forwarding server list of the aggregated messages. The aggregated message is sent to the remote server in the server list that is closest to the local server. The server list includes servers associated with the destination address based on the message identifier according to a mapping table. The mapping table includes the correspondence between the client and the server.

[0014] The message aggregation module is also used to de-aggregate the aggregated messages, obtain messages with destination addresses on the local machine, and send the messages to the destination addresses through the message sending module; after deleting the local server addresses from the server list in the aggregated messages, the aggregated messages are sent to other remote servers in the server list through the message sending module.

[0015] Furthermore, the original message includes a message ID, a flag block, a source address, a destination address, a message content, and a hash code; the aggregated message includes a message ID list, a flag block, a source address list, a destination address list, a forwarding path, a message content, and a hash code; the flag block is used to describe the aggregation type and message type of the message, the aggregation type includes unaggregated and aggregated, and the message type includes general messages, client registration messages, group management messages, and server registration messages.

[0016] Furthermore, the server node also includes a registration management module, which is used to maintain global or local mapping tables. The mapping tables include client-server mapping tables, client-group mapping tables, group-server mapping tables, and server-server mapping tables. The client-server mapping table is used to record the server nodes registered by client nodes, the client-group mapping table is used to record the client nodes included in the group, the group-server mapping table is used to record the list of server nodes registered by the client nodes belonging to the group, and the server-server mapping table is used to record the network cascading status between server nodes.

[0017] Furthermore, determining the list of forwarding servers for the aggregated messages specifically includes: determining the target client receiving the message based on the destination address in the original message statistics; determining the group to which the target client belongs based on the client-group mapping table; determining the list of server nodes receiving the message based on the client-server mapping table and the group-server mapping table; and determining the server nodes in the server node list that are cascaded with the local server node based on the server-server mapping table.

[0018] Furthermore, the client registration message is sent by the client node to the server node when the client node registers with the server node, providing the server node with its identification information and group information. The server node updates the mapping table according to the client registration message, then generates an update mapping table message, aggregates the update mapping table messages, and sends the aggregated messages to other server nodes.

[0019] The group management message is sent by the client node when adding or changing a group. The client node sends the client node information and group member information to the registered server node by sending the group management message. The server node updates the mapping table according to the group management message, then generates an update mapping table message, aggregates the update mapping table messages, and sends the aggregated message to other server nodes.

[0020] The server registration message is sent by a server node to register with cascaded server nodes. The cascaded server nodes update the mapping table according to the server registration message, then generate an update mapping table message, aggregate the update mapping table messages, and send the aggregated message to other server nodes.

[0021] Furthermore, it also includes a message caching module, which is used to cache received and pending messages. The message caching module includes a receive cache and a send cache. The receive cache is responsible for managing the received messages. After being processed by the message aggregation module, the messages are cached in the send cache and then sent to the next node by the message sending module. Both the receive cache and the send cache include a local client cache and a remote server cache, which are used to store messages received and pending to be sent by the local client and messages received and pending to be sent by the remote server, respectively.

[0022] The present invention also employs a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0023] The present invention also employs a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0024] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: First, it significantly saves bandwidth. By accurately calculating the set of forwarding server nodes that only need to receive messages, it reduces the forwarding of group messages to irrelevant servers, thereby greatly reducing redundant network traffic and saving valuable bandwidth resources. Second, it improves message distribution efficiency. Reducing redundant forwarding means less network latency and higher throughput, allowing messages to reach target member clients faster, improving user experience and the overall system response speed. Third, it has strong dynamic adaptability. The system can perceive changes in client registration servers and changes in the network topology between servers in real time or near real time, and dynamically adjust message forwarding strategies accordingly, ensuring reliable and efficient message delivery in complex and dynamic environments. Fourth, it has good scalability. Due to the reduction of unnecessary communication between servers, the system can more easily expand the number of server nodes and support more clients and groups without causing performance bottlenecks due to excessive internal communication overhead. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall framework of the dynamic aggregated message distribution system in this invention.

[0026] Figure 2 This is a schematic diagram of the client registration process in this invention.

[0027] Figure 3 This is a schematic diagram of the group management process in this invention.

[0028] Figure 4 This is a schematic diagram of the server registration process in this invention.

[0029] Figure 5 This is a schematic diagram of message caching in this invention.

[0030] Figure 6 This is a schematic diagram of the message aggregation process in this invention.

[0031] Figure 7 This is a schematic diagram of the message de-aggregation process in this invention. Detailed Implementation

[0032] like Figure 1 As shown, this embodiment of a dynamic aggregated message distribution system includes several server nodes and several client nodes. The server nodes can adapt to dynamically changing network topologies, and the client nodes can send and receive messages by registering with any server node. Each server node includes a message receiving module, a message caching module, a message sending module, a message aggregation module, and a registration management module.

[0033] The message receiving module receives messages sent by other server nodes or client nodes. Messages include raw messages and aggregated messages. Raw message content includes message ID, flag block, source address, destination address, message content, and hash code. Aggregated message content includes a list of message IDs, flag block, source address list, destination address list, forwarding path, message content, and hash code. The flag block describes the aggregation type and message type. Aggregation types include {NON, ZIP}, corresponding to unaggregated and aggregated categories. Message types include {NML, CREG, GMNG, SREG}, corresponding to general messages, client registration messages, group management messages, and server registration messages. The message aggregation module supports message de-aggregation based on flag block information, restoring aggregated messages to their original form. General messages, client registration messages, and group management messages are generated and sent by the client, while server registration messages are generated and sent by the server.

[0034] The message caching module is used to cache received and pending messages. The message caching module includes a receive cache and a send cache. Both the receive cache and the send cache include a local client cache and a remote server cache, which are used to store messages received and pending to be sent by the local client and the remote server, respectively. The receive cache also includes an offline client message cache, which caches messages received by the currently offline client. Once the user logs in online, the messages are migrated to the send cache for sending.

[0035] The message aggregation module is used to aggregate repeatedly sent messages and also to de-aggregate aggregated messages. Messages in the receive buffer are processed by the message aggregation module and then cached in the send buffer, and then sent to the next node by the message sending module.

[0036] The registration management module manages the network topology between clients, groups, and servers, and maintains global or local mapping tables. The module supports updating these mapping tables. The mapping tables include client-server, client-group, group-server, and server-server mapping tables. The system updates these mapping tables by sending client registration messages, group management messages, and server registration messages.

[0037] Client-Server Mapping Table Used to record the servers registered by the currently logged-in client. This indicates that client i is registered with server j; otherwise... .

[0038] Client-Group Mapping Table Used to record the list of clients included in the group. This indicates that client i belongs to group j; otherwise... ;

[0039] Group-Server Mapping Table Used to record the list of servers registered by the clients belonging to the group. This indicates that clients belonging to group i are registered with server j; otherwise... ;

[0040] Server-to-Server Mapping Table Used to record the network cascading status between servers. This indicates that server i and server j are cascaded; otherwise... .

[0041] The registration management module supports the synchronization of mapping tables between servers. For received client registration messages, group management messages, and server registration messages, the registration management module updates the local mapping table. If the mapping table is updated, the updated message is cached in the local client cache of the receiving cache. The message aggregation module aggregates the update messages that need to be aggregated and passes the updated values ​​to all reachable server nodes. If there is no update, no message is sent.

[0042] like Figure 2 As shown, the client registration message is the message sent by the client when registering with the server node. At that time, the client provides the server node with its identification information and group information (if any). The server node updates the mapping table based on the information provided by the client. renew ; and send a client registration message {CREG,} to the reachable server node. }

[0043] like Figure 3 As shown, the group management message is from the client. When adding or changing a group, the information should be sent to the registered server node. Send client Groups Group member collection Information, from server nodes Complete the update of the association mapping table and set it up. ; and send a group management message {GMNG,} to reachable server nodes. }

[0044] like Figure 4 As shown, the server registration message indicates the addition of a new server node. By sending to existing server nodes Register. Update the association mapping table ; and send a server registration message {SREG,} to the reachable server nodes. }

[0045] The registration management module also periodically checks the online status of clients and updates the association mapping table accordingly. And send the client registration message {CREG,} to the reachable server node. Update the association mapping table.

[0046] Other server nodes or client nodes send original messages or aggregated messages to this server node, such as Figure 5As shown, when the message receiving module of this service node receives a raw message sent by a local client, it stores the message in the local client cache of the receiving cache; when the message receiving module of this service node receives an aggregated message sent by a remote server, it stores the message in the remote server cache of the receiving cache.

[0047] like Figure 6 As shown, the message aggregation module judges the original messages cached locally by the receiving cache. If the client whose destination address is the original message is local, aggregation is not performed, and the message is directly cached in the local client cache of the sending cache. If the client whose destination address is the original message is registered on another server node, message aggregation is required. The message aggregation module judges whether the original messages to be aggregated in the local client cache are duplicates by judging the consistency of message type and hash code. For duplicate messages, an aggregated message is generated to reduce duplicate sending. For non-duplicate messages, they are directly cached in the remote server cache of the sending cache. The message aggregation module also stores the aggregated message in the remote server cache of the sending cache.

[0048] like Figure 7 As shown, the message aggregation module de-aggregates aggregated messages cached on the remote server receiving cache, obtains relevant client messages registered with the local server, and stores them in the local client cache of the sending cache. Simultaneously, it revises the destination address of the aggregated messages (removing the local server address) and stores them in the remote server cache of the sending cache, subsequently forwarding them to other servers. The message aggregation module can dynamically determine the target server set based on the distribution of server nodes registered by member clients within the group and the network topology between these server nodes, optimizing message forwarding paths and reducing redundant forwarding between servers.

[0049] For example, the original message content includes {msgID,TYPE,src,dst,data,hash}, where msgID represents the message ID, TYPE represents the flag block (including aggregation type and message type), src represents the source address (including client ID or group ID), dst represents the destination address (including client ID or group ID), data represents the message content, and hash represents the hash code (calculated from the message content).

[0050] The aggregated message content includes {msgIDList,TYPE,srcList,dstList,routeList,data,hash}, where msgIDList represents the list of message IDs, srcList represents the list of source addresses, dstList represents the list of destination addresses, and routeList represents the list of servers, which are calculated by the message aggregation module based on the mapping table.

[0051] Given original messages msg1={msgID1,TYPE,src1,dst1,data,hash} and msg2={msgID2,TYPE,src2,dst2,data,hash}, aggregation can be performed when the message type TYPE and hash code hash are consistent. The aggregated message is zipMSG.

[0052] msgIDList = {msgID1, msgID1}

[0053] srcList={src1,src2}

[0054] dstList={dst1,dst2}

[0055] Finally, the client ID and group ID in dstList can be used in conjunction with the client-server mapping table. and group-server mapping table Get the list of remote message receiving servers { } (excluding the local server), then routeList={ To avoid duplicate aggregation of messages, which could lead to a failure to converge the chain, the system only supports aggregating multiple original messages.

[0056] After message aggregation is complete, the zipMSG is saved to the remote server cache of the sending cache. For offline users, the message is saved to the offline user message cache and forwarded after the user comes online.

[0057] The message sending module retrieves messages from the local client cache of the sending cache and sends them to the client nodes registered with the destination address of this server.

[0058] The message sending module retrieves the aggregated message from the remote server cache of the sending cache, and then uses the server-to-server mapping table to determine the appropriate message type. Let the local server be... According to routeList={ Send aggregated messages to the server. ,in , , express arrive The distance.

[0059] For aggregated messages received by the server, the message receiving module caches them in the remote server cache of the receiving cache. The message aggregation module retrieves the aggregated message from the remote server cache of the receiving cache, deaggregates the portion that needs to be sent to the local client, and sends it to the local client. The remaining portion is then forwarded to other remote servers. Specifically:

[0060] For the received aggregated message zipMSG={msgIDList,TYPE,srcList,dstList,route,data,hash}, the message aggregation module uses dstList in conjunction with the client-server mapping table. and group-server mapping table It checks whether dstList contains local clients. If it does, it performs message de-aggregation, obtains one or more msg={msgID,TYPE,src,dst,data,hash}, and saves them to the local client cache of the sending cache. The message sending module then retrieves the message from the cache and sends it to the corresponding local client.

[0061] At the same time, set routeList={ }- ,in This represents the local server node. If routeList is empty, it means no sending is needed. If it is not empty, the revised aggregated message is saved to the remote server cache of the sending cache. The message sending module retrieves the aggregated message from the remote server cache of the sending cache and then uses the server-to-server mapping table. Send aggregated messages to ,in , , express arrive The distance.

[0062] For offline user message caching, a trigger is set to monitor the user-server mapping table. When the mapping table changes, or when a user is detected to be online, message sending is restarted.

Claims

1. A method of dynamic aggregation message distribution, characterized by, The method comprises the following steps: receiving a message, the message comprising an original message and an aggregated message; judging the destination address of the original message, and directly sending the original message to the destination address if the destination address is local; if the destination address of the original message is not local, judging whether the message is repeated by the message type and the hash code of the message content, performing message aggregation on the repeated message, the message aggregation comprising counting the message identification of the repeated message and determining the forwarding server list of the aggregated message, and sending the aggregated message to the remote server closest to the local server in the server list; the server list comprising the servers associated with the destination address obtained based on the message identification according to a mapping table, and the mapping table comprising the correspondence between the clients and the servers; de-aggregating the aggregated message, obtaining the message with the destination address being local, and sending the message to the destination address; after deleting the local server address in the server list from the aggregated message, sending the aggregated message to the other remote servers in the server list.

2. The dynamic, aggregated message distribution method of claim 1, wherein, The original message comprises a message ID, a flag block, a source address, a destination address, a message content and a hash code, the aggregated message comprises a message ID list, a flag block, a source address list, a destination address list, a forwarding path, a message content and a hash code; the flag block is used to describe the aggregation type and the message type of the message, the aggregation type comprising un-aggregated and aggregated, and the message type comprising a general message, a client registration message, a group management message and a server registration message.

3. A dynamic, aggregated message distribution system, characterized by, The system comprises a plurality of client nodes and a plurality of server nodes, and the server nodes comprise a message receiving module, a message sending module and a message aggregation module; the message receiving module is used to receive the message sent by other server nodes or client nodes, the message comprising an original message and an aggregated message; the message aggregation module is used to judge the destination address of the original message, and directly send the original message to the destination address by the message sending module if the destination address is local; if the destination address of the original message is not local, judging whether the message is repeated by the message type and the hash code of the message content, performing message aggregation on the repeated message, the message aggregation comprising counting the message identification of the repeated message and determining the forwarding server list of the aggregated message, and sending the aggregated message to the remote server closest to the local server in the server list; the server list comprising the servers associated with the destination address obtained based on the message identification according to a mapping table, and the mapping table comprising the correspondence between the clients and the servers; the message aggregation module is also used to de-aggregate the aggregated message, obtain the message with the destination address being local, and send the message to the destination address by the message sending module; after deleting the local server address in the server list from the aggregated message, send the aggregated message to the other remote servers in the server list by the message sending module.

4. The dynamic, aggregated message distribution system of claim 3, wherein, The original message includes a message ID, a flag block, a source address, a destination address, message content and a hash code, and the aggregated message includes a message ID list, a flag block, a source address list, a destination address list, a forwarding path, message content and a hash code; the flag block is used to describe the aggregation type and the message type of the message, the aggregation type includes un-aggregated and aggregated, and the message type includes a general message, a client registration message, a group management message and a server registration message.

5. The dynamic, aggregated message distribution system of claim 4, wherein, The server node further includes a registration management module, which is used to maintain a global or local mapping table including a client-server mapping table, a client-group mapping table, a group-server mapping table and a server-server mapping table; The client-server mapping table is used to record the server nodes registered by the client nodes, the client-group mapping table is used to record the client nodes included in the group, and the group-server mapping table is used to record the server node list registered by the client nodes belonging to the group; The server-server mapping table is used to record the network cascade state between the server nodes.

6. The dynamic, aggregated message distribution system of claim 5, wherein, The determination of the forwarding server list of the aggregated message specifically includes determining the target client receiving the message according to the destination address in the statistical original message, determining the group to which the target client belongs based on the client-group mapping table, determining the server node list receiving the message based on the client-server mapping table and the group-server mapping table, and determining the server node cascaded with the local server node in the server node list based on the server-server mapping table.

7. The dynamic, aggregated message distribution system of claim 6, wherein, The client registration message is used by the client node to provide its identification information and group information to the server node when the client node registers to the server node, the server node updates the mapping table according to the client registration message, then generates an updated mapping table message, aggregates the updated mapping table message, and sends the aggregated message to other server nodes; The group management message is used by the client node to send the client node information and group member information to the registered server node when the client node adds or changes a group, the server node updates the mapping table according to the group management message, then generates an updated mapping table message, aggregates the updated mapping table message, and sends the aggregated message to other server nodes; The server registration message is used by the server node to register to the cascaded server node, the cascaded server node updates the mapping table according to the server registration message, then generates an updated mapping table message, aggregates the updated mapping table message, and sends the aggregated message to other server nodes.

8. The dynamic, aggregated message distribution system of claim 3, wherein, The message buffer module is used for buffering received and to-be-sent messages, and includes a receiving buffer and a sending buffer. The receiving buffer is responsible for managing received messages, and the processed messages are buffered in the sending buffer and sent to the next node by the message sending module. The receiving buffer and the sending buffer both include a local client buffer and a remote server buffer, which are used for storing received and to-be-sent messages of the local client and the remote server respectively.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the steps of the method of claim 1 or 2.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of claim 1 or 2.

Images