Dynamic aggregation message distribution method and system

Abstract

The invention discloses a dynamic aggregation message distribution method and system, and the method comprises the steps: judging a destination address of an original message in a received message, and directly transmitting the received message if the destination address is local; if the destination address is not local, whether the messages are repeated or not is judged according to message types and contents, message aggregation is carried out on the repeated messages, and the message aggregation comprises statistics of message identifiers of the repeated messages and determination of a forwarding server list of the aggregated messages; sending the aggregation message to a remote server closest to the local server in a server list; deaggregating the aggregated message in the received message, and sending the message of which the destination address is local to the destination address; and after deleting the local address in the server list, sending the aggregated message to other remote servers. Through accurate calculation of the forwarding server node set which only needs to receive the message, forwarding of the group message to the irrelevant server is reduced, so that redundant network flow is greatly reduced, and precious bandwidth resources are saved.

Description

TECHNICAL FIELD

[0001] The present application relates to a distributed information system, in particular to a dynamic aggregation message distribution method and system. BACKGROUND

[0002] With the rapid development of Internet technology, distributed systems are widely used in various large-scale application scenarios due to their high availability, scalability and fault tolerance. The message distribution system is responsible for reliable and efficient message transmission between different nodes and clients, and is a widely used application system.

[0003] In the existing distributed message distribution system, when the client needs to send a group message, a common method is to send the message to the server it registers, and then broadcast the message to all other servers in the system or forward it according to some preset routing rules. However, this approach has significant drawbacks: first, bandwidth waste: if the member clients in the group are only distributed in a few servers, broadcasting the message to all servers or too many unnecessary servers will result in a large amount of redundant message transmission, which seriously wastes network bandwidth; second, low efficiency: redundant message forwarding increases the possibility of network congestion, prolongs the end-to-end delay of messages, and reduces the overall processing efficiency of the system; third, poor adaptability: it is difficult to quickly adapt to topology adjustments such as server node joining and exiting, which may cause message loss or repeated sending.

[0004] Therefore, how to achieve accurate and non-repetitive forwarding of group messages in a distributed environment with dynamic changes in server network topology, in order to save bandwidth and improve message distribution efficiency, is a technical problem that needs to be solved. SUMMARY

[0005] The present application provides a dynamic aggregation message distribution method and system that reduces the repeated forwarding of messages between servers, thereby effectively saving network bandwidth and improving the timeliness of message transmission and the overall performance of the system.

[0006] Technical solution: To solve the above problems, the present application adopts a dynamic aggregation message distribution method, comprising the following steps:

[0007] Receiving a message, the message including an original message and an aggregated message;

[0008] The destination address of the original message is judged, if the destination address of the original message is in the local, the original message is directly sent to the destination address; if the destination address of the original message is not in the local, whether the message is repeated is judged through the message type and the hash code of the message content, message aggregation is carried out for the repeated message, the message aggregation comprises that the message identification of the repeated message is counted and the forwarding server list of the aggregated message is determined, and the aggregated message is sent to the remote server closest to the local server in the server list; the server list comprises the server associated with the destination address obtained based on the message identification according to a mapping table, and the mapping table comprises the corresponding relationship between the client and the server.

[0009] The aggregated message is de-aggregated, the message whose destination address is in the local is acquired, and the message is sent to the destination address; after the local server address in the server list in the aggregated message is deleted, the aggregated message is sent to the other remote servers in the server list.

[0010] Further, the original message comprises a message ID, a flag block, a source address, a destination address, message content and a hash code, the aggregated message comprises a message ID list, a flag block, a source address list, a destination address list, a forwarding path, message content and a hash code; the flag block is used for describing the aggregation type and the message type of the message, the aggregation type comprises non-aggregation and aggregation, and the message type comprises a general message, a client registration message, a group management message and a server registration message.

[0011] The dynamic aggregated message distribution system comprises a client node and a server node, the server node comprises a message receiving module, a message sending module and a message aggregation module.

[0012] The message receiving module is used for receiving the message sent by other server nodes or client nodes, and the message comprises an original message and an aggregated message.

[0013] The message aggregation module is used for judging the destination address of the original message, if the destination address of the original message is in the local, the original message is directly sent to the destination address through the message sending module; if the destination address of the original message is not in the local, whether the message is repeated is judged through the message type and the hash code of the message content, message aggregation is carried out for the repeated message, the message aggregation comprises that the message identification of the repeated message is counted and the forwarding server list of the aggregated message is determined, and the aggregated message is sent to the remote server closest to the local server in the server list; the server list comprises the server associated with the destination address obtained based on the message identification according to a mapping table, and the mapping table comprises the corresponding relationship between the client and the server.

[0014] The message aggregation module is further configured to de-aggregate the aggregated message, obtain a message with a local destination address, and send the message to the destination address through the message sending module; after deleting a local server address in a server list in the aggregated message, the aggregated message is sent to other remote servers in the server list through the message sending module.

[0015] Further, the original message includes a message ID, a flag block, a source address, a destination address, message content, and a hash code, and the aggregated message includes a message ID list, a flag block, a source address list, a destination address list, a forwarding path, message content, and a hash code; the flag block is used to describe an aggregation type and a message type of the message, the aggregation type includes un-aggregated and aggregated, and the message type includes a general message, a client registration message, a group management message, and a server registration message.

[0016] Further, the server node further includes a registration management module, and the registration management module is configured to maintain a global or local mapping table, the mapping table including a client-server mapping table, a client-group mapping table, a group-server mapping table, and a server-server mapping table; the client-server mapping table is used to record server nodes registered by a client node, the client-group mapping table is used to record client nodes included in a group, the group-server mapping table is used to record a server node list registered by a server node of the group, and the server-server mapping table is used to record a network cascade state between server nodes.

[0017] Further, the determination of the forwarding server list of the aggregated message specifically includes determining a target client receiving the message according to a destination address in the original message, determining a group to which the target client belongs based on the client-group mapping table, determining a server node list receiving the message based on the client-server mapping table and the group-server mapping table, and determining a server node cascaded with a local server node in the server node list based on the server-server mapping table.

[0018] Further, the client registration message is used for a client node to provide its identification information and group information to a server node when the client node registers to the server node, the server node updates the mapping table according to the client registration message, generates an updated mapping table message, aggregates the updated mapping table message, and sends the aggregated message to other server nodes.

[0019] The group management message is used for sending client node information and group member information to the registered server node by the client node when adding or changing a group, the server node updates the mapping table according to the group management message, then generates an update mapping table message, aggregates the update mapping table message, and sends the aggregated message to other server nodes.

[0020] The server registration message is used for registering the cascade server node by the server node, the cascade server node updates the mapping table according to the server registration message, then generates an update mapping table message, aggregates the update mapping table message, and sends the aggregated message to other server nodes.

[0021] Further, the message cache module is further included, and the message cache module is used for caching received and to-be-sent messages; the message cache module includes a receiving cache and a sending cache, the receiving cache is responsible for managing received messages, and after the messages are processed by the message aggregation module, the messages are cached into the sending cache and sent to the next node by the message sending module; the receiving cache and the sending cache both include a local client cache and a remote server cache, and are used for storing local client received and to-be-sent messages and remote server received and to-be-sent messages.

[0022] The application also adopts a computer device, including a memory, a processor and a computer program stored on the memory and capable of running on the processor, and the processor implements the steps of the above method when executing the computer program.

[0023] The application also adopts a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to implement the steps of the above method.

[0024] Advantages: compared with the prior art, the application has the following advantages: first, the bandwidth is significantly saved, the forwarding server node set for receiving messages is accurately calculated, unnecessary forwarding of group messages to irrelevant servers is reduced, thereby greatly reducing redundant network traffic and saving valuable bandwidth resources; second, the message distribution efficiency is improved, and less redundant forwarding means less network delay and higher throughput, messages can reach target member clients faster, and user experience and overall response speed of the system are improved; third, the dynamic adaptability is strong, the system can real-time or near real-time perceive changes of client registered servers and changes of network topologies between servers, and dynamically adjusts the message forwarding strategy accordingly, thereby ensuring reliable and efficient message transmission in a complex dynamic environment; fourth, the scalability is good, since unnecessary communication between servers is reduced, the system can more easily expand the number of server nodes and support more clients and groups, and will not cause performance bottlenecks due to excessive internal communication overhead. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall framework of the dynamic aggregated message distribution system in this invention.

[0026] Figure 2 This is a schematic diagram of the client registration process in this invention.

[0027] Figure 3 This is a schematic diagram of the group management process in this invention.

[0028] Figure 4 This is a schematic diagram of the server registration process in this invention.

[0029] Figure 5 This is a schematic diagram of message caching in this invention.

[0030] Figure 6 This is a schematic diagram of the message aggregation process in this invention.

[0031] Figure 7 This is a schematic diagram of the message de-aggregation process in this invention. Detailed Implementation

[0032] like Figure 1 As shown, this embodiment of a dynamic aggregated message distribution system includes several server nodes and several client nodes. The server nodes can adapt to dynamically changing network topologies, and the client nodes can send and receive messages by registering with any server node. Each server node includes a message receiving module, a message caching module, a message sending module, a message aggregation module, and a registration management module.

[0033] The message receiving module receives messages sent by other server nodes or client nodes. Messages include raw messages and aggregated messages. Raw message content includes message ID, flag block, source address, destination address, message content, and hash code. Aggregated message content includes a list of message IDs, flag block, source address list, destination address list, forwarding path, message content, and hash code. The flag block describes the aggregation type and message type. Aggregation types include {NON, ZIP}, corresponding to unaggregated and aggregated categories. Message types include {NML, CREG, GMNG, SREG}, corresponding to general messages, client registration messages, group management messages, and server registration messages. The message aggregation module supports message de-aggregation based on flag block information, restoring aggregated messages to their original form. General messages, client registration messages, and group management messages are generated and sent by the client, while server registration messages are generated and sent by the server.

[0034] The message caching module is used to cache received and pending messages. The message caching module includes a receive cache and a send cache. Both the receive cache and the send cache include a local client cache and a remote server cache, which are used to store messages received and pending to be sent by the local client and the remote server, respectively. The receive cache also includes an offline client message cache, which caches messages received by the currently offline client. Once the user logs in online, the messages are migrated to the send cache for sending.

[0035] The message aggregation module is used to aggregate repeatedly sent messages and also to de-aggregate aggregated messages. Messages in the receive buffer are processed by the message aggregation module and then cached in the send buffer, and then sent to the next node by the message sending module.

[0036] The registration management module manages the network topology between clients, groups, and servers, and maintains global or local mapping tables. The module supports updating these mapping tables. The mapping tables include client-server, client-group, group-server, and server-server mapping tables. The system updates these mapping tables by sending client registration messages, group management messages, and server registration messages.

[0037] Client-Server Mapping Table Used to record the servers registered by the currently logged-in user's client. This indicates that client i is registered with server j; otherwise... .

[0038] Client-Group Mapping Table Used to record the list of clients included in the group. This indicates that client i belongs to group j; otherwise... ;

[0039] Group-Server Mapping Table Used to record the list of servers registered by the clients belonging to the group. This indicates that clients belonging to group i are registered with server j; otherwise... ;

[0040] Server-to-Server Mapping Table Used to record the network cascading status between servers. This indicates that server i and server j are cascaded; otherwise... .

[0041] The registration management module supports synchronization of the inter-server mapping table. For a received client registration message, group management message, and server registration message, the registration management module updates the local mapping table. If the mapping table is updated, the updated message is cached to the local client cache of the receiving cache. The message aggregation module aggregates the updated messages that need to be aggregated, and transmits the updated values to all reachable server nodes. If there is no update, no message is sent.

[0042] As shown in Figure 2 , the client registration message is provided by a client to a server node when the client registers to the server node, and provides the server node with its identification information and group information (if any). The server node updates the mapping relationship table according to the information provided by the client, and updates the server node, and sends the client registration message {CREG, } to the reachable server nodes.

[0043] As shown in Figure 3 , the group management message is sent by a client to a registered server node when the client adds or changes a group, and sends the server node the client information, group information, and group member set information. The server node updates the associated mapping table, and sets ; and sends the group management message {GMNG, } to the reachable server nodes.

[0044] As shown in Figure 4 , the server registration message is sent by a new server node to an existing server node , and updates the associated mapping table ; and sends the server registration message {SREG, } to the reachable server nodes.

[0045] The registration management module also periodically detects the online status of the client, updates the associated mapping relationship table , and sends the client registration message {CREG, } to the reachable server nodes, and updates the associated mapping table.

[0046] Other server nodes or client nodes send original messages or aggregated messages to the server node, such as Figure 5 ​​​As shown, when the message receiving module of the service node receives the original message sent by the local client, the message is stored in the local client cache of the receiving cache; when the message receiving module of the service node receives the aggregated message sent by the remote server, the message is stored in the remote server cache of the receiving cache.

[0047] As shown in the figure, the message aggregation module judges the original message in the local client cache of the receiving cache, if the client of the destination address of the original message is in the local, it will not be aggregated and directly cached to the local client cache of the sending cache, if the client of the destination address of the original message is registered in other server nodes, it needs to be aggregated. The message aggregation module judges whether the original message cached in the local client cache needs to be aggregated by judging the consistency of the message type and the hash code, judges whether the message is repeated, for repeated messages, reduces repeated sending by generating aggregated messages, for non-repeated messages, directly caches to the remote server cache of the sending cache, and the message aggregation module also stores the aggregated message in the remote server cache of the sending cache. Figure 6 As shown in the figure, the message aggregation module judges the original message in the local client cache of the receiving cache, if the client of the destination address of the original message is in the local, it will not be aggregated and directly cached to the local client cache of the sending cache, if the client of the destination address of the original message is registered in other server nodes, it needs to be aggregated. The message aggregation module judges whether the original message cached in the local client cache needs to be aggregated by judging the consistency of the message type and the hash code, judges whether the message is repeated, for repeated messages, reduces repeated sending by generating aggregated messages, for non-repeated messages, directly caches to the remote server cache of the sending cache, and the message aggregation module also stores the aggregated message in the remote server cache of the sending cache.

[0048] Figure 7 As shown in the figure, the message aggregation module judges the original message in the local client cache of the receiving cache, if the client of the destination address of the original message is in the local, it will not be aggregated and directly cached to the local client cache of the sending cache, if the client of the destination address of the original message is registered in other server nodes, it needs to be aggregated. The message aggregation module judges whether the original message cached in the local client cache needs to be aggregated by judging the consistency of the message type and the hash code, judges whether the message is repeated, for repeated messages, reduces repeated sending by generating aggregated messages, for non-repeated messages, directly caches to the remote server cache of the sending cache, and the message aggregation module also stores the aggregated message in the remote server cache of the sending cache.

[0049] For example: the original message content includes {msgID, TYPE, src, dst, data, hash}, where msgID represents message ID, TYPE represents flag block (including aggregation type, message type), src represents source address (including client ID or group ID), dst represents destination address (including client ID or group ID), data represents message content, and hash represents hash code (calculated from message content).

[0050] The aggregated message content includes {msgIDList, TYPE, srcList, dstList, routeList, data, hash}, where msgIDList represents message ID list, srcList represents source address list, dstList represents destination address list, and routeList represents server list, which is calculated by the message aggregation module according to the mapping table. ​

[0051] For original messages msg1={msgID1,TYPE,src1,dst1,data,hash} and msg2{msgID2,TYPE,src2,dst2,data,hash}, when the message type TYPE and hash code hash are consistent, aggregation can be performed, and the aggregated message zipMSG of

[0052] msgIDList={msgID1, msgID1}

[0053] srcList={src1,src2}

[0054] dstList={dst1,dst2}

[0055] Finally, according to the client IDs and group IDs in the dstList, in combination with the client-server mapping table and the group-server mapping table , the remote message receiving server list (not including the local server) is obtained, and the routeList={ } is obtained. To avoid repeated aggregation of aggregated messages, resulting in a non-convergent hand chain, the system only supports aggregation of multiple original messages.

[0056] After message aggregation is completed, the zipMSG is saved to the remote server cache of the sending cache, and for offline users, the message is saved to the offline user message cache, and then forwarded after the user is online.

[0057] The message sending module obtains the message from the local client cache of the sending cache and sends it to the client node of the destination address registered to the local server.

[0058] The message sending module obtains the aggregated message from the remote server cache of the sending cache, according to the server-server mapping table , the local server is , according to the routeList={ }, the aggregated message is sent to the server , wherein , , represents the distance to .

[0059] For the server received aggregated message, the message receiving module is cached to the receiving cache of the remote server cache. The message aggregation module obtains the aggregated message from the receiving cache of the remote server cache, and sends the part to be sent to the local client to the local client after de-aggregation, and continues to forward the remaining part to other remote servers. Specifically:

[0060] For the received aggregated message zipMSG={ msgIDList,TYPE,srcList,dstList,route,data,hash}, the message aggregation module judges whether the local client is contained in the dstList according to the client-server mapping table and the group-server mapping table , if yes, the message de-aggregation is performed to obtain one or more msg={msgID,TYPE,src,dst,data,hash}, and save to the sending cache of the local client cache, and the message sending module obtains and sends to the corresponding local client from the cache.

[0061] Meanwhile, the routeList={ } is set. Among them represents the local server node, if the routeList is empty, it means that it does not need to be sent, if it is not empty, the revised aggregated message is saved to the sending cache of the remote server cache, the message sending module obtains the aggregated message from the sending cache of the remote server cache, and sends the aggregated message to according to the server-server mapping table Among them , , represents the distance to .

[0062] For the offline user message cache, a trigger is set to listen to the user-server mapping table When the mapping table changes, when the user is detected to be online, the message sending is restarted.

Claims

1. A dynamic aggregation message distribution method, characterized in that, Includes the following steps: Receive messages, including raw messages and aggregated messages; Determine the destination address of the original message. If the destination address of the original message is local, send it directly to the destination address. If the destination address of the original message is not local, the message type and hash code of the message content are used to determine whether the message is duplicated. For duplicate messages, message aggregation is performed. Message aggregation includes counting the message identifiers of duplicate messages and determining the forwarding server list of the aggregated message. The aggregated message is sent to the remote server in the server list that is closest to the local server. The server list includes servers associated with the destination address based on the message identifier according to a mapping table. The mapping table includes the correspondence between the client and the server. De-aggregate the aggregated messages, obtain the messages destined for the local address, and send the messages to the destination address; After deleting the local server address from the server list in the aggregate message, the aggregate message is sent to other remote servers in the server list.

2. The dynamic aggregation message distribution method according to claim 1, characterized in that, The original message includes a message ID, a flag block, a source address, a destination address, a message content, and a hash code. The aggregated message includes a list of message IDs, a flag block, a list of source addresses, a list of destination addresses, a forwarding path, a message content, and a hash code. The flag block is used to describe the aggregation type and message type of the message. The aggregation type includes unaggregated and aggregated. The message type includes general messages, client registration messages, group management messages, and server registration messages.

3. A dynamic aggregated message distribution system, characterized in that, It includes several client nodes and several server nodes, wherein the server nodes include a message receiving module, a message sending module and a message aggregation module; The message receiving module is used to receive messages sent by other server nodes or client nodes, including original messages and aggregated messages; The message aggregation module is used to determine the destination address of the original message. If the destination address of the original message is local, it is directly sent to the destination address through the message sending module. If the destination address of the original message is not local, it determines whether the message is duplicated by using the message type and the hash code of the message content. For duplicate messages, message aggregation is performed. Message aggregation includes counting the message identifiers of duplicate messages and determining the forwarding server list of the aggregated messages. The aggregated message is sent to the remote server in the server list that is closest to the local server. The server list includes servers associated with the destination address based on the message identifier according to a mapping table. The mapping table includes the correspondence between the client and the server. The message aggregation module is also used to de-aggregate the aggregated messages, obtain messages with destination addresses on the local machine, and send the messages to the destination addresses through the message sending module; after deleting the local server addresses from the server list in the aggregated messages, the aggregated messages are sent to other remote servers in the server list through the message sending module.

4. The dynamic aggregated message distribution system according to claim 3, characterized in that, The original message includes a message ID, a flag block, a source address, a destination address, a message content, and a hash code. The aggregated message includes a list of message IDs, a flag block, a list of source addresses, a list of destination addresses, a forwarding path, a message content, and a hash code. The flag block is used to describe the aggregation type and message type of the message. The aggregation type includes unaggregated and aggregated. The message type includes general messages, client registration messages, group management messages, and server registration messages.

5. The dynamic aggregated message distribution system according to claim 4, characterized in that, The server node also includes a registration management module, which is used to maintain global or local mapping tables, including client-server mapping tables, client-group mapping tables, group-server mapping tables, and server-server mapping tables. The client-server mapping table records the server nodes registered by the client nodes, the client-group mapping table records the client nodes contained in the group, and the group-server mapping table records the list of server nodes registered by the client nodes belonging to the group. The server-to-server mapping table is used to record the network cascading status between server nodes.

6. The dynamic aggregated message distribution system according to claim 5, characterized in that, The process of determining the list of forwarding servers for the aggregated messages specifically includes: determining the target client receiving the messages based on the destination address in the original messages; determining the group to which the target client belongs based on the client-group mapping table; determining the list of server nodes receiving the messages based on the client-server mapping table and the group-server mapping table; and determining the server nodes in the server node list that are cascaded with the local server node based on the server-server mapping table.

7. The dynamic aggregated message distribution system according to claim 6, characterized in that, The client registration message is sent by the client node when it registers with the server node, providing the server node with its identification information and group information. The server node updates the mapping table according to the client registration message, then generates an update mapping table message, aggregates the update mapping table messages, and sends the aggregated messages to other server nodes. The group management message is sent by the client node when adding or changing a group. The client node sends the client node information and group member information to the registered server node by sending the group management message. The server node updates the mapping table according to the group management message, then generates an update mapping table message, aggregates the update mapping table messages, and sends the aggregated message to other server nodes. The server registration message is sent by a server node to register with cascaded server nodes. The cascaded server nodes update the mapping table according to the server registration message, then generate an update mapping table message, aggregate the update mapping table messages, and send the aggregated message to other server nodes.

8. The dynamic aggregated message distribution system according to claim 3, characterized in that, It also includes a message caching module, which is used to cache received and pending messages. The message caching module includes a receive cache and a send cache. The receive cache is responsible for managing the received messages. After being processed by the message aggregation module, the messages are cached in the send cache and then sent to the next node by the message sending module. Both the receive cache and the send cache include a local client cache and a remote server cache, which are used to store messages received and pending to be sent by the local client and messages received and pending to be sent by the remote server, respectively.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 1 or 2.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1 or 2.

Images