Subscription processing method, device, and storage medium

Abstract

The application provides a subscription processing method, device and storage medium, comprising the following steps: when a first network node is online, a subscription engine management class in the first network node acquires all subscriptions in the first network node after deduplication, and a subscription engine management class in all target network nodes acquires all subscriptions in the target network nodes after deduplication; the first network node sends all subscriptions in the first network node to the target network nodes, and all target network nodes send all subscriptions in all target network nodes to the first network node; the subscription engine management class in all target network nodes resets the subscriptions of all subscribers in all target network nodes based on all subscriptions in the first network node, so as to cover the original subscriptions; and the subscription engine management class in the first network node resets the subscriptions of all subscribers in the first network node based on all subscriptions in all target network nodes. The application can improve the processing speed of the distributed information system subscription.

Description

Technical Field

[0001] This invention relates to the field of distributed information system technology, and in particular to a subscription processing method, device and storage medium. Background Technology

[0002] With the rapid development of computer technology, the functions and structures of information systems are becoming increasingly complex, which places higher demands on message transmission. To ensure fast and stable message transmission, message middleware has emerged.

[0003] Message middleware, also known as message bus, message queue, or message system, typically employs a topic-subscription pattern for message distribution to decouple message producers (the senders) from message consumers (the receivers). In this pattern, each message has a topic, which is an attribute of the message and is set by the message producer when sending the message based on its business meaning. Message subscribers can set any number of subscriptions according to their business needs. If the topic of a message sent by the message producer matches any of the subscriber's subscriptions, the subscriber receives the message.

[0004] However, the subscription processing speed between message middleware in current distributed messaging systems is slow and cannot meet the requirements of distributed information systems. Summary of the Invention

[0005] This invention provides a subscription processing method, device, and storage medium to solve the problem of slow subscription processing speed between message middleware in existing distributed messaging systems.

[0006] In a first aspect, embodiments of the present invention provide a subscription processing method. The method is applied to a distributed messaging system, which includes multiple network nodes, each with interconnected message middleware. Each message middleware includes a subscription management class, a subscriber management class, and a subscription engine management class. The subscription management class stores the subscriber encoding bitmap for each subscription, the subscriber management class stores the subscription encoding bitmap for each subscriber, and the subscription engine management class is responsible for subscription management and matching within the local network node, as well as subscription management and matching with remote network nodes. Both the subscriber encoding bitmap and the subscription encoding bitmap are highly compressed bitmaps. The method is applicable to each network node in the distributed messaging system, including:

[0007] When the first network node comes online, the subscription engine management class within the first network node obtains all subscriptions within the first network node after deduplication, and the subscription engine management classes within all target network nodes obtain all subscriptions within the target network nodes after deduplication; where the target network node is any network node that is working normally in the distributed messaging system.

[0008] The first network node sends all subscriptions within the first network node to the target network node, and all target network nodes send all subscriptions within all target network nodes to the first network node.

[0009] The subscription engine management class in all target network nodes resets the subscriptions of subscribers in all target network nodes based on all subscriptions in the first network node, overriding their original subscriptions; the subscription engine management class in the first network node resets the subscriptions of subscribers in the first network node based on all subscriptions in all target network nodes, overriding their original subscriptions.

[0010] When the target network node detects that the first network node is offline, the subscription engine management class in the target network node resets the subscriptions of the subscribers in the target network node and removes the subscribers in the first network node from the subscriptions in the target network node.

[0011] In some implementations, the message middleware includes a local process subscription management module and a remote node subscription management module;

[0012] The subscription engine management class within the first network node retrieves all deduplicated subscriptions within the first network node, including:

[0013] The subscription management module within the local process of the first network node or the subscription engine management class within the remote node's subscription management module iterates through the subscription information index and adds the subscriptions from the subscription management class within the first network node to the subscription encoding index in the output parameters of the first network node; or

[0014] The subscription management module of the local process within the first network node or the subscription engine management class of the remote node subscription management module traverses the subscription encoding index, skips null pointers, and adds the subscriptions in the subscription management class within the first network node to the subscription encoding index in the output parameters of the first network node.

[0015] In some implementations, the subscription engine management class in all target network nodes resets the subscriptions of subscribers in all target network nodes based on all subscriptions within the first network node, overriding their original subscriptions, including:

[0016] The subscription engine management class in the target network node looks up the target subscriber based on the target subscriber name and the subscriber name index;

[0017] The subscription engine management class in the target network node creates a temporary subscription encoding bitmap for the target subscriber, retrieves the target subscription from all subscriptions in the first network node, and searches for all target subscriptions in the subscription information index in the target network node.

[0018] The subscription engine management class in the target network node adds the target subscriber's code to the subscriber code bitmap of all target subscriptions, and adds the codes of all target subscriptions to the temporary subscription code bitmap, resulting in the added temporary subscription code bitmap;

[0019] The subscription engine management class in the target network node determines the codes of the subscriptions to be deleted based on the added temporary subscription code bitmap and the original subscription code bitmap of the target subscriber.

[0020] The subscription engine management class in the target network node obtains instance pointers of all target subscriptions based on the subscription encoding index, and removes the encoding of the target subscriber from the subscription encoding bitmap of all target subscriptions;

[0021] When the subscription engine management class in the target network node determines that the added temporary subscription code bitmap is a non-empty set, it assigns the added temporary subscription code bitmap to the original subscription code bitmap of the target subscriber.

[0022] When the subscription engine management class in the target network node determines that the added temporary subscription encoding bitmap is an empty set, it removes the target subscriber name from the subscriber name index.

[0023] In some implementations, the subscription engine management class in the target network node looks up the target subscriber based on the target subscriber name and subscriber name index, including:

[0024] When the subscription engine management class in the target network node does not find a subscriber name that matches the target subscriber name in the subscriber name index, it checks whether there is an unused subscriber code in the queue of unused subscriber codes.

[0025] When the subscription engine management class in the target network node determines that there are unused subscriber codes, it selects a code from the queue of unused subscriber codes as the target subscriber code, assigns the subscriber management class pointer constructed from the target subscriber code to the element in the subscriber code index with the target subscriber code as the subscript, and adds the target subscriber name to the subscriber name index.

[0026] When the subscription engine management class in the target network node determines that there are no unused subscriber codes, it constructs a subscriber management class using the number of elements in the subscriber code index as the target subscriber's code, adds a pointer to the subscriber management class to the end of the subscriber code index, and adds the target subscriber name to the subscriber name index.

[0027] In some implementations, all target subscriptions are retrieved from the subscription information index in the target network node, including:

[0028] When the target subscription is not found in the subscription engine management class of the target network node, it is determined whether there is an unused subscription code in the queue of unused subscription codes;

[0029] When the subscription engine management class in the target network node determines that there is an unused subscription code, it selects a code from the queue of unused subscription codes as the target subscription code, and assigns the subscription management class pointer constructed from the target subscription code to the element in the subscription code index with the target subscriber's code as the subscript.

[0030] When the subscription engine management class in the target network node determines that there is no unused subscription code, it uses the number of elements in the subscription code index as the target subscription code, constructs a subscription management class with the target subscription code, and adds a pointer to the subscription management class to the end of the subscription code index.

[0031] The subscription engine management class in the target network node adds the target subscription information to the subscription information index;

[0032] The subscription engine management class in the target network node obtains the bitmap encoding that matches all target segments of the target subscription information in the subscriber bitmap index. If the target subscriber's encoding is not found in the bitmap encoding, the target subscriber's encoding is added to the bitmap encoding. Here, the target segment is any topic segment of the target subscriber's subscription information.

[0033] In some implementations, the method further includes:

[0034] When the target network node receives a request from the first subscriber to add first subscription information, the target network node's subscription engine management class looks up the first subscriber name in the subscriber name index; where the first subscriber name is the name of the first subscriber.

[0035] If the target network node's subscription engine management class does not find a subscriber name that matches the first subscriber name, then add the first subscriber name to the subscriber name index.

[0036] When the subscription engine management class of the target network node finds or adds the first subscriber name, it searches for the first subscription in the subscription information index; where the first subscription is the subscription that matches the first subscription information.

[0037] When the target network node's subscription engine management class finds the first subscription, it sets the return value to the first preset value; when the first subscription is not found, it adds the first subscription and sets the return value to the second preset value; when the target network node's subscription engine management class finds or adds the first subscription, it adds the first subscription's code to the subscriber code bitmap of the first subscription information, adds the first subscriber's code to the subscription code bitmap of the first subscription information, and returns the return value; where the first preset value is used to indicate that the overall subscription has not changed, and the second preset value is used to indicate that the overall subscription has changed.

[0038] In some implementations, the method further includes:

[0039] When the subscription engine management class of the target network node receives a request from the second subscriber to delete the second subscription information, it searches for the second subscriber name in the subscriber name index; where the second subscriber name is the name of the second subscriber.

[0040] When the target network node's subscription engine management class determines that the second subscriber name does not exist, it sets the return value to the first preset value.

[0041] When the subscription engine management class of the target network node determines that the second subscriber name exists, it searches for the second subscription in the subscription information index; where the second subscription is the subscription corresponding to the second subscription information.

[0042] When the target network node's subscription engine management class determines that the second subscription does not exist, it sets the return value to the first preset value.

[0043] When the target network node's subscription engine management class finds the second subscription, it deletes the second subscription's code from the second subscriber's subscription code bitmap;

[0044] When the subscription engine management class of the target network node determines that the subscription code bitmap of the second subscriber after deletion is an empty set, it sets the element with the second subscriber's code as the index in the subscriber code index to a null pointer, adds the second subscriber's code to the queue of unused subscriber codes, deletes the second subscriber's name from the subscriber name index, and deletes the second subscriber's code from the second subscriber's subscriber code bitmap.

[0045] When the target network node's subscription engine management class determines that the second subscriber's subscription code bitmap after deletion is a non-empty set, it deletes the second subscriber's code from the second subscriber's subscriber code bitmap and returns the return value; where the first preset value is used to indicate that the overall subscription has not changed, and the second preset value is used to indicate that the overall subscription has changed.

[0046] In a second aspect, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation thereof.

[0047] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation thereof.

[0048] This invention provides a subscription processing method, device, and storage medium. The method is applicable to each network node in a distributed messaging system and includes: First, when a first network node comes online, the subscription engine management class within the first network node obtains all deduplicated subscriptions within the first network node, and the subscription engine management classes within all target network nodes obtain all deduplicated subscriptions within the target network nodes. Then, the first network node sends all subscriptions within its own network to the target network nodes, and all target network nodes send all subscriptions within their respective target network nodes back to the first network node. Next, the subscription engine management classes in all target network nodes reset the subscriptions of subscribers in all target network nodes based on all subscriptions within the first network node, overwriting their original subscriptions; the subscription engine management class in the first network node resets the subscriptions of subscribers in the first network node based on all subscriptions within the target network nodes, overwriting their original subscriptions. Finally, when a target network node detects that the first network node is offline, the subscription engine management class in the target network node resets the subscriptions of subscribers in the target network node and deletes the subscribers from the first network node's subscriptions. This invention improves space utilization and reduces memory waste by using efficient compressed bitmaps. By using deduplication when nodes are offline or online, all local subscriptions can be quickly retrieved without the tedious step-by-step collection of each subscription, thereby improving the subscription processing speed of the distributed messaging system. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the implementation of the distributed bitmap matching method provided in this embodiment of the invention.

[0051] Figure 2 This is a flowchart illustrating the implementation of the subscription processing method provided in this embodiment of the invention.

[0052] Figure 3 This is a flowchart of the implementation of setSubtionOfSuber for setting a subscription for a subscriber, overriding their original subscription, provided by an embodiment of the present invention;

[0053] Figure 4 This is a flowchart illustrating the implementation of the newSuber sub-process for adding a new subscriber provided in this embodiment of the invention.

[0054] Figure 5 This is a flowchart illustrating the implementation of the newSubtion sub-process for adding a new subscription, as provided in this embodiment of the invention.

[0055] Figure 6 This is a flowchart illustrating the implementation of addSubtionOfSuber for adding a subscription to a subscriber, provided in an embodiment of the present invention.

[0056] Figure 7 This is a flowchart illustrating the implementation of delSubtionOfSuber for deleting a subscription, provided by an embodiment of the present invention.

[0057] Figure 8 This is a flowchart illustrating the implementation of delSuberFromSubtion for deleting a subscription, provided by an embodiment of the present invention.

[0058] Figure 9 This is a flowchart illustrating the implementation of delSuber for deleting subscribers, provided in an embodiment of the present invention.

[0059] Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0062] As introduced in the background section, various message middleware, also known as message buses, message queues, or message systems, such as RabbitMQ, ActiveMQ, and Kafka, almost all provide a topic-subscription pattern to decouple message producers (sending messages) and consumers (receiving messages). How to quickly manage subscribed messages and match message topics with subscriptions has become one of the key factors affecting the performance of message middleware.

[0063] Messaging systems can be centralized or distributed in structure, with distributed messaging systems further including hierarchical and peer-to-peer structures. For distributed structures, whether hierarchical or peer-to-peer, the subscription management module must possess the following characteristics:

[0064] First, obtain the overall subscriptions of a subscription management module instance (i.e., this level, this node, collectively referred to as local), so that the local whole can be added as a subscriber to the remote end (the upper level, or other peer nodes). There is no need to display the internal subscriber information of the local instance; the subscriptions of all local subscribers are deduplicated. Second, when the subscriptions of local subscribers change (subscriber addition, deletion, resetting subscriptions, deleting subscribers, etc.), return the changes to the overall local subscriptions resulting from this operation, so as to notify the remote end.

[0065] Before introducing the subscription processing method provided by this invention, we will first introduce the segmented bitmap matching method used in this invention.

[0066] A bitmap is a commonly used data structure. Its basic principle is to use a single bit to represent the existence of the corresponding integer. Externally, a bitmap is a container that can store and retrieve integers. For example, adding 3, 4, and 7 to a bitmap internally sets bits 3, 4, and 7 to 1. If you add 3 again, bit 3 will still be 1, so bitmaps inherently have deduplication properties. Furthermore, two bitmaps can be ANDed or ORed. For instance, if a bitmap contains 3, 4, 5, and 8, an AND operation with the previous bitmap will result in 3 and 4; an OR operation will result in 3, 4, 5, 7, and 8.

[0067] like Figure 1 The distributed bitmap matching method shown below has the following specific steps:

[0068] S11. Based on the target segment topic, determine the encoded bitmap that is exactly the same as the target segment topic in the subscriber bitmap index of the target segment subscription information stored in the network node, and determine it as the first encoded bitmap.

[0069] S12. The encoded bitmap that matches any topic in the subscriber bitmap index of the target segment subscription information is determined as the second encoded bitmap.

[0070] S13. Find the union of the first and second coded bitmaps to obtain the coded bitmap that matches the topic of the target segment.

[0071] S14. Find the intersection of all encoded bitmaps that match the topic of the target segment to obtain the subscription encoded bitmap that matches the topic of the message.

[0072] S15. Based on the subscription encoding bitmap, determine the subscribers who subscribe to the message, send the message to the subscribers who subscribe to the message, and convert the subscriber bitmap into a set of output parameter subscriber names.

[0073] in, Figure 2 In the diagram, mapSection1Bm is the subscriber bitmap index of the first subscription information segment, and mapSection2Bm is the subscriber bitmap index of the second subscription information segment.

[0074] For ease of understanding, assume the following subscriptions, where * indicates that the segment matches any: "ab.11", "ab.21", "bb.11", "bb.12", "*.11", "*.*". First, number the subscriptions, since bitmaps can only store integers, as follows: 0: "ab.11", 1: "ab.21", 2: "bb.11", 3: "bb.12", 4: "*.11", 5: "*.*".

[0075] Based on the subscribed segments, construct the number of bitmap groups for each segment. In this example, the subscription has two segments. One bitmap group (each row, excluding the header, is a bitmap) is shown in the table below:

[0076]

[0077]

[0078] The second set of bitmaps is shown in the following figure:

[0079] 11 1 0 1 0 1 0 21 0 1 0 0 0 0 12 0 0 0 1 0 0 * 0 0 0 0 0 1

[0080] If a message has the subject "ab.11", the matching method is as follows:

[0081] First, extract the bitmaps representing ab and * from the first group, and perform an OR operation to obtain the bitmap x representing the first matching segment.

[0082]

[0083] Next, extract the bitmaps representing 11 and * from the second group, and perform OR operation to obtain the bitmap y representing the second matching segment.

[0084]

[0085] Finally, AND x and y to get the matching subscriptions.

[0086]

[0087] The elements in the result are 0, 4, and 5, which represent the matched subscriptions as "ab.11", "*.11", and "*.*".

[0088] Although bitmap matching is faster than trie matching, bitmaps consume a lot of memory. For example, if a very large integer 99999 needs to be put into a bitmap, the bitmap will need to allocate at least 99999 bits of space.

[0089] Furthermore, how to satisfy subscription processing in a distributed messaging system based on bitmap matching and improve the speed of subscription processing has become an urgent problem to be solved.

[0090] To address the problems of existing technologies, embodiments of the present invention provide a subscription processing method, apparatus, device, and storage medium. The subscription processing method provided by these embodiments is described first, and this method is applied to a distributed messaging system.

[0091] The overall structure of the distributed architecture will be described below to facilitate understanding of the entire solution.

[0092] A distributed messaging system consists of multiple network nodes, with message middleware within each node interconnected. Each message middleware includes a subscription management class, a subscriber management class, and a subscription engine management class.

[0093] The subscription management class `SeSubtion` stores the subscriber code bitmap for each subscription. One instance corresponds to one subscription, storing relevant information for that subscription. Each subscription has a unique code, or subscription ID. This ID is an integer starting from 0, used to store the subscriber code bitmap. A subscriber code bitmap `bmSuber` is a bitmap that stores the codes of subscribers who subscribed to that subscription. The subscriber code bitmap is stored using an efficient compressed bitmap called a Roaring bitmap, further avoiding wasted space. Roaring is a memory-saving bitmap optimization implementation.

[0094] The subscriber management class `SeSuber` stores the subscription encoding bitmap for each subscriber. One instance corresponds to one subscriber, storing relevant information for that subscriber. Each subscriber has a unique identifier, also known as its ID, which is an integer starting from 0. This ID is then stored in the subscription encoding bitmap (`BmSubtion`). A `BmSubtion` is a bitmap that stores the IDs of all subscriptions for that subscriber. The subscription encoding bitmap is stored using a highly efficient compressed bitmap (`Roaring bitmap`) format.

[0095] The subscription engine management class is responsible for subscription management and matching within the local network node, as well as subscription management and matching with remote network nodes. One instance of this class is responsible for subscription management and message topic matching for all subscribers locally (within this level and among this peer node). The overall subscription status locally is synchronized to the remote node. Another instance of this class is responsible for subscription management and message topic matching for remote nodes (the upper level and other peer nodes). If a message on the local node matches a subscription on a remote node, it is sent to that remote node.

[0096] For ease of understanding, the accompanying drawings of this invention use some characters instead of commonly used text, such as:

[0097] The unused subscription encoding queue is queUnusedSubtionId, indicating that when a subscription has no subscribers, this subscription management class instance is deleted, and its ID is placed in this queue for use by the next new subscription. This method avoids empty IDs, thereby reducing bitmap memory waste.

[0098] The subscription encoding index is vecSubtion, and a pointer to the subscription management class is stored according to the ID of the subscription management class, so that the instance of the class can be quickly accessed according to its ID.

[0099] The subscription information index is mapSubtion. An index is created based on the subscription information of the subscription management class as the key, and a pointer to the class is stored. The instance of the class can be quickly accessed based on its subscription information.

[0100] The queue of unused subscribers is queUnusedSuberId. When a subscriber stops subscribing, the instance of the subscriber management class is deleted, and its ID is placed in this queue for use by the next new subscriber.

[0101] The subscriber encoding index is vecSuber, and a pointer to the subscriber management class is stored according to its ID, allowing for quick access to the class instance based on its ID. Only memory offset operations are required, making it very fast.

[0102] The subscriber name index is mapSuber, which is created by using the subscriber name of the subscriber management class as the key. The index stores a pointer to the class, and the instance of the class can be quickly accessed based on its subscriber name.

[0103] The first segment of the subscription information has a subscriber bitmap index of mapSection1Bm. An index is built using the first segment of the subscription information as the key, storing (i.e., the value is) a bitmap containing the subscription ID of that segment. This is used for quick access to the bitmap based on the first segment of the subscription information. For example, if the ID of the subscription "aaa.bbb" is 3 (nIndex = 3 for the SeSubtion instance, strSubtion = "aaa.bbb"), and the segment is divided by ".", then the first segment is "aaa". Using this as the key of the map, the corresponding bitmap contains the value 3.

[0104] The subscriber bitmap index for the second subscription information segment is mapSection2Bm.

[0105] Adding a subscription to a subscriber is done using the function `addSubtionOfSuber(const string& strSuber, const string& strSubtion)`. `strSuber` is the subscriber name, and `strSubtion` is the subscription information to be added. The function returns whether it affects global subscriptions for this instance. For example, if the added subscription is not currently subscribed to by any subscriber, it will add the subscription globally and return `true`.

[0106] The function `delSubtionOfSuber(const string& strSuber, const string& strSubtion)` removes a subscription from a user's list of subscribers. `strSuber` is the subscriber name, and `strSubtion` is the subscription information to be removed. The function returns whether it affects global subscriptions for this instance. For example, if the subscription is not subscribed to by any other user after deletion, it is globally deleted, and the function returns `true`.

[0107] To set up a subscription for a subscriber, overriding its existing subscription, use `setSubtionOfSuber(const string&strSuber, const vector)`. <string>&vecSetSubtion,vector <string>&vecAddedSubtion,vector <string>&vecDeledSubtion). strSuber: Input parameter, subscriber name. vecSetSubtion: Input parameter, subscription information to be set, can be 0 or more. vecAddedSubtion: Output parameter, output of additions to the global subscription. vecDeledSubtion: Output parameter, output of deletions to the global subscription.

[0108] Matching subscribers based on message topic: matchSuberOfTopic(const string&strTopic, vector) <string>&vecSuber), strTopic: input parameter, message topic; vecSuber: output parameter, output matched subscribers, 0 to multiple.

[0109] To retrieve all subscriptions of a subscriber, use `getSubtionOfSuber(const string&strSuber, vector)`. <string>&vecSubtion), strSuber: input parameter, subscriber name; vecSubtion: output parameter, outputting all subscription information of the subscriber.

[0110] To retrieve the global subscriptions for this instance, i.e., to retrieve the subscriptions of all subscribers after deduplication, use getSubtionOfAll(

[0111] vector <string>&vecSubtion), vecSubtion: Output parameter, outputs global subscription information.

[0112] Deleting a subscriber is equivalent to deleting all subscriptions of that subscriber using `delSuber(const string&strSuber, vector)`. <string>&vecDeledSubtion), strSuber: input parameter, subscriber name, vecDeledSubtion: output parameter, the deletion of global subscriptions.

[0113] The method to create a new instance of the subscriber management class is newSuber(const string& strSuber), where strSuber is the input parameter, the subscriber name, and the return value is a pointer to the newly created SeSuber class instance.

[0114] The new subscription management class instance is newSubtion(const string& strSubtion), where strSubtion is the input parameter, the subscription information; the return value is a pointer to the newly created SeSubtion class instance.

[0115] To remove a subscriber from a subscription, the method `delSuberFormSubtion(SeSuberSp spSuber, SeSubtionSp spSubtion)` removes the subscription automatically if the subscription has no subscribers after the operation. `spSuber`: Input parameter, the subscriber to be deleted. `spSubtion`: Input parameter, the subscription, the object being operated on. The return value is whether the subscription has no subscribers.

[0116] The subscription processing method provided by this invention is applicable to each network node in a distributed message shuffling system. See [link to relevant documentation]. Figure 2 The flowchart illustrating the implementation of the subscription processing method provided in this embodiment of the invention is described in detail below:

[0117] Step S210: When the first network node comes online, the subscription engine management class in the first network node obtains all the deduplicated subscriptions in the first network node, and the subscription engine management classes in all target network nodes obtain all the deduplicated subscriptions in the target network nodes.

[0118] The target network node is any network node that is functioning normally in the distributed messaging system.

[0119] In some embodiments, the message middleware includes a local process subscription management module and a remote node subscription management module. The local process subscription management module manages subscriptions local to the message middleware, while the local process subscription management module manages subscriptions between the local network node and other network nodes.

[0120] There are two ways to remove duplicates using getSubtionOfAll:

[0121] The subscription management module within the local process of the first network node or the subscription engine management class within the remote node's subscription management module iterates through the subscription information index and adds the subscriptions from the subscription management class within the first network node to the subscription encoding index in the output parameters of the first network node; or

[0122] The subscription management module of the local process within the first network node or the subscription engine management class of the remote node subscription management module traverses the subscription encoding index, skips null pointers, and adds the subscriptions in the subscription management class within the first network node to the subscription encoding index in the output parameters of the first network node.

[0123] Step S220: The first network node sends all subscriptions within the first network node to the target network node, and all target network nodes send all subscriptions within all target network nodes to the first network node.

[0124] Step S230: The subscription engine management class in all target network nodes resets the subscriptions of subscribers in all target network nodes based on all subscriptions in the first network node, so as to overwrite their original subscriptions; The subscription engine management class in the first network node resets the subscriptions of subscribers in the first network node based on all subscriptions in all target network nodes, so as to overwrite their original subscriptions.

[0125] In some embodiments, such as Figure 3 The example shown is setSubtionOfSuber, which sets a subscription for a subscriber, overriding its existing subscription. The specific process is as follows (S31-S37):

[0126] S31. The subscription engine management class in the target network node searches for the target subscriber based on the target subscriber name and the subscriber name index mapSuber.

[0127] S32. The subscription engine management class in the target network node creates a temporary subscription encoding bitmap for the target subscriber, retrieves the target subscription from all subscriptions in the first network node, and searches for all target subscriptions in the subscription information index mapSubtion in the target network node.

[0128] S33. The subscription engine management class in the target network node adds the target subscriber's code to the subscriber code bitmap of all target subscriptions, and adds the codes of all target subscriptions to the temporary subscription code bitmap, resulting in the added temporary subscription code bitmap.

[0129] S34. The subscription engine management class in the target network node determines the code of the subscription to be deleted based on the added temporary subscription code bitmap and the original subscription code bitmap of the target subscriber.

[0130] Specifically, the code for the subscription to be deleted can be obtained by subtracting the added temporary subscription code bitmap from the original subscription code bitmap of the target subscriber.

[0131] S35. The subscription engine management class in the target network node obtains instance pointers of all target subscriptions based on the subscription encoding index, and removes the encoding of the target subscriber from the subscription encoding bitmap of all target subscriptions.

[0132] S36. When the subscription engine management class in the target network node determines that the added temporary subscription encoding bitmap is a non-empty set, the added temporary subscription encoding bitmap is assigned to the original subscription encoding bitmap of the target subscriber.

[0133] S37. When the subscription engine management class in the target network node determines that the added temporary subscription encoding bitmap is an empty set, the target subscriber name is deleted from the subscriber name index.

[0134] In some embodiments, such as Figure 4 The diagram shows the sub-process of adding a new subscriber to `newSuber`. When a new subscriber appears, it needs to be added. The specific process is as follows (S41-S43):

[0135] S41. When the subscription engine management class in the target network node does not find a subscriber name matching the target subscriber name in the subscriber name index, it determines whether there is an unused subscriber code in the queue of unused subscriber codes.

[0136] S42. When the subscription engine management class in the target network node determines that there is an unused subscriber code, a code is selected from the queue of unused subscriber codes as the target subscriber code. The subscriber management class pointer constructed from the target subscriber code is assigned to the element in the subscriber code index with the target subscriber code as the subscript, and the target subscriber name is added to the subscriber name index.

[0137] S43. When the subscription engine management class in the target network node determines that there is no unused subscriber code, the number of elements in the subscriber code index is used as the target subscriber code to construct a subscriber management class, the pointer of the subscriber management class is added to the end of the subscriber code index, and the target subscriber name is added to the subscriber name index.

[0138] In some embodiments, such as Figure 5 The diagram shows the sub-process for adding a new subscription using `newSubtion`. A new subscription needs to be added whenever a new subscription appears. The specific process is as follows (S51-S55):

[0139] S51. When the target subscription is not found in the subscription engine management class of the target network node, determine whether there is an unused subscription code in the queue of unused subscription codes.

[0140] S52. When the subscription engine management class in the target network node determines that there is an unused subscription code, it selects a code from the queue of unused subscription codes as the target subscription code, and assigns the subscription management class pointer constructed from the target subscription code to the element in the subscription code index with the target subscriber's code as the subscript.

[0141] S53. When the subscription engine management class in the target network node determines that there is no unused subscription code, the number of elements in the subscription code index is used as the code of the target subscription, and a subscription management class is constructed with the code of the target subscription. The pointer of the subscription management class is added to the end of the subscription code index.

[0142] S54. The subscription engine management class in the target network node adds the target subscription information to the subscription information index.

[0143] S55. The subscription engine management class in the target network node obtains the bitmap encoding that matches all target segments of the target subscription information in the subscriber bitmap index. If the target subscriber's encoding is not in the bitmap encoding, the target subscriber's encoding is added to the bitmap encoding.

[0144] The target segment is any topic segment of the subscription information of the target subscriber.

[0145] Step S240: When the target network node detects that the first network node is offline, the subscription engine management class in the target network node resets the subscriptions of the subscribers in the target network node and removes the subscribers in the first network node from the subscriptions in the target network node.

[0146] Furthermore, in some embodiments, when it is necessary to add a subscription to a subscriber using `addSubtionOfSuber`, the following steps can be used: Figure 6 As shown:

[0147] S61. When the target network node receives a request from the first subscriber to add the first subscription information, the target network node's subscription engine management class looks up the first subscriber name in the subscriber name index.

[0148] The first subscriber name is the name of the first subscriber.

[0149] S62. If the target network node's subscription engine management class does not find a subscriber name that matches the first subscriber name, then add the first subscriber name to the subscriber name index.

[0150] S63. When the subscription engine management class of the target network node finds or adds the first subscriber name, it searches for the first subscription in the subscription information index; where the first subscription is the subscription that matches the first subscription information.

[0151] S64. When the target network node's subscription engine management class finds the first subscription, the return value is set to the first preset value; when the first subscription is not found, the first subscription is added and the return value is set to the second preset value; when the target network node's subscription engine management class finds or adds the first subscription, the encoding of the first subscription is added to the subscriber encoding bitmap of the first subscription information, the encoding of the first subscriber is added to the subscription encoding bitmap of the first subscription information, and the return value is returned.

[0152] The first preset value is used to indicate that the overall subscription remains unchanged, while the second preset value is used to indicate that the overall subscription has changed.

[0153] Furthermore, in some embodiments, such as Figure 7 The following shows the steps for a subscriber to delete their subscription (delSubtionOfSuber), as detailed in S71-S77:

[0154] S71. When the subscription engine management class of the target network node receives a request from the second subscriber to delete the second subscription information, it searches for the second subscriber name in the subscriber name index.

[0155] The second subscriber's name is the name of the second subscriber.

[0156] S72. When the subscription engine management class of the target network node determines that the second subscriber name does not exist, the return value is set to the first preset value.

[0157] S73. When the subscription engine management class of the target network node determines that the second subscriber name exists, it searches for the second subscription in the subscription information index.

[0158] The second subscription is the subscription corresponding to the second subscription information.

[0159] S74. When the subscription engine management class of the target network node determines that the second subscription does not exist, the return value is set to the first preset value.

[0160] S75. When the subscription engine management class of the target network node finds the second subscription, delete the code of the second subscription from the subscription code bitmap of the second subscriber.

[0161] S76. When the subscription engine management class of the target network node determines that the subscription code bitmap of the second subscriber after deletion is an empty set, it sets the element with the second subscriber's code as the index in the subscriber code index to a null pointer, adds the second subscriber's code to the queue of unused subscriber codes, deletes the second subscriber's name from the subscriber name index, and deletes the second subscriber's code from the second subscriber's subscriber code bitmap.

[0162] S77. When the subscription engine management class of the target network node determines that the subscription encoding bitmap of the second subscriber after deletion is a non-empty set, it deletes the encoding of the second subscriber from the subscriber encoding bitmap of the second subscriber and returns the return value.

[0163] The first preset value is used to indicate that the overall subscription remains unchanged, while the second preset value is used to indicate that the overall subscription has changed.

[0164] In some embodiments, such as Figure 8 The following shows the steps for a subscriber to delete a subscription (delSuberFromSubtion), as detailed in S81-S86:

[0165] S81, The target network node's subscription engine management class removes the subscriber's encoding from the subscriber encoding bitmap.

[0166] S82. Determine whether the subscriber encoding bitmap is empty, i.e. whether there are no subscribers. If there are still subscribers, return false.

[0167] S83. If there are no subscribers, set the element in the subscription encoding index with the encoding of the subscription as the subscript to a null pointer.

[0168] S84. Add the subscription code to the tail of the unused subscription code queue.

[0169] S85. Delete the record from the subscription information index using the subscription information as the key.

[0170] S86. Obtain the bitmap encodings in the subscriber bitmap index that match all target segments of the subscription information, and delete the encoding of the subscription from the bitmap encodings. Here, the target segment is any topic of the subscription information.

[0171] In some embodiments, such as Figure 9 The diagram shows how to delete a subscriber using the delSuber operator, which is equivalent to deleting all subscriptions of that subscriber. The specific steps are as follows: S91-S94:

[0172] S91. When the subscription engine management class of the target network node receives a notification that a third subscriber has unsubscribed, it searches for the third subscriber's name in the subscriber name index.

[0173] The third subscriber's name is the name of the third subscriber.

[0174] S92. When the target network node's subscription engine management class finds the name of the third subscriber, the target network node's subscription engine management class traverses the subscription code bitmap of the third subscriber, accesses the subscription code index with the code of the third subscriber as the subscript, and obtains the subscription represented by the code of the third subscriber.

[0175] S93. The target network node's subscription engine management class removes the third subscriber from the subscription represented by the third subscriber's code, sets the return value to the second preset value, and adds the third subscriber's subscription information to the output parameters.

[0176] S94. The target network node's subscription engine management class sets the element with the third subscriber's code as its index to a null pointer from the subscriber code index, adds the third subscriber's code to the unused subscriber code queue, and removes the third subscriber from the subscriber name index.

[0177] The second preset value is used to represent the overall subscription change.

[0178] This invention reduces "empty numbers" and avoids sparse and scattered encoded ID values ​​by recycling and reusing unused encoded IDs. This avoids wasting space when storing encoded IDs in bitmaps. Furthermore, typical bitmaps have low space utilization, and inner layers can cause waste. This invention uses a Roaring bitmap; when values ​​are sparse, Roaring only allocates memory near the array, saving a significant amount of blank space and avoiding wasted space.

[0179] For distributed information systems, to enable rapid processing of local or remote subscriptions, the `getSubtionOfAll` deduplication method in this invention can retrieve all local subscriptions after deduplication, providing the complete local subscription to the remote node upon initial connection or after reconnection. `addSubtionOfSuber` returns whether adding a subscription by a local (or subordinate) subscriber affects the global subscription, notifying the remote node. `delSubtionOfSuber` returns whether deleting a subscription by a local (or subordinate) subscriber affects the global subscription, also notifying the remote node. `setSubtionOfSuber` efficiently sets all subscriptions sent by a remote node, and can also be used locally. It outputs the addition or deletion of global subscriptions resulting from the operation, notifying the parent node. `delSuber` quickly deletes a subscriber when it exits, outputting the global deletion result to notify the remote node. Utilizing the characteristics of bitmaps for fast intersection, union, and difference calculations, and their inherent deduplication properties, the subscription processing speed can be further improved.

[0180] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0181] Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Figure 10 As shown, the electronic device 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, it implements the steps in the various subscription processing method embodiments described above, for example... Figure 2 Steps 210 to 240 are shown.

[0182] For example, the computer program 102 can be divided into one or more modules, which are stored in the memory 101 and executed by the processor 100 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 102 in the electronic device 10. The electronic device 10 may include, but is not limited to, the processor 100 and the memory 101. Those skilled in the art will understand that... Figure 10 This is merely an example of electronic device 10 and does not constitute a limitation on electronic device 10. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0183] The processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0184] The memory 101 can be an internal storage unit of the electronic device 10, such as a hard disk or memory of the electronic device 10. The memory 101 can also be an external storage device of the electronic device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 10. Furthermore, the memory 101 can include both internal and external storage units of the electronic device 10. The memory 101 is used to store the computer program and other programs and data required by the electronic device. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0185] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various subscription processing method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0186] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.< / string> < / string> < / string> < / string> < / string> < / string> < / string>

Claims

1. A subscription processing method, characterized in that, The method is applied to a distributed messaging system, which includes multiple network nodes, and message middleware within each network node is interconnected. Each message middleware includes a subscription management class, a subscriber management class, and a subscription engine management class. The subscription management class stores the subscriber encoding bitmap for each subscription, and the subscriber management class stores the subscription encoding bitmap for each subscriber. The subscription engine management class is responsible for subscription management and matching within the local network node, as well as subscription management and matching with remote network nodes. Both the subscriber encoding bitmap and the subscription encoding bitmap are highly compressed bitmaps. The method is applicable to each network node in the distributed messaging system, including: When the first network node comes online, the subscription engine management class within the first network node obtains all subscriptions within the first network node after deduplication, and the subscription engine management classes within all target network nodes obtain all subscriptions within the target network nodes after deduplication; wherein, the target network node is any network node that is working normally in the distributed messaging system; The first network node sends all subscriptions within the first network node to the target network node, and all the target network nodes send all subscriptions within the target network nodes to the first network node; The subscription engine management class in all the target network nodes resets the subscriptions of all subscribers in the target network nodes based on all subscriptions within the first network node, thereby overwriting their original subscriptions; the subscription engine management class in the first network node resets the subscriptions of subscribers in the first network node based on all subscriptions within the target network node, thereby overwriting their original subscriptions. When the target network node detects that the first network node is offline, the subscription engine management class in the target network node resets the subscriptions of the subscribers in the target network node and removes the subscribers in the first network node from the subscriptions in the target network node.

2. The subscription processing method as described in claim 1, characterized in that, The message middleware includes a local process subscription management module and a remote node subscription management module; The subscription engine management class within the first network node retrieves all deduplicated subscriptions within the first network node, including: The subscription management module within the local process of the first network node or the subscription engine management class within the remote node's subscription management module iterates through the subscription information index and adds the subscriptions from the subscription management class within the first network node to the subscription encoding index in the output parameters of the first network node; or The subscription engine management class in the local process subscription management module or the remote node subscription management module of the first network node traverses the subscription encoding index, skips null pointers, and adds the subscription in the subscription management class of the first network node to the subscription encoding index in the output parameters of the first network node.

3. The subscription processing method as described in claim 1, characterized in that, The subscription engine management class in all the target network nodes resets the subscriptions of subscribers in all the target network nodes based on all subscriptions within the first network node, overriding their original subscriptions, including: The subscription engine management class in the target network node searches for the target subscriber based on the target subscriber name and the subscriber name index; The subscription engine management class in the target network node creates a temporary subscription encoding bitmap for the target subscriber, retrieves the target subscription from all subscriptions in the first network node, and searches for all the target subscriptions in the subscription information index of the target network node; The subscription engine management class in the target network node adds the code of the target subscriber to the subscriber code bitmap of all the target subscriptions, and adds the code of all the target subscriptions to the temporary subscription code bitmap to obtain the added temporary subscription code bitmap; The subscription engine management class in the target network node determines the codes of the subscriptions to be deleted based on the added temporary subscription code bitmap and the original subscription code bitmap of the target subscriber. The subscription engine management class in the target network node obtains instance pointers of all target subscriptions based on the subscription encoding index, and removes the encoding of the target subscriber from the subscription encoding bitmap of all target subscriptions; When the subscription engine management class in the target network node determines that the added temporary subscription code bitmap is a non-empty set, it assigns the added temporary subscription code bitmap to the original subscription code bitmap of the target subscriber. When the subscription engine management class in the target network node determines that the added temporary subscription encoding bitmap is an empty set, it deletes the target subscriber name from the subscriber name index.

4. The subscription processing method as described in claim 3, characterized in that, The subscription engine management class in the target network node searches for the target subscriber based on the target subscriber name and subscriber name index, including: When the subscription engine management class in the target network node does not find a subscriber name matching the target subscriber name in the subscriber name index, it determines whether there is an unused subscriber code in the queue of unused subscriber codes; When the subscription engine management class in the target network node determines that there is an unused subscriber code, it selects a code from the queue of unused subscriber codes as the target subscriber code, assigns the subscriber management class pointer constructed from the target subscriber code to the element in the subscriber code index with the target subscriber code as the subscript, and adds the target subscriber name to the subscriber name index. When the subscription engine management class in the target network node determines that there are no unused subscriber codes, it constructs a subscriber management class using the number of elements in the subscriber code index as the target subscriber's code, adds a pointer to the subscriber management class to the end of the subscriber code index, and adds the target subscriber name to the subscriber name index.

5. The subscription processing method as described in claim 3, characterized in that, The step of searching for all the target subscriptions from the subscription information index in the target network node includes: If the target subscription is not found in the subscription engine management class of the target network node, it is determined whether there is an unused subscription code in the queue of unused subscription codes; When the subscription engine management class in the target network node determines that there is an unused subscription code, it selects a code from the queue of unused subscription codes as the code for the target subscription, and assigns the subscription management class pointer constructed from the code for the target subscription to the element in the subscription code index with the code of the target subscriber as the subscript. When the subscription engine management class in the target network node determines that there is no unused subscription code, the number of elements in the subscription code index is used as the code of the target subscription, and a subscription management class is constructed with the code of the target subscription. A pointer to the subscription management class is added to the end of the subscription code index. The subscription engine management class in the target network node adds the target subscription information to the subscription information index; The subscription engine management class in the target network node obtains the bitmap encoding that matches all target segments of the target subscription information in the subscriber bitmap index. If the target subscriber's encoding is not found in the bitmap encoding, the target subscriber's encoding is added to the bitmap encoding. The target segment is any topic segment of the target subscriber's subscription information.

6. The subscription processing method as described in claim 1, characterized in that, The method further includes: When the target network node receives a request from the first subscriber to add first subscription information, the target network node's subscription engine management class searches for the first subscriber name in the subscriber name index; wherein, the first subscriber name is the name of the first subscriber. When the subscription engine management class of the target network node does not find a subscriber name that matches the first subscriber name, the first subscriber name is added to the subscriber name index. When the subscription engine management class of the target network node finds or adds the first subscriber name, it searches for the first subscription in the subscription information index; wherein, the first subscription is the subscription that matches the first subscription information; When the subscription engine management class of the target network node finds the first subscription, it sets the return value to a first preset value; when the first subscription is not found, it adds the first subscription and sets the return value to a second preset value; when the subscription engine management class of the target network node finds or adds the first subscription, it adds the code of the first subscription to the subscriber code bitmap of the first subscription information, adds the code of the first subscriber to the subscription code bitmap of the first subscription information, and returns the return value; wherein, the first preset value is used to indicate that the overall subscription has not changed, and the second preset value is used to indicate that the overall subscription has changed.

7. The subscription processing method as described in claim 1, characterized in that, The method further includes: When the subscription engine management class of the target network node receives a request from the second subscriber to delete the second subscription information, it searches for the second subscriber name in the subscriber name index; where the second subscriber name is the name of the second subscriber. When the subscription engine management class of the target network node determines that the second subscriber name does not exist, the return value is set to the first preset value; When the subscription engine management class of the target network node determines that the second subscriber name exists, it searches for the second subscription in the subscription information index; wherein, the second subscription is the subscription corresponding to the second subscription information; When the subscription engine management class of the target network node determines that the second subscription does not exist, the return value is set to the first preset value; When the subscription engine management class of the target network node finds the second subscription, it deletes the code of the second subscription from the subscription code bitmap of the second subscriber; When the subscription engine management class of the target network node determines that the subscription code bitmap of the second subscriber after deletion is an empty set, it sets the element with the second subscriber's code as the index in the subscriber code index to a null pointer, adds the second subscriber's code to the queue of unused subscriber codes, deletes the second subscriber's name from the subscriber name index, and deletes the second subscriber's code from the second subscriber's subscriber code bitmap. When the subscription engine management class of the target network node determines that the subscription encoding bitmap of the second subscriber after deletion is a non-empty set, it deletes the encoding of the second subscriber from the subscriber encoding bitmap of the second subscription and returns the return value; wherein, the first preset value is used to indicate that the overall subscription has not changed.

8. The subscription processing method as described in claim 1, characterized in that, The method further includes: When the subscription engine management class of the target network node receives a notification that a third subscriber has unsubscribed, it searches for the name of the third subscriber in the subscriber name index; wherein, the name of the third subscriber is the name of the third subscriber. When the subscription engine management class of the target network node finds the name of the third subscriber, the subscription engine management class of the target network node traverses the subscription code bitmap of the third subscriber, accesses the subscription code index with the code of the third subscriber as the subscript, and obtains the subscription represented by the code of the third subscriber. The target network node's subscription engine management class removes the third subscriber from the subscription represented by the third subscriber's code, sets the return value to a second preset value, and adds the third subscriber's subscription information to the output parameters; The target network node's subscription engine management class sets the element with the third subscriber's code as its index to a null pointer from the subscriber code index, adds the third subscriber's code to the unused subscriber code queue, and deletes the third subscriber from the subscriber name index. The second preset value is used to represent the overall subscription change.

9. An electronic device, characterized in that, The method includes a memory and a processor, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images