Parallel integrated matching method and system for publish-subscribe systems based on subscription partitioning

By dividing the subscription into multiple sub-subscriptions for parallel processing and combining the TAMA and REIN algorithms, the problem of matching speed fluctuation in high-dimensional space is solved, and efficient data distribution services are achieved.

CN115828010BActive Publication Date: 2025-10-10SHANGHAI JIAOTONG UNIV +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202211213607.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-30
Publication Date
2025-10-10
Estimated Expiration
2042-09-30

AI Technical Summary

Technical Problem

In high-dimensional space, the event matching performance of content-based publish/subscribe systems is affected by the number of subscriptions and predicates, resulting in fluctuations in matching speed, which becomes a performance bottleneck and makes it difficult to provide stable data distribution services.

Method used

A single subscription containing multiple interval predicates is divided into multiple sub-subscriptions, and the sub-subscriptions in the TAMA and REIN algorithm libraries are processed in parallel. The dividing line SP is adjusted through an adaptive feedback mechanism to ensure the balance of the algorithm library and achieve parallel matching.

Benefits of technology

It significantly improves the matching speed of large-scale high-dimensional data, ensures the time balance between the model's multi-threaded algorithms, and provides fast and stable data distribution services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115828010B_ABST
    Figure CN115828010B_ABST
Patent Text Reader

Abstract

The application provides a kind of parallel integrated matching method and system of publish-subscribe system based on subscription division, comprising: step 1: based on the matching probability of interval predicate, a single subscription containing multiple interval predicates is divided into multiple sub-subscriptions;Step 2: insert each sub-subscription divided into the corresponding algorithm of algorithm library;Step 3: after triggering the event, all algorithms in the algorithm library are processed in parallel, the outputs of all algorithms are aggregated to obtain the final matching result.The application greatly improves the matching speed under large-scale data through integrated matching method, and can efficiently distribute data sets under high-dimensional space;At the same time, in order to ensure the relative balance of time between multi-threaded algorithms of the model, an adaptive feedback mechanism is proposed to balance the dynamic matching process of the model and obtain the best division scheme of the subscription.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data distribution, and in particular, to a publish-subscribe system parallel integrated matching method and system based on subscription division. BACKGROUND

[0002] In the face of data explosion, many fields need to provide data distribution services on demand. For example, the stock market produces a large amount of data every day, and investors need to subscribe and receive information about events of interest. Another example is the intelligent transportation scenario where a large number of data collection devices are deployed, and drivers need a mechanism to timely obtain congestion and accident information of the road ahead specific to their driving route. These applications stimulate the demand for an effective way for data to propagate from publishers (sources) to subscribers (destinations).

[0003] Content-based publish-subscribe systems are an effective paradigm to realize on-demand event distribution. To express their interest in data, subscribers first define subscriptions, which usually contain multiple predicates, and then publish them to a broker (server). Publishers generate events composed of multiple attribute-value pairs and send them to the broker. For each event, the broker needs to match it with the subscriptions to identify the target subscribers to which the event information should be forwarded. In this way, publishers and subscribers are loosely coupled, which is the most attractive feature of content-based publish / subscribe systems.

[0004] Obviously, event matching is a key component in content-based publish / subscribe systems. Given a high-dimensional space, an event represents a point and a subscription represents a rectangle. Event matching is essentially a point-in-rectangle search problem, and its time cost is expensive in high-dimensional space. Worse still, when the number of subscriptions is large, the matching performance decreases, becoming a potential performance bottleneck of content-based publish / subscribe systems.

[0005] Patent document CN109547529A (application number: CN201811203990.3) discloses a distributed industrial data distribution method based on DDS, including the following steps: S1: creating a device interface file for each abstract device; S2: creating a subscription publication relationship mapping table to establish the mapping relationship between data subscribers and data publishers, and between data subscribers and DDS topics; S3: creating a real-time database; S4: collecting interface data of the data publisher and publishing it in the form of a DDS topic to the network; S5: decoding the DDS topic data and writing the decoded data into the real-time database; S6: reverse lookup of the data subscriber corresponding to the DDS topic according to the subscription publication relationship mapping table, and calling the device interface of the data subscriber to notify the data subscriber to read the DDS topic data.

[0006] To improve matching performance, researchers have proposed numerous data structures for storing subscriptions, such as trees, tables, and Bloom filters. These novel data structures support efficient event matching. However, most existing data structures index predicates in the same manner, regardless of their matching probability. Researchers have concluded that predicate matching probability affects the performance of most matching algorithms. Increasing or decreasing the matching probability leads to performance degradation. One problem with matching algorithms with fluctuating performance is that they cannot guarantee fast and stable data distribution services. Furthermore, matching time is closely related to the number of predicates in a subscription. Each subscription contains multiple interval predicates. When the number of interval predicates is large, this is called a high-dimensional space. The speed of subscription-event matching in a high-dimensional space is significantly affected, and matching performance fluctuates as the number of predicates increases. Summary of the Invention

[0007] In view of the defects in the prior art, the present invention aims to provide a publish-subscribe system parallel integration matching method and system based on subscription partitioning.

[0008] The publish-subscribe system parallel integration matching method based on subscription partitioning provided by the present invention includes:

[0009] Step 1: Based on the matching probability of interval predicates, a single subscription containing multiple interval predicates is divided into multiple sub-subscriptions;

[0010] Step 2: Insert the divided sub-subscriptions into the corresponding algorithms in the algorithm library;

[0011] Step 3: After the event is triggered, all algorithms in the algorithm library are processed in parallel, and the outputs of all algorithms are aggregated to obtain the final matching result.

[0012] Preferably, an integrated matching framework is implemented based on the TAMA algorithm and the REIN algorithm. The TAMA algorithm uses a counter to record the number of times the predicate of each subscription is met during matching, and the REIN algorithm uses a bit set to mark all unmatched subscriptions. If the event value meets a predicate in a subscription, the counter corresponding to the subscription in the TAMA algorithm will be increased by one; on the contrary, if the event value does not meet a predicate in a subscription, the bit corresponding to the subscription in the REIN algorithm bit set will be marked as unmatched.

[0013] Preferably, the matching probability of an interval predicate is estimated by the width and cardinality of the predicate, as expressed in:

[0014]

[0015] Among them, events are uniformly distributed, and the cardinality of each predicate is C, w i is the width of the i-th interval predicate, pi is the matching probability.

[0016] Preferably, a split point SP is defined as the dividing line of the matching probability of the TAMA algorithm and the REIN algorithm. After normalization, the matching probability of the predicate is a value between 0 and 1. The initial value of SP is set to 0.5. When a new subscription arrives, it is necessary to process each interval predicate in the subscription separately, calculate its matching probability, and reconstruct an empty sub-subscription inserted into the REIN algorithm data structure and an empty sub-subscription of the TAMA algorithm. When the matching probability of the interval predicate is greater than SP, it is inserted into the sub-subscription of the REIN algorithm; when the matching probability of the interval predicate is less than SP, it is inserted into the sub-subscription of the TAMA algorithm. After processing all interval predicates in this subscription, two new sub-subscriptions are obtained, which are located in the data structures of the REIN algorithm and the TAMA algorithm respectively. After the event arrives, they are matched in parallel with the subscription sets in the REIN algorithm and the TAMA algorithm, and partial matching results are obtained respectively.

[0017] Preferably, a subscription is split into two sub-subscriptions, and partial matching results with the event are obtained for each of them. The results are represented by bits of 0 or 1. If the matching results of the TAMA algorithm and the REIN algorithm are both 1, the subscription and the event match, otherwise they do not match.

[0018] Set a subscription window of a preset size, record the average matching time of the TAMA algorithm and the REIN algorithm within this window, and update the value of the dividing line SP after each window ends:

[0019]

[0020] Among them, R max and R min are the upper and lower limits of the SP value fluctuation, t TAMA and T are the running time and total time of the TAMA algorithm in the window, respectively. If the running time of the TAMA algorithm is too long, the size of the dividing line SP is reduced; if the running time of the REIN algorithm is too long, the size of the dividing line SP is increased.

[0021] The publish-subscribe system parallel integration matching system based on subscription partitioning provided by the present invention includes:

[0022] Module M1: divides a single subscription containing multiple interval predicates into multiple sub-subscriptions based on the matching probability of the interval predicates;

[0023] Module M2: inserts the divided sub-subscriptions into the corresponding algorithms in the algorithm library;

[0024] Module M3: After the event is triggered, all algorithms in the algorithm library are processed in parallel, and the outputs of all algorithms are aggregated to obtain the final matching result.

[0025] Preferably, an integrated matching framework is implemented based on the TAMA algorithm and the REIN algorithm. The TAMA algorithm uses a counter to record the number of times the predicate of each subscription is met during matching, and the REIN algorithm uses a bit set to mark all unmatched subscriptions. If the event value meets a predicate in a subscription, the counter corresponding to the subscription in the TAMA algorithm will be increased by one; on the contrary, if the event value does not meet a predicate in a subscription, the bit corresponding to the subscription in the REIN algorithm bit set will be marked as unmatched.

[0026] Preferably, the matching probability of an interval predicate is estimated by the width and cardinality of the predicate, as expressed in:

[0027]

[0028] Among them, events are uniformly distributed, and the cardinality of each predicate is C, w i is the width of the i-th interval predicate, p i is the matching probability.

[0029] Preferably, a split point SP is defined as the dividing line of the matching probability of the TAMA algorithm and the REIN algorithm. After normalization, the matching probability of the predicate is a value between 0 and 1. The initial value of SP is set to 0.5. When a new subscription arrives, it is necessary to process each interval predicate in the subscription separately, calculate its matching probability, and reconstruct an empty sub-subscription inserted into the REIN algorithm data structure and an empty sub-subscription of the TAMA algorithm. When the matching probability of the interval predicate is greater than SP, it is inserted into the sub-subscription of the REIN algorithm; when the matching probability of the interval predicate is less than SP, it is inserted into the sub-subscription of the TAMA algorithm. After processing all interval predicates in this subscription, two new sub-subscriptions are obtained, which are located in the data structures of the REIN algorithm and the TAMA algorithm respectively. After the event arrives, they are matched in parallel with the subscription sets in the REIN algorithm and the TAMA algorithm, and partial matching results are obtained respectively.

[0030] Preferably, a subscription is split into two sub-subscriptions, and partial matching results with the event are obtained for each of them. The results are represented by bits of 0 or 1. If the matching results of the TAMA algorithm and the REIN algorithm are both 1, the subscription and the event match, otherwise they do not match.

[0031] Set a subscription window of a preset size, record the average matching time of the TAMA algorithm and the REIN algorithm within this window, and update the value of the dividing line SP after each window ends:

[0032]

[0033] Among them, R max and R min are the upper and lower limits of the SP value fluctuation, t TAMA and T are the running time and total time of the TAMA algorithm in the window, respectively. If the running time of the TAMA algorithm is too long, the size of the dividing line SP is reduced; if the running time of the REIN algorithm is too long, the size of the dividing line SP is increased.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) This invention greatly improves the matching speed of large-scale data through an integrated matching method, and can efficiently distribute data sets in high-dimensional space. At the same time, to ensure the relative time balance between the multi-threaded algorithms of the model, an adaptive feedback mechanism is proposed to balance the dynamic matching process of the model and obtain the optimal partitioning scheme for subscriptions.

[0036] (2) The present invention aims to solve the problem of fast matching in a publish / subscribe system based on data content in large-scale and high-dimensional space, and invents an integrated matching framework. By running multiple matching algorithms with complementary behaviors in parallel, the characteristics of the matching algorithms are fully utilized to significantly improve the matching speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0038] Figure 1 It is a logical framework diagram of the present invention;

[0039] Figure 2 This is a framework diagram for implementing the present invention. DETAILED DESCRIPTION

[0040] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0041] Example:

[0042] The present invention is a publish / subscribe parallel matching algorithm based on subscription partitioning, applicable to large-scale, high-dimensional data processing. For example, in the stock market, people often subscribe to information about stocks of interest, intending to buy immediately when the stock price drops. This requires an efficient and fast matching algorithm to deliver this information to subscribers immediately after the price fluctuation. The stock market typically contains a large amount of data, and the present invention can quickly process millions of subscriptions. Furthermore, the stock market contains a large amount of information, such as opening prices, K-line charts, and moving averages, which contains high-dimensional information. The present invention can efficiently process high-dimensional event information.

[0043] This paper proposes a fine-grained parallel ensemble matching algorithm, called fgPEM, which uses a multithreaded strategy to improve and stabilize matching performance and rapidly distribute large amounts of data in high-dimensional spaces. Similar to algorithms that build libraries of matching algorithms with complementary behaviors, the basic idea of ​​fgPEM is to partition a subscription into multiple sub-subscriptions with varying matching probabilities. Each algorithm is efficiently utilized by indexing each sub-subscription with an appropriate algorithm. When matching events, all algorithms run in parallel, each in a separate thread.

[0044] Two key points require attention when designing fgPEM. First, subscriptions must be partitioned based on each algorithm's performance under different subscription matching probabilities. Second, to prevent overloading of specific algorithms, a dynamic feedback mechanism must be established to ensure performance balance among multiple algorithms. Taking into account the characteristics of different algorithms and the balance of multithreading, this paper designs a flexible subscription partitioning method in fgPEM.

[0045] This paper implements a fgPEM prototype system based on two existing matching algorithms, REIN and TAMA. First, REIN and TAMA are selected to form an algorithm library. Second, incoming single subscriptions are divided into two sub-subscriptions based on their matching probabilities. Finally, this paper uses a parallel matching framework and an adaptive dynamic feedback mechanism to reduce the model's matching overhead.

[0046] frame structure

[0047] like Figure 1 As shown in Figure 1, the integrated matching framework fgPEM proposed in this paper consists of two main parts: a partitioner and an algorithm library. The partitioner partitions a single subscription containing multiple interval predicates into multiple sub-subscriptions based on the matching probability of interval predicates, and inserts each of these partitioned sub-subscriptions into the corresponding algorithms in the algorithm library. The algorithm library consists of multiple matching algorithms with complementary behaviors.

[0048] The matching process is as follows: Step 1: Incoming subscriptions are divided into multiple sub-subscription sets based on the matching probability through a divider; Step 2: Sub-subscriptions that match their matching probabilities are inserted into the corresponding data structures maintained by all algorithms in the algorithm library; Step 3: When an event arrives, all algorithms in the algorithm library are processed in parallel; Step 4: The outputs of all algorithms are aggregated to obtain the final matching result.

[0049] Module Process

[0050] (1) Algorithm library

[0051] When constructing the algorithm library, the present invention selects algorithms based on three criteria. First, the algorithms in the library should complement each other. In other words, the algorithms should have different performance in terms of predicate matching probability. Second, the selected algorithms should have similar overall matching performance to achieve good integrated matching results. Finally, for greater versatility, the algorithms in the library should support different subscription data models.

[0052] (2) Divider

[0053] The partitioner is responsible for partitioning each incoming subscription into multiple sub-subscriptions. Its design requirements are mainly reflected in three aspects. First, the partitioner needs to accurately partition a single subscription into multiple sub-subscriptions according to the matching probability. Second, since subscriptions and events are constantly arriving, the model needs to be designed as an online service (such as Figure 2 ), with the ability to continuously accept subscriptions and events. Finally, the multi-threading selected by the model should have a performance balance, so a feedback mechanism is needed to ensure the stability of the model's performance.

[0054] System Implementation

[0055] The present invention implements an integrated matching framework fgPEM based on two existing algorithms REIN and TAMA.

[0056] (1) Algorithm library

[0057] According to the established algorithm library selection principle, the application selects two algorithms: REIN and TAMA to realize the integrated matching framework. The two algorithms have similar overall performance and different performance according to different subscription matching probabilities. TAMA uses a counter to record the number of times that each predicate of a subscription is satisfied when matching, while REIN uses a bit set to mark all non-matching subscriptions. If the event value satisfies a certain predicate in a subscription, the counter corresponding to the subscription in TAMA will be incremented by one. Conversely, if the event value does not satisfy a certain predicate in a subscription, the bit corresponding to the subscription in the REIN bit set will be marked as non-matching. In summary, given an event, if more predicates are satisfied, the performance of REIN will improve and the performance of TAMA will decrease, and vice versa. The pseudo codes of REIN and TAMA are as follows:

[0058]

[0059]

[0060] A subscription is composed of multiple interval predicates, and an interval predicate is composed of a low value and a high value. The matching probability of a subscription can be estimated by the width of all interval predicates and the number of interval predicates. Similarly, the matching probability of an interval predicate can be estimated by the width of the predicate and the cardinality. Assuming that the events are uniformly distributed and the cardinality of each predicate is C, the matching probability can be estimated as:

[0061]

[0062] where w i is the width of the i-th interval predicate, and p i is its matching probability.

[0063] The application defines a split point SP as the dividing line of the matching probabilities of REIN and TAMA. After normalization, the matching probability of a predicate is a value between 0 and 1, and the initial value of SP is set to 0.5. When a new subscription arrives, each interval predicate in the subscription needs to be processed separately to calculate its matching probability. A new empty sub-subscription is inserted into the REIN data structure and a new empty sub-subscription is constructed for TAMA. When the matching probability of an interval predicate is greater than SP, it is inserted into the sub-subscription of REIN; when the matching probability of an interval predicate is less than SP, it is inserted into the sub-subscription of TAMA. After processing all interval predicates in this subscription, two new sub-subscriptions are obtained, which are located in the data structures of REIN and TAMA respectively. When an event arrives, parallel matching is performed with the subscription set in REIN and TAMA, and partial matching results are obtained respectively.

[0064] (3) Aggregation and feedback mechanism

[0065] In the splitter, a subscription is split into two sub-subscriptions, each of which obtains partial matches with the event. The results are represented by bits representing either 0 or 1. If both REIN and TAMA match results are 1, the subscription matches the event; otherwise, it does not. Therefore, when the number of subscriptions reaches millions, only two bitsets of one million in length are needed to obtain aggregated results. Bitset aggregation is also very fast. In our implementation, aggregation time accounts for a negligible fraction of the overall matching time.

[0066] Since matching is done using multiple threads, the performance of each thread affects the performance of the overall algorithm. A feedback mechanism is needed to ensure a balanced matching time across multiple threads. Therefore, a window of a certain size (usually 100 subscriptions) is set, and the average matching time of the REIN and TAMA algorithms within this window is recorded. After each window, the value of the dividing line SP is updated:

[0067]

[0068] Among them, R max and R min These are the upper and lower limits of the SP value fluctuation, respectively. This ensures that even in extreme data situations, subscriptions can be evenly divided into the two algorithms. TAMA and T are the TAMA execution time and total execution time within the window, respectively. If the TAMA execution time is too long, reduce the size of the SP cutoff; if the REIN execution time is too long, increase the size of the SP cutoff.

[0069] The present invention performs performance evaluation on the proposed algorithm based on a 100-dimensional data space. Experimental results show that the matching performance is improved by at least 31% compared with the baseline algorithm fgPEM.

[0070] Because the performance of most matching algorithms is affected by subscription matching probabilities, this invention leverages multiple existing algorithms with complementary performance to achieve integrated matching and improve matching efficiency. A single subscription containing multiple interval predicates is split into multiple sub-subscriptions based on matching probabilities, each with its own matching algorithm. When an event arrives, multiple matching algorithms process their respective sub-subscriptions in parallel, and the outputs of these algorithms are aggregated to produce a matching result, significantly reducing average matching time.

[0071] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0072] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A parallel integration matching method for publish-subscribe systems based on subscription partitioning, characterized in that: include: Step 1: Based on the matching probability of interval predicates, a single subscription containing multiple interval predicates is divided into multiple sub-subscriptions; Step 2: Insert the divided sub-subscriptions into the corresponding algorithms in the algorithm library; Step 3: After the event is triggered, all algorithms in the algorithm library are processed in parallel, and the outputs of all algorithms are aggregated to obtain the final matching result; An integrated matching framework is implemented based on the TAMA algorithm and the REIN algorithm. The TAMA algorithm uses a counter to record the number of times the predicate of each subscription is satisfied during matching, while the REIN algorithm uses a bit set to mark all unmatched subscriptions. If the event value satisfies a predicate in a subscription, the counter corresponding to the subscription in the TAMA algorithm will be incremented by one; conversely, if the event value does not satisfy a predicate in a subscription, the bit corresponding to the subscription in the REIN algorithm bit set will be marked as unmatched. The matching probability of an interval predicate is estimated by its width and cardinality, as follows: Among them, events are uniformly distributed, and the cardinality of each predicate is C, w i is the width of the i-th interval predicate, p i is the matching probability.

2. The publish-subscribe system parallel integration matching method based on subscription partitioning according to claim 1 is characterized in that: Define a split point SP as the dividing line between the matching probabilities of the TAMA algorithm and the REIN algorithm. After normalization, the matching probability of the predicate is a value between 0 and 1. The initial value of SP is set to 0.

5. When a new subscription arrives, it is necessary to process each interval predicate in the subscription separately, calculate its matching probability, and reconstruct an empty sub-subscription inserted into the REIN algorithm data structure and an empty sub-subscription of the TAMA algorithm. When the matching probability of the interval predicate is greater than SP, it is inserted into the sub-subscription of the REIN algorithm; when the matching probability of the interval predicate is less than SP, it is inserted into the sub-subscription of the TAMA algorithm. After processing all interval predicates in this subscription, two new sub-subscriptions are obtained, which are located in the data structures of the REIN algorithm and the TAMA algorithm respectively. After the event arrives, they are matched in parallel with the subscription sets in the REIN algorithm and the TAMA algorithm, and partial matching results are obtained respectively.

3. The publish-subscribe system parallel integration matching method based on subscription partitioning according to claim 2 is characterized in that: Split a subscription into two sub-subscriptions and obtain partial matching results with the event for each. The results are represented by bits of 0 or 1. If the matching results of the TAMA algorithm and the REIN algorithm are both 1, the subscription and the event match; otherwise, they do not match. Set a subscription window of a preset size, record the average matching time of the TAMA algorithm and the REIN algorithm within this window, and update the value of the dividing line SP after each window ends: Among them, R max and R min are the upper and lower limits of the SP value fluctuation, t TAMA and T are the running time and total time of the TAMA algorithm in the window, respectively. If the running time of the TAMA algorithm is too long, the size of the dividing line SP is reduced; if the running time of the REIN algorithm is too long, the size of the dividing line SP is increased.

4. A parallel integration matching system for publish-subscribe systems based on subscription partitioning, characterized in that: include: Module M1: divides a single subscription containing multiple interval predicates into multiple sub-subscriptions based on the matching probability of the interval predicates; Module M2: inserts the divided sub-subscriptions into the corresponding algorithms in the algorithm library; Module M3: After the event is triggered, all algorithms in the algorithm library are processed in parallel, and the outputs of all algorithms are aggregated to obtain the final matching result; An integrated matching framework is implemented based on the TAMA algorithm and the REIN algorithm. The TAMA algorithm uses a counter to record the number of times the predicate of each subscription is satisfied during matching, while the REIN algorithm uses a bit set to mark all unmatched subscriptions. If the event value satisfies a predicate in a subscription, the counter corresponding to the subscription in the TAMA algorithm will be incremented by one; conversely, if the event value does not satisfy a predicate in a subscription, the bit corresponding to the subscription in the REIN algorithm bit set will be marked as unmatched. The matching probability of an interval predicate is estimated by its width and cardinality, as follows: Among them, events are uniformly distributed, and the cardinality of each predicate is C, w i is the width of the i-th interval predicate, p i is the matching probability.

5. The publish-subscribe system parallel integration matching system based on subscription partitioning according to claim 4 is characterized in that: Define a split point SP as the dividing line between the matching probabilities of the TAMA algorithm and the REIN algorithm. After normalization, the matching probability of the predicate is a value between 0 and 1. The initial value of SP is set to 0.

5. When a new subscription arrives, it is necessary to process each interval predicate in the subscription separately, calculate its matching probability, and reconstruct an empty sub-subscription inserted into the REIN algorithm data structure and an empty sub-subscription of the TAMA algorithm. When the matching probability of the interval predicate is greater than SP, it is inserted into the sub-subscription of the REIN algorithm; when the matching probability of the interval predicate is less than SP, it is inserted into the sub-subscription of the TAMA algorithm. After processing all interval predicates in this subscription, two new sub-subscriptions are obtained, which are located in the data structures of the REIN algorithm and the TAMA algorithm respectively. After the event arrives, they are matched in parallel with the subscription sets in the REIN algorithm and the TAMA algorithm, and partial matching results are obtained respectively.

6. The publish-subscribe system parallel integration matching system based on subscription partitioning according to claim 5 is characterized in that: Split a subscription into two sub-subscriptions and obtain partial matching results with the event for each. The results are represented by bits of 0 or 1. If the matching results of the TAMA algorithm and the REIN algorithm are both 1, the subscription and the event match; otherwise, they do not match. Set a subscription window of a preset size, record the average matching time of the TAMA algorithm and the REIN algorithm within this window, and update the value of the dividing line SP after each window ends: Among them, R max and R min are the upper and lower limits of the SP value fluctuation, t TAMA and T are the running time and total time of the TAMA algorithm in the window, respectively. If the running time of the TAMA algorithm is too long, the size of the dividing line SP is reduced; if the running time of the REIN algorithm is too long, the size of the dividing line SP is increased.

Citation Information

Patent Citations

  • Distributed industrial data distribution method based on DDS

    CN109547529A

  • A Distributed Industrial Data Distribution Method Based on DDS

    CN109547529B

  • Optimization method and system based on matching real-time performance in publish-subscribe system

    CN110413927A

  • Combined matching system and method based on content publishing and subscribing

    CN112100557A