Event matching method and system based on hardware computing perception in content distribution system
Through the hardware-aware event matching algorithm HEM, the data structure of the multi-level index layer and predetermined group bit sets are used to perform bits or operations, combined with the subscription pre-market cache optimization method, the problem of repeated execution efficiency in the event matching process in the prior art is solved, and significant efficiency improvement and matching time reduction is achieved.
Patent Information
- Application Number
- CN202111678440.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2041-12-31
AI Technical Summary
The existing event matching algorithm repeatedly performs inefficient operations during the matching process, resulting in a longer matching time.
Using hardware-aware event matching algorithm HEM, bits or operations are performed through a multi-level index layer and a data structure of a predetermined group of bit sets to reduce duplicate operations, combined with a subscription pre-market cache optimization method.
It significantly improves the efficiency of event matching, reduces matching time, and improves the efficiency of bit computing in the computer execution cache by making full use of hardware features.
Smart Images

Figure CN114491239B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of matching in event distribution networks, and in particular, to an event matching method and system based on hardware computing perception in a content distribution system. In particular, it preferably relates to an event matching algorithm based on hardware computing perception in a content publishing and subscription system. Background Art
[0002] As a flexible communication paradigm, content-based publish / subscribe systems are widely used to build distributed applications such as stock trading systems, recommendation systems, social messaging systems, and monitoring systems. There are three basic roles in content-based publish / subscribe systems to achieve fine-grained data distribution through event matching: subscribers, publishers, and proxy servers. Subscribers submit subscriptions to the proxy server based on their interests and concerns. Publishers generate events and send them to the proxy server. For each arriving event, the proxy server executes an event matching algorithm to search for matching subscriptions and forwards the events to the subscribers to whom those matching subscriptions belong.
[0003] Obviously, matching algorithms play an irreplaceable role in content-based publish / subscribe systems. With the growth of subscription numbers, event arrival rates, and data dimensions, event matching algorithms inevitably become the performance bottleneck of the entire content-based publish / subscribe system. In the past few decades, the need to improve the end-to-end data distribution latency in content-based publish / subscribe systems has driven the research and development of event matching algorithms.
[0004] Researchers have proposed a variety of algorithms to improve the efficiency of event matching. Depending on the primary search goal (matching subscriptions or not matching subscriptions), there are forward algorithms (such as TAMA [Table MAtchin] and OpIndex [OperatorIndex]) and reverse algorithms (such as REIN [REctangle INtersectio] and GEM [Geometrical EventMatching]). Each matching algorithm has its own design ideas. From a hardware perspective, TAMA and OpIndex perform a large number of addition and comparison operations during the matching process, while REIN and GEM perform a large number of traversal and marking operations. In fact, because the data movement overhead between cache and memory is optimized, computers perform bit operations in cache more efficiently. In addition, as memory prices fall and large-capacity memory becomes more common, cache mechanisms become a potential way to trade space for time.
[0005] The Chinese invention patent document with publication number CN105740337A discloses a method for fast event matching in a content-based publish-subscribe system, comprising the following steps: 1. Establishing an event multi-level index tree and a subscription multi-level index tree; 2. The user sends a subscription request or a published event to an event matching node; 3. The event matching node inserts the received published event into the event multi-level index tree; 4. Simplifying the subscription expression according to the event multi-level index tree, and inserting the simplified predicate into the subscription multi-level index tree; 5. For the received subscription request or published event, finding matching events or subscription candidate sets, and then matching them one by one; 6. After completing the matching, the event matching node sends the event to the corresponding subscriber according to a certain routing protocol.
[0006] With respect to the above-mentioned related technologies, the inventors believe that the existing algorithms repeatedly perform a large number of inefficient operations (such as assignment operations in REIN and addition operations in TAMA) during the matching process, resulting in a long matching time. Summary of the invention
[0007] In view of the defects in the prior art, the object of the present invention is to provide an event matching method and system based on hardware computing perception in a content distribution system.
[0008] According to the present invention, an event matching method based on hardware computing perception in a content distribution system includes the following steps:
[0009] Subscription steps: Submit subscription;
[0010] Release step: Generate release events;
[0011] Matching step: Perform event matching to search for matching subscriptions, and send the events of matching subscriptions to the subscribers to which the matching subscriptions belong.
[0012] Preferably, in the matching step, the time for performing repeated operations in the event matching process is reduced by using a hardware-aware event matching algorithm HEM.
[0013] Preferably, in the matching step, the HEM is optimized by subscribing to a pre-tagged cache swap space.
[0014] Preferably, in the matching step, a performance analysis model is constructed for the HEM to quantify the relationship between memory consumption and performance improvement.
[0015] Preferably, in the matching step, a data structure of the HEM is constructed, and subscriptions are indexed through a hardware-aware data structure; the data structure of the HEM includes a multi-level index layer and a predetermined group of digital sets, and each attribute corresponds to a predetermined digital set; the first-level index layer is constructed based on the attribute, the second-level index layer indexes the low-value end and the high-value end of the predicate, and the third-level index layer is constructed by dividing the value range of the attribute into multiple cells; each cell corresponds to a storage bucket, which is used to store the predicate value of the predicate and the corresponding subscription number.
[0016] Preferably, in the matching step, each low value end or high value end is associated with a predetermined set of digits, and the predetermined set of digits is used to pre-mark the subscription as unmatched.
[0017] Preferably, in the matching step, when the subscription S is inserted into the data structure of the HEM, the predicate value of each predicate in the subscription S will be mapped to the cell containing the predicate value at the low value end or the high value end of the corresponding attribute respectively;
[0018] The predicate value and subscription number are stored in pairs in the bucket corresponding to the cell.
[0019] Preferably, in the matching step, the HEM uses a cache mechanism to distribute cells into different groups; each group is associated with a preset number of bit sets; and the HEM pre-marks subscriptions in each group as unmatched in the corresponding bit set.
[0020] According to the present invention, an event matching system based on hardware computing perception in a content distribution system includes the following modules:
[0021] Subscription module: submit subscription;
[0022] Release module: generates release events;
[0023] Matching module: performs event matching to search for matching subscriptions and sends events of matching subscriptions to subscribers to which the matching subscriptions belong.
[0024] Preferably, in the matching module, the time for performing repeated operations in the event matching process is reduced by a hardware-aware event matching algorithm HEM.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] 1. The present invention is different from the existing work. HEM stores the subscription status in the cache and mainly performs some efficient bit OR operations during the matching process. By making full use of the characteristics of the hardware, the HEM algorithm can significantly improve the efficiency of event matching;
[0027] 2. This paper builds a performance analysis model to quantify the trade-off between memory consumption and performance improvement;
[0028] 3. The present invention performs as many bit OR operations as possible during event matching, which is the most efficient for computer hardware;
[0029] 4. Compared with similar matching algorithms, the HEM of the present invention can reduce a lot of matching time. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0031] Figure 1 It is a data structure diagram of the present invention;
[0032] Figure 2 A flow chart of the subscription insertion algorithm of the present invention;
[0033] Figure 3 This is a subscription example diagram of the present invention;
[0034] Figure 4 This is a flow chart of the event matching algorithm of the present invention. DETAILED DESCRIPTION
[0035] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0036] The embodiment of the present invention discloses an event matching algorithm based on hardware computing perception of a content publishing / subscribing system, such as Figure 1 As shown, the following steps are included: Subscription step: Submit subscription. Publishing step: Generate publishing event.
[0037] Matching step: Perform event matching to search for matching subscriptions, and send the events of matching subscriptions to the subscribers to which the matching subscriptions belong. Reduce the time of performing repeated operations during event matching through the hardware-aware event matching algorithm HEM [Hardware-aware Event Matching]. Optimize HEM by pre-marking the cache swap space through subscriptions. Build a performance analysis model for HEM to quantify the relationship between memory consumption and performance improvement.
[0038] In order to achieve efficient event matching, the present invention firstly proposes a hardware-aware event matching algorithm HEM, which aims to reduce the time of performing repeated operations in the event matching process. Secondly, the present invention uses a subscription pre-marking cache method to optimize the matching efficiency of HEM by swapping space. Finally, the present invention constructs a theoretical performance analysis model to quantify the relationship between memory consumption and performance improvement.
[0039] Data structure design: It is quite challenging to ensure that the matching algorithm runs at high speed and stably in different application scenarios. First, the number and size of subscriptions may vary greatly, and the impact of such differences on the performance of the matching algorithm should be as small as possible. Second, the dimensions of the content space may range from tens to tens of thousands, and the matching algorithm should be able to adapt to low-dimensional and high-dimensional situations. Third, the matching algorithm should support multiple event types, and the size of the event should not seriously affect the matching performance. Fourth, the insertion and deletion overhead of subscriptions should be as small as possible to adapt to dynamic environments. Fifth, the skewed distribution of attributes, event values, and predicate values should not cause large performance fluctuations in the matching algorithm. Sixth, the width of the predicate in the subscription should not have much impact on the matching time. Among them, the number and size of subscriptions, the spatial dimension, and the event size are hard parameters of the matching algorithm performance, and the value distribution and predicate width are soft parameters.
[0040] When the number of subscriptions is large, for most algorithms, matching an event requires a large number of repeated operations, such as comparison, addition, or bit marking. From a hardware perspective, the bitwise OR operation is more efficient than the operations repeatedly performed by existing multi-part matching algorithms. Therefore, replacing other types of operations with bitwise OR operations as much as possible is a feasible solution to improve matching performance. With this idea in mind, the present invention designs a hardware-aware data structure to index subscriptions. Based on this structure, the main operation performed by HEM during the matching process is bitwise OR.
[0041] like Figure 1As shown, the data structure of HEM is constructed, and the subscription is indexed by the hardware-aware data structure; the data structure of HEM includes a multi-level index layer and a predetermined group of bit sets, and each attribute in the content space corresponds to a predetermined number of bit sets; the first-level index layer is constructed based on the attribute, the second-level index layer indexes the low value end and the high value end of the predicate in the subscription, and the third-level index layer is constructed by dividing the value range of the attribute into multiple cells; each cell corresponds to a bucket for storing the predicate value of the predicate and the number (ID) of the corresponding subscription. Specifically, the data structure of HEM consists of a three-level index layer and a set of bit sets, and each attribute corresponds to two bit sets. The first-level index is constructed based on the attribute, the second level indexes the low value end (LVE) and the high value end (HVE) of the predicate, and the third level is constructed by dividing the value range of the attribute into multiple cells. Each cell corresponds to a bucket for storing the low value or high value of the predicate and the corresponding subscription ID.
[0042] Each low-valued end or high-valued end is associated with a predetermined set of bits, which are used to pre-mark the subscription as unmatched. When a subscription S is inserted into the HEM data structure, the predicate value of each predicate in S will be mapped to the cell containing the predicate value at the low-valued end or high-valued end of the corresponding attribute; the predicate value and subscription ID are stored in pairs in the bucket corresponding to the cell.
[0043] Specifically, each LVE or HVE is associated with a set of bits that are used to pre-mark certain subscriptions as unmatched. When a subscription S is inserted into the HEM data structure, the low / high value of each predicate in S will be mapped to the cell containing the predicate value on the LVE / HVE side of the corresponding attribute. The predicate value (low value or high value) and the subscription ID are stored in pairs in the bucket corresponding to the cell.
[0044] HEM uses a reverse matching method, first searching for unmatched subscriptions to indirectly obtain matching subscriptions. Given a cell partitioning scheme on the attribute value domain, if an attribute event value v falls into a cell c j (c j denotes the jth cell number), all low-valued predicates with LVE ends greater than v definitely do not match the event. Specifically, the unsatisfied predicates are stored in the j+1 Similarly, on the HVE side, the cells stored in the range from c1 to c j-1 All predicates with high values less than v in cells of should also be marked as unmatched. On the LVE and HVE sides, the event value should be matched with cell c respectively. j The predicates in the are compared one by one to identify unmatched subscriptions.
[0045] Subscription pre-marking cache optimization method: HEM uses a cache mechanism to allocate cells into different groups; each group is associated with a preset number of bit sets; HEM pre-marks subscriptions in each group as mismatched in the corresponding bit set.
[0046] Specifically, HEM avoids marking unmatched subscriptions as much as possible during the matching process. HEM uses a cache mechanism to distribute cells into different groups. Each group is associated with a bit set. HEM pre-marks the subscriptions in each group as unmatched in the corresponding bit set. This optimization is called the Subscription Premark Cache (SPC) method. In this way, HEM avoids performing costly traversal and marking operations during event matching, and instead performs bitwise OR operations that are more efficient for hardware.
[0047] Let g be the number of groups and c be the number of cells at the LVE or HVE end. In particular, for each group g i Contains a continuous cell sequence, where 1≤i≤the number of groups. At the LVE end, group g i Before including cells, and at the HVE end, group g i After including Since the number of bit sets is equal to the number of groups, bit set B i Used to mark group g i In this way, the coverage length of the bit set constitutes a Arithmetic progression of common difference. g i Indicates the i-th group number, bit set B i Indicates the i-th bit set number.
[0048] Figure 1 The HEM data structure shown is indexed on two attributes a1 and a2. The range of each attribute is divided into 16 cells and assigned to 4 groups. Specifically, at the LVE end of each attribute, group g1 contains the values from c 13 to c 16 Cells, G2 from C9 to C 16 , g3 from c5 to c 16 , g4 from c1 to c 16 In contrast, the cells on the HVE side are grouped starting from c1. The size of the bit set associated with each group is equal to the number of subscriptions n. Each bit set is used to mark the subscriptions stored in the cells belonging to this group.
[0049] Subscription Insertion Step: Inserting a subscription S into the HEM data structure involves two steps. First, for each predicate in S, the predicate value and subscription ID are stored in pairs in the cell containing the predicate value, based on the attribute and low / high value (collectively referred to as the predicate value). Second, the subscription is pre-marked as mismatched in one or more bit sets, depending on the cell grouping scheme. Figure 2 A flowchart of the insertion algorithm is shown. a i represents the i-th attribute; l represents the low value of the predicate, h represents the high value of the predicate, and [l,h] as a whole defines an interval constraint; (l, S.ID) means combining the low value l and the ID of the subscription S.
[0050] Figure 1 Shows the Figure 3 The five sample subscriptions (S1, S2, S3, S4, and S5) shown are in the state after being inserted into the data structure. For example, when subscription S1 is inserted, for its first predicate, based on its low / high value and the cell partitioning scheme of the attribute, it is calculated and So at the LVE and HVE ends of the a1 attribute, S1 is mapped to cell c respectively. 15 and c 16 For brevity, Figure 1 The predicate value is omitted. Then, according to the grouping scheme, c 15 is included in all 4 groups on the LVE side, so S1 is marked in 4 bit sets. On the HVE side, c 16 is only contained in the g4 group, so S1 is marked in the corresponding bit set B4. The second predicate of S1 is processed similarly.
[0051] Event matching step: HEM uses bit set B to record the matching results. Each unmarked bit represents a matching subscription. Figure 4 The matching process of HEM is given in six steps. The first four steps are to process each attribute-value pair in the event. j represents the jth attribute; c l represents the cell where the attribute value v falls on the LVE side; c h represents the cell where the attribute value v falls on the HVE side; g l Indicates the number of the group selected by the LVE end; g h Indicates the number of the group selected by the HVE end.
[0052] Step 1 performs comparison operations in cells where event values fall on the LVE side and the HVE side respectively to determine unmatched subscriptions, and marks these unmatched subscriptions in bit set B.
[0053] Step 2 Select the largest group that does not contain the cell to which the event value belongs. Note: On the LVE side, when the cell ID is greater than or equal to , or on the HVE side, when the cell ID is less than or equal to , no such group exists.
[0054] Step 3 For each non-null attribute in event, when there is a maximum group on both the LVE side and the HVE side that contains the cell to which the event value belongs, perform a bitwise OR operation between B and the bit set of the selected group.
[0055] Step 4 marks in B the unmatched subscriptions stored in the cells not covered by the selected group.
[0056] Step 5 performs a bitwise OR operation between B and the bit set of the maximal group of each empty attribute of the event. When an event does not contain a certain attribute, it is certain that all predicates defined on the attribute are not satisfied.
[0057] Step 6 checks the unmarked bits in B for a matching subscription.
[0058] HEM reduces the traversal and marking operations in the matching process by setting a set of bits to pre-mark certain subscriptions as unmatched. Therefore, HEM mainly performs bitwise OR operations when matching events. Generally speaking, it is more efficient to perform bitwise OR operations to simultaneously mark unmatched subscriptions stored in multiple cells than to traverse each cell to mark each unmatched subscription. When the selected maximum group covers all cells that need to be traversed, the marking time spent by HEM is almost zero. For each attribute, the total number of cells to be traversed on the LVE side and the HVE side is a constant, that is,
[0059] Performance quantification model: We build an analysis model for HEM to quantify the relationship between performance improvement and memory consumption. To evaluate the improvement of marking time, we use the HVE side of an attribute as a breakthrough point, because the LVE side and HVE side of each attribute have similar characteristics.
[0060] Given a size of ψ S For n subscriptions, without using the subscription pre-marking cache optimization method, the marking time of HEM is Because the HEM marking time is proportional to the number of operations of marking subscription, and the calculation formula of marking times is:
[0061]
[0062] Among them, ψ S represents the subscription size, dx can be seen as a probability, and x is a possible value of an event. The interval to be traversed is [0,x].
[0063] Given the number of subscriptions n and the subscription size ψ S and the number of groups g, the subscription pre-marking cache optimization method doubles g, and the HEM marking time is halved. Therefore, the number of tagged subscriptions when using the subscription pre-tagging cache optimization method is calculated as follows:
[0064]
[0065] in Any event value The length of the interval that needs to be marked. Since the marking time is proportional to the number of mark subscriptions, doubling g will halve the marking time.
[0066] Given the number of subscriptions n and the event size ψ E , subscription sizeψ S , the number of attributes d and the number of groups g, when ψ E =d,g>1 and the predicate values are evenly distributed in the value range [0,1], the reduction rate of marking time brought by subscription pre-marking cache optimization method to HEM is Based on formula (1) and formula (2), the decrease rate of HEM marking times is This means that when g is set to 2 or 32, respectively, 50% or 96.875% of the marking operations can be reduced. This ratio does not hold when g = 1, because it assumes that the unique bit set at the LVE end or the HVE end covers all cells. However, when g = 1, it is possible to configure the unique bit set at the LVE end or the HVE end to only cover half of the cells. Therefore, when the event size ψ E Equal to the number of attributes d, the improvement of g=1 is equivalent to the improvement of g=2.
[0067] The embodiment of the present invention also discloses an event matching system based on hardware computing perception in a content distribution system, including the following modules: a subscription module: submitting a subscription. A publishing module: generating a publishing event. A matching module: performing event matching to search for a matching subscription, and sending the event of the matching subscription to the subscriber to which the matching subscription belongs. The hardware-aware event matching algorithm HEM is used to reduce the time of performing repeated operations in the event matching process.
[0068] The embodiment of the present invention also discloses an event matching method based on hardware computing perception in a content distribution system. The typical application scenario is to realize the distribution of real-time stock events. Subscribers subscribe to events related to their investment strategies, for example, when the price of a given stock exceeds or falls below a specified threshold. Based on the content publishing / subscribing system, an event matching algorithm is executed to quickly send events that meet the subscription conditions to the corresponding subscribers.
[0069] The present invention discloses a novel hardware-aware event matching algorithm HEM based on bit set OR operation. The design concept of HEM is to make full use of bit set OR operation performed in parallel in the event matching process to improve the event matching performance. Specifically, HEM adopts a reverse method to obtain matching results, similar to REIN and GEM. HEM uses a subscription pre-marking cache mechanism to reduce repeated traversal and marking operations in the matching process, and replaces them with efficient bit OR operations. HEM follows the concept of trading space for time, and establishes a theoretical model to quantify the trade-off between performance improvement and memory consumption.
[0070] Content-based publish / subscribe systems are widely used in many fields to achieve selective fine-grained data distribution. Event matching is a key component in content-based publish / subscribe systems. Its performance directly affects the latency and service quality of event distribution and is one of the bottlenecks restricting the overall performance of the system. Researchers have proposed many efficient algorithms to improve matching performance. However, existing work rarely considers hardware characteristics when designing matching algorithms. Since a large number of operations such as comparison, addition, and assignment need to be repeatedly performed during event matching, using hardware to perform costly operations may lead to performance degradation of matching algorithms. In the present invention, a hardware-aware event matching algorithm called HEM is proposed. The basic idea of HEM design is to perform as many bitwise OR operations as possible during event matching, which is the most efficient for computer hardware. In addition, the present invention constructs a performance analysis model to quantify the trade-off between memory consumption and performance improvement. The present invention also conducts a large number of experiments to evaluate the performance of HEM, which can reduce the matching time by about 86.8% compared with similar matching algorithms.
[0071] Since the efficiency of the event matching algorithm is crucial to the performance of the content-based publish / subscribe system, the data structure designed by the present invention and the matching algorithm proposed can fully consider the efficiency differences in hardware executing different operations, and improve the matching performance of the matching algorithm by performing efficient bit OR operations during the matching process.
[0072] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.
[0073] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. An event matching method based on hardware computing perception in a content distribution system, characterized in that: The steps include: Subscription steps: Submit subscription; Release step: Generate release events; Matching step: Event matching is performed through the hardware-aware event matching algorithm HEM to search for matching subscriptions, and the events of the matching subscriptions are sent to the subscribers to which the matching subscriptions belong; In the matching step, a data structure of the HEM is constructed, and the subscription is indexed through the hardware-aware data structure; the data structure of the HEM includes a multi-level index layer and a predetermined set of digital bits, and each attribute corresponds to a predetermined set of digital bits; the first level index layer is constructed based on the attribute, the second level index layer indexes the low-value end and the high-value end of the predicate, and the third level index layer is constructed by dividing the value range of the attribute into multiple cells; each cell corresponds to a storage bucket for storing the predicate value of the predicate and the corresponding subscription number; The matching step specifically includes the following steps: Step 1: Perform comparison operations in cells where event values fall on the low value side and the high value side respectively to determine unmatched subscriptions, and mark these unmatched subscriptions in bit set B; Step 2: Select the largest group that does not contain the cell to which the event value belongs; Step 3: For each non-null attribute in the event, when there is a maximum group containing the cell to which the event value belongs on both the low-value side and the high-value side, perform a bitwise OR operation between B and the bit set of the selected group; Step 4: Mark in B the unmatched subscriptions stored in the cells not covered by the selected group; Step 5: Perform a bitwise OR operation between B and the bit set of the maximal group of each empty attribute of the event. When the event does not contain a certain attribute, it must not satisfy all predicates defined on the attribute. Step 6: Check the unmarked bits in B for matching subscriptions.
2. The event matching method based on hardware computing perception in a content distribution system according to claim 1, characterized in that: In the matching step, the time of performing repeated operations in the event matching process is reduced by using a hardware-aware event matching algorithm HEM.
3. The event matching method based on hardware computing perception in a content distribution system according to claim 1, characterized in that: In the matching step, the HEM is optimized by subscribing to the pre-tagged cache swap space.
4. The event matching method based on hardware computing perception in a content distribution system according to claim 1, characterized in that: In the matching step, a performance analysis model is built for the HEM to quantify the relationship between memory consumption and performance improvement.
5. The event matching method based on hardware computing perception in a content distribution system according to claim 1, characterized in that: In the matching step, each low-value end or high-value end is associated with a predetermined set of digits, which are used to pre-mark the subscription as unmatched.
6. The event matching method based on hardware computing perception in a content distribution system according to claim 1, characterized in that: In the matching step, when the subscription S is inserted into the data structure of the HEM, the predicate value of each predicate in the subscription S will be mapped to the cell containing the predicate value at the low value end or the high value end of the corresponding attribute respectively; The predicate value and subscription number are stored in pairs in the bucket corresponding to the cell.
7. The event matching method based on hardware computing perception in a content distribution system according to claim 6, characterized in that: In the matching step, HEM uses a cache mechanism to distribute cells into different groups; each group is associated with a preset number of bit sets; HEM pre-marks subscriptions in each group as unmatched in the corresponding bit set.
8. An event matching system based on hardware computing perception in a content distribution system, characterized in that: Includes the following modules: Subscription module: submit subscription; Release module: generates release events; Matching module: performs event matching through the hardware-aware event matching algorithm HEM to search for matching subscriptions, and sends the events of matching subscriptions to the subscribers to which the matching subscriptions belong; In the matching module, a data structure of the HEM is constructed, and a subscription is indexed through a hardware-aware data structure; the data structure of the HEM includes a multi-level indexing layer and a predetermined set of digits, and each attribute corresponds to a predetermined set of digits; the first-level indexing layer is constructed based on the attribute, the second-level indexing layer indexes the low-value end and the high-value end of the predicate, and the third-level indexing layer is constructed by dividing the value range of the attribute into multiple cells; each cell corresponds to a storage bucket for storing the predicate value of the predicate and the corresponding subscription number; The matching module specifically includes the following modules: Module 1: Perform comparison operations in cells where event values fall on the low value side and the high value side respectively to determine unmatched subscriptions, and mark these unmatched subscriptions in bit set B; Module 2: Select the largest group that does not contain the cell to which the event value belongs; Module 3: For each non-null attribute in the event, when there is a maximum group containing the cell to which the event value belongs on both the low-value side and the high-value side, perform a bitwise OR operation between B and the bit set of the selected group; Module 4: marking unmatched subscriptions stored in cells not covered by the selected group in B; Module 5: Perform a bitwise OR operation between B and the bit set of the maximal group of each empty attribute of the event, when the event does not contain a certain attribute, it is certain that all predicates defined on this attribute are not satisfied; Module 6: Check the unmarked bits in B for matching subscriptions.
9. The event matching system based on hardware computing perception in a content distribution system according to claim 8, characterized in that: In the matching module, the time for performing repeated operations in the event matching process is reduced by using a hardware-aware event matching algorithm HEM.
Citation Information
Patent Citations
Rapid event matching method in content-based publishing subscription system
CN105740337A
Efficient matching method and system for space-time perception publishing / subscribing system
CN111949913A