Complex event processing method based on extended regular tree mode

Abstract

The invention discloses a complex event processing method based on an extended regular tree mode, and relates to the technical field of complex event processing. According to the method, the ERTP is used as a unified query mode description mode and comprises a node group, a regular type group, a structural relation axis group, a regular type assignment axis group, a conditional expression group and a return requirement, so that a structure, a time sequence and attribute constraints are expressed in a single mode at the same time, and query processing based on the ERTP comprises the steps that structure matching is executed on a tree mode part; partial matching results are stored in an instance set; when the root node list is updated, recursively enumerating and checking the condition expression, and outputting an event corresponding to a return node; performing symbol storage automaton (SMA)-based incremental timing matching on the regular portion; according to the method and the device, intermediate result generation and overhead can be reduced, and streaming complex event processing efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of complex event processing technology, specifically a complex event processing method based on an extended normal tree pattern. Background Technology

[0002] Complex Event Processing (CEP) is a technology used to identify combinations of events that satisfy preset patterns in a continuously input event stream and output matching results. CEP can be applied to scenarios such as real-time data monitoring, alerting, and analysis. With the evolution of data sources and business needs, input data is often semi-structured. For this type of data, query patterns typically need to express multiple types of constraints simultaneously, including: structural constraints between events (e.g., hierarchical relationships such as parent-child, ancestor-descendant, etc.), temporal constraints (e.g., sequence constraints such as order of occurrence, repetition, and selection), and attribute-based relational constraints (e.g., comparisons between multiple event attributes, equality or threshold judgments, etc.).

[0003] Some existing CEP (Continuous Event Processing) solutions use automata and other models to represent and execute event sequence patterns. They incrementally match input events through state transitions and filter event attributes using predicate conditions. This type of solution is common when handling sequence-based temporal constraints. However, when query patterns further include constraints on hierarchical structures between events (e.g., simultaneously satisfying tree-like and sequence relationships), pattern representation and execution typically require additional data structures or phased processing mechanisms to meet both structural and sequence matching needs. In this case, the matching process may generate numerous intermediate results, increasing storage and computational overhead. Furthermore, in query scenarios that simultaneously include structural constraints, temporal constraints, and cross-event attribute relationship constraints, some solutions lack sufficient support in terms of the organization of relationship constraints and a unified execution flow. Therefore, how to uniformly describe structural constraints, temporal constraints, and cross-event attribute relationship constraints within the same query pattern, and reduce intermediate results and processing overhead during streaming processing, remains one of the technical problems that need to be solved in this field. Summary of the Invention

[0004] This application provides a complex event processing method based on the extended normal tree pattern, which allows users to describe queries containing structural constraints, temporal constraints, and cross-event attribute relationship constraints in the same query pattern through ERTP. It eliminates the need to describe queries in stages, and can comprehensively consider various constraints in the query pattern to filter events and organize some results. This reduces the generation of useless intermediate results to a certain extent, improves query efficiency, and can effectively solve the problems in the background technology.

[0005] To achieve the above objectives, this application provides the following technical solution: a complex event processing method based on the extended normal tree pattern, wherein the ERTP includes a node group, a normal expression group, a structural relationship axis group, a normal expression assignment axis group, a conditional expression group, and a return requirement.

[0006] The structure is as follows: Node groups consist of nodes that are identifiers used to indicate the selection of a class of events; Regular expression groups are composed of nodes combined using operators such as sequence, selection, and Clyne closure, used to describe the temporal constraints between events selected by nodes; Structural relationship axis groups include parent-child (PC) and ancestor-descendant (AD) axes, used to indicate parent-child or ancestor-descendant relationships between events selected by nodes; Regular expression assignment axis groups indicate that each event in a sequence of events matched by a regular expression is a child event of an event selected by another node; Conditional expression groups are expressions that reference multiple nodes or constants, used to describe relational constraints across event attributes; Return requirements indicate which events corresponding to which nodes should be returned as output.

[0007] Furthermore, the nodes in the ERTP form a tree pattern through structural relationship axes, with each node having at most one parent node; nodes with child nodes are not regular nodes; nodes that are bound to at least another node in a structural relationship or are themselves regular nodes are called nodes in the tree pattern.

[0008] Based on the aforementioned ERTP, this application provides a query method for matching ERTP from an input event stream and outputting results. The input event stream arrives in a structural preorder order, and a structural relationship can be determined between any two events. The method performs structural matching processing on the tree-pattern portion of the ERTP, composed of nodes and structural relationship axes, and performs automata-based temporal matching processing on the regular expression portion. The partial matching results generated by structural and temporal matching are written to a shared instance set. When the instance set is updated, the matching results are enumerated and checked to determine and output the events that need to be returned.

[0009] In addition to a set of states, a start state, and a set of end states, the automaton also includes a predicate-based transition and a return requirement. The predicate-based transition is associated with the conditional expression and is used to check the events of multiple node selections during state transitions. The return requirement is used to indicate which events in the matching results need to be output.

[0010] The instance set includes multiple lists, each corresponding to a node in the tree pattern, used to store events or event sequences of partial matching results; each event in the list is associated with a set of pointer pairs (including a start pointer and an end pointer) to define the range of descendant events of the corresponding child node or associated regular expression; the events in the list corresponding to the regular expression node are additionally associated with the automaton running instance group, the candidate list, and the result list.

[0011] Compared with the prior art, the beneficial effects of this application are:

[0012] This application achieves a unified description of structural constraints, temporal constraints, and cross-event attribute relationship constraints through ERTP, avoiding the complexity brought about by staged queries; it reduces the generation and storage overhead of intermediate results through a shared instance set and pointer mechanism; and it realizes unified checking of conditional expressions through predicate transfer in the automaton, improving query efficiency and accuracy. Attached Figure Description

[0013] Figure 1 This is an example diagram of the semi-structured event flow and its tree structure in this application.

[0014] Figure 2 This is an example diagram of the ERTP in this application.

[0015] Figure 3 This is an example diagram of SMA corresponding to the formal "D:F" of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] Please see Figure 1-3 This application provides the following technical solutions:

[0018] Example 1:

[0019] A complex event processing method based on the Extended Regular Tree (ERTP) pattern is presented in this embodiment, running on a general-purpose computing device including a processor, memory, and input / output interfaces. The input is the ERTP as the query pattern, and the event stream arriving in the preorder traversal order. Each event includes at least an event identifier, event type, structural position code (using prefix encoding, such as "1", "1.2", "1.2.3"), and a set of attributes (key-value pairs). The output is the set of events corresponding to the returned nodes from all matching results obtained by matching the query pattern.

[0020] The input includes: the extended normal tree pattern ERTP to be queried, as the query pattern; and a semi-structured event stream arrived in the structured preorder traversal order.

[0021] Each event must contain at least the following information: event identifier; event type (a string used to match nodes in ERTP); structural location encoding (using prefix encoding, such as "1", "1.2", "1.2.3"); and attribute set (represented as key-value pairs).

[0022] The output is: the set of all events corresponding to the ERTP returned nodes from all matching results obtained by matching the ERTP on the input event stream.

[0023] Structural position coding and structural relationship determination

[0024] This embodiment uses prefix coding as the structural position coding. For any two events e1 and e2, if the code of e2 is a strict prefix of e1, then e1 is an ancestor event of e2 and e2 is a descendant event of e1; if the code of e2 is a prefix of e1 and the lengths of the two differ by 1, then e1 is a parent event of e2 and e2 is a child event of e1; otherwise, there is no ancestor / parent-child relationship between the two.

[0025] ERTP data structure

[0026] ERTP is defined as an octet: ERTP := (N, root, R, Γ, α, ι, p, o), where:

[0027] N is the set of nodes that appear in ERTP. Each node is identified by a string and represents a type of event.

[0028] root ∈ N is the root node;

[0029] R is the set of regular expressions that appear in ERTP;

[0030] Γ is the set of conditional expressions that appear in ERTP;

[0031] α : (N - {root}) → (N × {PC, AD}) is a structural relationship binding function that binds a given node to another node with a parent-child (PC) or ancestor-descendant (AD) relationship;

[0032] ι : N ⇀ R is a partial function that associates a regular expression with a node, indicating that each event in the sequence of events matched by the regular expression is a sub-event of the event selected by the node;

[0033] p : N → Γ is a function that binds a conditional expression to a node;

[0034] o : N → {true, false} is a function that indicates whether a node is a return node. If o(n) = true, then the event selected by n should be returned as output.

[0035] In this embodiment, the supported forms of conditional expressions include:

[0036] A.value = B.value (meaning that the value property of the selection events of nodes A and B are equal);

[0037] D.value = F.value (same as above);

[0038] T (meaning no constraints are imposed).

[0039] Symbolic Storage Automaton (SMA)

[0040] For time-series matching in regular expressions, it is defined as a quadruple: SMA := (Q, qs, QT, ∆), where:

[0041] Q is a finite set of states;

[0042] qs ∈ Q is the initial state;

[0043] QT ⊆ Q is the set of termination states;

[0044] ∆ is the transition set.

[0045] Each transition δ ∈ ∆ is a quintuple (qf, qt, nδ, c, ρ), where:

[0046] qf ∈ Q is the source state;

[0047] qt ∈ Q is the target state;

[0048] nδ ∈ N are associated nodes;

[0049] c ∈ Γ is the transition predicate (conditional expression);

[0050] ρ ∈ {true, false} indicates whether the event read in through this transition should be output.

[0051] The running instance is defined as run = (q, φ), where q ∈ Q represents the current state, and φ is the mapping from nodes to events (partial functions), referred to as the current environment. The initial running instance is q = qs with φ being empty.

[0052] Instance collection data structure

[0053] The instance set L consists of multiple lists, corresponding to a list Ln for each node n in the tree pattern of ERTP.

[0054] For any list Ln corresponding to node n:

[0055] When event e is added to Ln, e is associated with two pointer sets, starts and ends (each set contains k pointers, where k is the number of child nodes of n). For each child node m of n, start_m and end_m are the start pointer and end pointer, respectively, used to define the range of events in Ln that belong to the selection of m.

[0056] If n is a normal node, then e has additional associations:

[0057] runs: A collection of SMA run instances used to maintain multiple timing match attempts in the sub-event sequence of e;

[0058] CList: A candidate list that stores candidate result events that have not yet been fully matched.

[0059] RList: A list of results that stores the sequence of matching events.

[0060] Events in a list can be appended with sibling pointers to indicate the position of the next sibling event in the same list (optional implementation).

[0061] The query method in this embodiment includes the following main steps:

[0062] S1 Initialization: Initialize an empty list for each node in the tree pattern of ERTP, forming the instance set L. Construct the corresponding SMA for each regular expression R in ERTP: in the transition, node nδ is set according to o(nδ) in ERTP; the predicate c is directly taken from the conditional expression of the corresponding transition.

[0063] S2 Structure Matching Processing: The arriving event stream is processed using a matching stack. When event e arrives:

[0064] The corresponding node n is determined based on the event type of e (if no match is found, it is discarded).

[0065] If n is a node in a tree pattern: initialize the starts and ends pointers for e according to the number of child nodes of n, with each pointer initially pointing to the position after the current end of the corresponding child node list; push e onto the matching stack to wait;

[0066] If n is a regular node: additionally initialize runs (empty), CList (empty), and RList (empty) for e.

[0067] When all descendants of the event 'e' at the top of the matching stack have been processed (i.e., an event that is not a descendant of 'e' is encountered, or the stream ends):

[0068] Pop e from the stack;

[0069] Update the starts and ends pointers of e to point to the current end position of the corresponding child node list (including all possible descendants).

[0070] Check the structural relationships between 'e' and each of its child nodes:

[0071] Ancestors and Descendants (AD): Check if there are any events in the corresponding start_m and end_m ranges;

[0072] Parent-Child (PC): Check if there are direct child events (length difference of 1) within the range;

[0073] If all the binding structural relationships are satisfied, and if n is a regular node then runs is not empty, then add e to Ln;

[0074] If the currently arriving event is a child event of the regular node event at the top of the stack, then the timing matching process (S3) of that regular node is also triggered.

[0075] S3 Time-Sequence Matching Processing: For a regular expression associated with a regular node n, when its child events arrive:

[0076] The parent event (corresponding to n) adds the initial running instance of SMA to its runs;

[0077] For each run instance in runs:

[0078] Update the run environment φ: Add the mapping from node nδ to e corresponding to the currently arriving event;

[0079] Attempt to perform state transition based on the transition in ∆: check the transition predicate c (evaluate after replacing the event attribute in φ);

[0080] If the migration is successful and ρ=true, then the current event is added to the parent event's CList;

[0081] If the current state q ∈ QT after the migration, then all of the parent event's CList will be merged into RList, and CList will be cleared.

[0082] S4 Result Enumeration: Result enumeration is triggered when the list Lroot corresponding to the root node in the instance collection is updated (a new event is added): Starting from the newly added event in Lroot, the enumeration process is executed recursively.

[0083] Update the current environment: Add the node corresponding to the current event → Event Mapping;

[0084] Check the conditional expression p(n) bound to the current node (pass if it is T); if it fails, return "failed" and no output event.

[0085] If it passes, then recursively check the descendants:

[0086] If the current node is not a regular node: For each child node m, select events that satisfy the structural constraints within the range of start_m ~ end_m, and recursively call the enumeration. For each child node m, at least one must pass.

[0087] If the current node is a normal node: recursively call the enumeration for each event in RList, at least one must pass;

[0088] Collect all events corresponding to nodes on the path where o(n)=true, and return them as the output.

[0089] by Figure 1 The semi-structured event flow and its tree structure shown are as follows: Figure 2 The ERTP shown below is an example:

[0090] The ERTP requires: event A has a child event B and a descendant event C, and the values ​​of A and B are equal; B has a sequence D:F (the values ​​of D and F are equal), and F has a child event G; return events C, D, and G.

[0091] The SMA corresponding to the regular form "D:F" is as follows: Figure 3 As shown. During the processing:

[0092] Events arrive one by one, the structure matches and the pointer is updated and written to the instance collection;

[0093] Event B, acting as a regular node, triggers a "D:F" match.

[0094] When D and F arrive, update runs, check if the value is equal to the predicate, and record the results in RList after the matching is completed.

[0095] After the root list A is updated, enumerate starting from A, pass the environment step by step, pass the condition check, and finally output the matching C, D, and G events.

[0096] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A complex event processing method based on the extended normal tree pattern, characterized in that, include: An Extended Regular Tree (ERTP) schema is constructed to uniformly describe structural constraints, temporal constraints, and cross-event attribute relationship constraints within the same query schema. The ERTP includes node groups, regular expression groups, structural relationship axis groups, regular expression assignment axis groups, conditional expression groups, and return requirements. The structural relationship axis groups include parent-child axes and ancestor-descendant axes. Based on the ERTP, matching is performed from the event stream that arrives in structural preorder and the structural relationship between any two events can be determined, and the events that need to be returned in the matching results are output. The matching process includes structural matching for the tree pattern part, automata-based temporal matching for the regular form part, and writing the partial matching results of both into a shared instance set, and performing enumeration and condition checks when the instance set is updated to determine the output result.

2. The complex event processing method based on the extended normal tree pattern according to claim 1, characterized in that: In the ERTP, nodes form a tree pattern through structural relationship axes, and each node has at most one parent node; a node with child nodes is not a regular node; a node that is bound to at least another node in a structural relationship or is a regular node itself is called a node in the tree pattern.

3. The complex event processing method based on the extended normal tree pattern according to claim 1, characterized in that: In addition to the state set, start state, and end state set, the automaton used in the timing matching process also includes: A transition with a predicate, the predicate being associated with the conditional expression, for checking events selected from multiple nodes during state transition; and a return requirement for indicating which events should be output.

4. The complex event processing method based on the extended normal tree pattern according to claim 1, characterized in that: The instance set contains multiple lists, each corresponding to a node in the tree pattern of ERTP, used to store events or event sequences of partial matching results; Each event in the list is associated with a pair of pointers, including a start pointer and an end pointer, which are used to define the scope of the corresponding child nodes or the descendant events of the associated regular expression.

5. A complex event processing method based on an extended normal tree pattern according to claim 4, characterized in that: In the list of corresponding regular nodes, each event is additionally associated with: A set of automaton instances is used to maintain multiple temporal matching attempts in the sequence of sub-events of this event; A candidate list is used to store candidate results that have not yet been fully matched. The results list is used to store the results of the completed matches.

6. The complex event processing method based on the extended normal tree pattern according to claim 1, characterized in that: Each event in the list of the instance set is appended with a sibling pointer to indicate the position of the next sibling event in the same list.

7. A complex event processing method based on an extended normal tree pattern according to claim 1, characterized in that, Includes the following steps: S1 Initialization: Create an empty list to form an instance set for the nodes in the tree pattern of ERTP, and build a corresponding automaton for each regular expression; S2 Structure Matching Processing: Filter the corresponding nodes according to the arrival event type, initialize pointer pairs, and check the structural relationship satisfaction after all descendants of the event have been processed. Write the events that meet the conditions into the corresponding list of the instance set. If it is a regular node, then additionally associate it with the automaton running instance group, the candidate list, and the result list; If it is a child event of a regular node event, then timing matching is triggered; S3 time-series matching processing: Performs incremental matching based on automata on the regular expression associated with regular nodes, updates the multi-event cache of the running instance, determines the state transition based on the predicate transition check condition, and maintains the candidate list and result list accordingly; S4 Result Enumeration: When the list corresponding to the root node is updated, recursively enumerate from the newly added event, combine environment update and condition expression validation, collect and output all events corresponding to the returned nodes that pass the validation.

8. A complex event processing method based on an extended normal tree pattern according to claim 7, characterized in that: The structure matching process uses a matching stack mechanism. When all descendants of an event have been processed, the event is popped from the stack, and the range is updated according to the pointer pair before checking whether the parent-child / ancestor-descendant relationship is satisfied.

9. A complex event processing method based on an extended normal tree pattern according to claim 7, characterized in that: In the timing matching process, when reading events for each running instance: Update the multi-event cache; The event attributes in the current cache are checked based on the transfer predicate to determine whether to perform a state transfer; If the transfer is successful and the event needs to be output, add it to the candidate list; If the termination state is reached, the candidate list is merged into the result list and the candidate list is cleared.

10. A complex event processing method based on an extended normal tree pattern according to claim 7, characterized in that: The result enumeration process includes: updating the current environment and checking the conditional expression bound to the node; recursively calling the enumeration on descendants; for non-normal nodes, each child node must have at least one descendant that passes the check; for normal nodes, at least one descendant of an event in the result list must pass the check; and collecting the events corresponding to all returned nodes on the path as output.

Images