A complex event detection method and device, and a storage medium

Abstract

The embodiment of the application discloses a complex event detection method and device, and a storage medium, comprising: acquiring an operation event stream, the number of events in the operation event stream being multiple; binding multiple event rules and multiple event determination logics to each event in the operation event stream; determining the matching between the operation event stream and the bound multiple event rules by using a calculation engine; in the case of matching, screening a target event with an operation event time less than a target water level from the operation event stream; the target water level is a time determined according to multiple operation event times corresponding to the operation event stream; determining a determination result of the target event according to the multiple event determination logics, so as to determine that the target event is a complex event according to the determination result.

Description

Technical Field

[0001] This application relates to the field of financial technology, and in particular to a method and apparatus for detecting complex events, and a storage medium. Background Technology

[0002] With the development of computer technology, more and more technologies are being applied in the financial sector, and the traditional financial industry is gradually transforming into Fintech. However, due to the security and real-time requirements of the financial industry, Fintech also places higher demands on technology. In the field of Fintech, complex event detection is becoming increasingly important, and existing technologies mostly utilize Flink for complex event detection.

[0003] In existing technologies, Flink complex event detection relies on rule broadcast streams and predefined n rule computation units to achieve complex event detection. If the newly added rule is not in the predefined rule computation unit, the code for the new rule computation unit needs to be added first, and then the code needs to be compiled and run. This reduces the speed of adding complex event detection rules. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application aim to provide a complex event detection method, apparatus, and storage medium that can improve the speed of adding complex event detection rules.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides a method for detecting complex events, the method comprising:

[0007] Obtain an operation event stream, wherein the number of events in the operation event stream is multiple;

[0008] For each event in the operation event stream, bind multiple event rules and multiple event determination logic;

[0009] The matching between the operation event stream and the bound event rules is determined using a computing engine;

[0010] In the case of a match, target events with operation times less than the target water level are filtered from the operation event stream; the target water level is a time determined based on the operation times of multiple events corresponding to the operation event stream.

[0011] The determination result of the target event is determined based on the multiple event determination logic, and the target event is determined to be a complex event based on the determination result.

[0012] This application provides a complex event detection device, the device comprising:

[0013] An acquisition unit is used to acquire an operation event stream, wherein the operation event stream contains multiple events;

[0014] A binding unit is used to bind multiple event rules and multiple event determination logic to each event in the operation event stream;

[0015] The determination unit is used to use a computing engine to determine the matching between the operation event stream and the bound multiple event rules; and to determine the determination result of the target event based on the multiple event determination logic, so as to determine the target event as a complex event based on the determination result.

[0016] A filtering unit is used to filter target events whose operation time is less than a target water level from the operation event stream, provided that a match is found; the target water level is a time determined based on the operation times of multiple events corresponding to the operation event stream.

[0017] This application provides a complex event detection device, the device comprising:

[0018] The system includes a memory, a processor, and a communication bus. The memory communicates with the processor via the communication bus. The memory stores a complex event detection program that can be executed by the processor. When the complex event detection program is executed, the complex event detection method described above is executed by the processor.

[0019] This application provides a storage medium storing a computer program applied to a complex event detection device, characterized in that the computer program, when executed by a processor, implements the complex event detection method described above.

[0020] This application provides a method, apparatus, and storage medium for complex event detection. The method includes: acquiring an operation event stream, wherein the number of events in the operation event stream is multiple; binding multiple event rules and multiple event determination logic to each event in the operation event stream; using a calculation engine to determine the matching between the operation event stream and the bound multiple event rules; if matching is achieved, filtering target events from the operation event stream whose operation time is less than a target water level; the target water level is a time determined based on the operation time of the multiple events corresponding to the operation event stream; and determining the determination result of the target event based on the multiple event determination logic, thereby determining the target event as a complex event based on the determination result. Using the above method, the complex event detection apparatus, because the multiple event rules and multiple event determination logic are in the form of expressions, can directly broadcast the newly added complex event detection rules and event determination logic to the calculation engine when adding new complex event detection rules and event determination logic. The calculation engine can then parse and calculate the newly added rules in the form of expressions, eliminating the need to add new rule calculation unit code or compile and run the code, thus improving the speed of adding complex event detection rules. Attached Figure Description

[0021] Figure 1 A schematic diagram of a prior art complex event detection method provided in this application embodiment;

[0022] Figure 2 A flowchart of a complex event detection method provided in this application embodiment;

[0023] Figure 3 A flowchart illustrating an exemplary complex event detection method provided in this application embodiment;

[0024] Figure 4 An exemplary schematic diagram of complex event detection provided for embodiments of this application;

[0025] Figure 5 A schematic diagram of the composition structure of a complex event detection device provided in this application embodiment. Figure 1 ;

[0026] Figure 6 A schematic diagram of the composition structure of a complex event detection device provided in this application embodiment. Figure 2 . Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0028] The current mainstream Flink real-time complex event detection technology steps are as follows: Figure 1 As shown, Flink reads complex event detection rules to form a rule stream, then converts the rules into a broadcast stream using a broadcast operator. This broadcast stream is then bound to the event stream. Next, the keyBy operator groups and calculates the rules based on a fixed field (usually user ID or device ID). During the calculation process, all rules are first traversed, then each rule is identified using Java reflection. Finally, each event is fed into one of the corresponding n predefined rule calculation units for calculation, yielding the result. Specifically, the process includes the following steps:

[0029] 1. Flink reads complex event detection rules to form a rule stream. The rule format is typically a JSON string. For example, the complex event detection rule for "dcp user privacy authorization" is defined as:

[0030] {

[0031] "rule_id":"dcp_privacy_grant",

[0032] "rule_name":"dcp user privacy authorization",

[0033] "rule_begin_key":"defined_name",

[0034] "rule_begin_value":"dcp-login-succ",

[0035] "rule_middle_key":"action_name",

[0036] "rule_middle_value":"read-privacy-statement",

[0037] "rule_end_key":"click_name",

[0038] "rule_end_value":"agree",

[0039] "rule_calc_id":"calc_grant"

[0040] }

[0041] 2. Flink extracts the event time field from the real-time event stream to generate the waterline of the real-time event stream.

[0042] 3. Flink uses broadcast operators to convert rule streams into broadcast streams.

[0043] 4. Flink uses the Connect operator to bind the rule stream to the event stream, generating an event stream with the bound rules. For example, a certain event in the event stream might be:

[0044]

[0045] A specific event in the event stream of the binding rule is:

[0046]

[0047]

[0048] 5. Flink uses the keyBy operator to extract a fixed field (usually user ID, device ID, etc.) and transforms the event stream of the bound rules into a grouped stream for grouped computation. This includes the following specific steps:

[0049] 5.1. Flink iterates through each rule of each event in the grouped stream.

[0050] 5.2 Flink uses Java reflection to identify complex event detection rules bound to each event, including rule ID, rule name, start rule key, start rule value, intermediate rule key, intermediate rule value, end rule key, end rule value, rule calculation unit ID, and other information.

[0051] 5.3 Flink feeds each event into one of its predefined n rule computation units to calculate the result. The calculation methods for complex events are predefined within each rule computation unit. For example, the computation unit `calc_grant` for detecting the complex event "User Privacy Authorization" is defined as follows: If all three events occur (login success event + reading agreement action event + clicking the agree button event), it outputs YES, indicating that the complex event "User Privacy Authorization" has been detected.

[0052] The problems existing in the prior art can be solved by the method in Embodiment 1.

[0053] This application provides a method for detecting complex events, which is applied to a complex event detection device. Figure 2 A flowchart of a complex event detection method provided in this application embodiment is shown below. Figure 2 As shown, complex event detection methods may include:

[0054] S101. Obtain the operation event stream, where there are multiple events in the operation event stream.

[0055] The complex event detection method provided in this application embodiment is applicable to scenarios involving the detection of complex events.

[0056] In the embodiments of this application, the complex event detection apparatus can be implemented in various forms. For example, the complex event detection apparatus described in this application may include devices such as servers.

[0057] In this embodiment of the application, when the event tracking system detects a real-time operation event, the event tracking system reports the operation event to the Kafka message queue. The complex event detection device can periodically obtain multiple operation events from the Kafka message queue to obtain an operation event stream.

[0058] It should be noted that the complex event detection device can also obtain multiple operation events from the Kafka message queue within a preset time period to obtain an operation event stream; the complex event detection device can also obtain multiple operation events from the Kafka message queue to obtain an operation event stream when the number of operation events in the Kafka message queue is a preset number; the specifics can be determined according to the actual situation, and this application embodiment does not limit this.

[0059] It should be noted that the preset time period can be a time period configured in the complex event detection device; the preset time period can also be a time period input by the user into the complex event detection device; the specific way the complex event detection device obtains the preset time period can be determined according to the actual situation, and this application embodiment does not limit it.

[0060] It should be noted that the preset quantity can be the quantity configured in the complex event detection device; the preset quantity can also be the quantity transmitted to the complex event detection device by other devices; the specific way in which the complex event detection device obtains the preset quantity can be determined according to the actual situation, and this application embodiment does not limit it in this regard.

[0061] For example, taking a small loan product (DCP) as an example, the process of detecting the operation event "DCP user authorization behavior time consumption" is as follows: The event tracking system reports the operation event to the Kafka message queue in real time, and Flink consumes the real-time operation events in Kafka in real time to form a real-time operation event stream. A certain operation event in the operation event stream can be:

[0062]

[0063] It should be noted that each operation event in the operation event stream carries an event operation time.

[0064] S102. Bind multiple event rules and multiple event determination logic to each event in the operation event stream.

[0065] In this embodiment of the application, after the complex event detection device acquires the operation event stream, the complex event detection device can bind multiple event rules and multiple event determination logic to each event in the operation event stream.

[0066] In this embodiment of the application, after the complex event detection device binds multiple event rules and multiple event determination logics to each event in the operation event stream, if the multiple event rules or multiple event determination logics change, the complex event detection device will also change the multiple event rules or multiple event determination logics bound to the operation events.

[0067] It should be noted that multiple event rules can correspond one-to-one with multiple event determination logics, or multiple event rules can not correspond one-to-one with multiple event determination logics; the specifics can be determined according to the actual situation, and this application embodiment does not limit this.

[0068] In this embodiment, the complex event detection device can use Flink to bind multiple event rules and multiple event determination logic to each event in the operation event stream through the Connect operator; the complex event detection device can also use other methods to bind multiple event rules and multiple event determination logic to each event in the operation event stream; the specific method by which the complex event detection device binds multiple event rules and multiple event determination logic to each event in the operation event stream can be determined according to the actual situation, and this embodiment does not limit it.

[0069] In this embodiment of the application, before the complex event detection device binds multiple event rules and multiple event determination logic to each event in the operation event stream, the complex event detection device also receives the configured multiple event rules and multiple event determination logic; the complex event detection device broadcasts the multiple event rules and multiple event determination logic to the computing engine.

[0070] In this embodiment, the multiple event rules and multiple event determination logic can be rules and logic defined using Aviator expressions; the multiple event rules and multiple event determination logic can also be rules and logic defined using other methods; the specifics can be determined according to the actual situation, and this embodiment does not limit them.

[0071] In this embodiment of the application, after the complex event detection device receives the configured multiple event rules and multiple event determination logic, the complex event detection device can store the multiple event rules and multiple event determination logic in the database.

[0072] It should be noted that multiple event rules and multiple event determination logic include: rule number, whether the rule is enabled, creation time, last update time, product number, metric category, metric name, field name of the metric stored in the database, complex event category, complex event subclass, rule start condition, rule intermediate condition, rule end condition, calculation logic, and other information.

[0073] In this embodiment, the complex event detection device can use Flink to read Aviator expressions stored in the database every n seconds as multiple event rules and multiple event determination logic.

[0074] In this embodiment of the application, taking DCP as an example, multiple event rules and multiple event determination logics are defined using Aviator expressions. Flink reads the Aviator expressions stored in the database every n seconds to form "multiple event rules and multiple event determination logics":

[0075]

[0076]

[0077] In this embodiment, the complex event detection device can use Flink broadcast operators to broadcast multiple event rules and multiple event determination logics. The complex event detection device can also use other broadcast methods to broadcast multiple event rules and multiple event determination logics. The specific method by which the complex event detection device broadcasts multiple event rules and multiple event determination logics to the computing engine can be determined according to the actual situation, and this embodiment does not limit it.

[0078] In this embodiment of the application, taking DCP as an example, a broadcast operator is used to broadcast the "rules and computational logic flow" to form a broadcast flow.

[0079] S103. Use the calculation engine to determine the matching between the operation event flow and the multiple bound event rules.

[0080] In this embodiment of the application, after the complex event detection device binds multiple event rules and multiple event determination logic to each event in the operation event stream, the complex event detection device can use the calculation engine to determine the matching between the operation event stream and the bound multiple event rules.

[0081] In this embodiment, the calculation engine can be the Aviator expression engine; the calculation engine can also be other engines used for calculation functions; the specific engine can be determined according to the actual situation, and this embodiment does not limit it.

[0082] In this embodiment of the application, the Aviator expression engine can be called in stateFunc to determine the matching between the operation event stream and the multiple bound event rules.

[0083] In this embodiment of the application, the process of the complex event detection device using a computing engine to determine the matching between an operation event stream and multiple bound event rules includes: the complex event detection device acquiring preset fields in the operation event stream to obtain multiple field information; grouping the operation event stream according to the multiple field information to obtain multiple groups of operation events; determining the matching between each group of operation events and the multiple bound event rules, until the matching between the multiple groups of operation events and the multiple bound event rules is determined.

[0084] In this application embodiment, the preset field can be a field configured in the complex event detection device; the preset field can also be a field transmitted to the complex event detection device from other devices; the preset field can also be a field obtained by the complex event detection device in other ways; the specific way in which the complex event detection device obtains the preset field can be determined according to the actual situation, and this application embodiment does not limit this.

[0085] It should be noted that the preset field can be a user ID, a device ID, or other field information; the specific preset field can be determined according to the actual situation, and this application embodiment does not limit it.

[0086] It should also be noted that each operation event in the operation event stream carries field information corresponding to a preset field.

[0087] In this embodiment, the complex event detection device can use Flink's map operator to define a custom state calculation function `stateFunc`. This `stateFunc` then iterates through the multiple event rules and event determination logic bound to each of the multiple sets of operation events. Taking DCP as an example, it iterates through the multiple event rules and event determination logic bound to each of the multiple sets of operation events.

[0088] In this embodiment of the application, the process of the complex event detection device grouping the operation event stream according to multiple field information to obtain multiple sets of operation events includes: the complex event detection device verifying the multiple field information respectively to obtain multiple verification values; the complex event detection device performing hash processing on the multiple verification values ​​to obtain multiple hash information; and the complex event detection device grouping the operation event stream according to the multiple hash information to obtain multiple sets of operation events.

[0089] In this embodiment of the application, the complex event detection device performs hash processing on multiple check values ​​to obtain multiple hash information in several ways. These include the complex event detection device performing modulo processing on the operator parallelism based on multiple check values ​​to obtain multiple hash information; other methods may also be used by the complex event detection device to obtain multiple hash information based on multiple check values. The specific method can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0090] It should be noted that operator parallelism can be defined as the number of central processing units (CPUs) in a complex event detection device. For example, operator parallelism can be defined as a CPU with 200 cores.

[0091] In this embodiment of the application, the complex event detection device verifies multiple fields of information separately to obtain multiple verification values. This can be done by the complex event detection device using CRC32 verification to verify multiple fields of information separately to obtain multiple verification values; or by the complex event detection device using other verification methods to verify multiple fields of information separately to obtain multiple verification values. The specific method can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0092] It should be noted that multiple fields of information correspond one-to-one with multiple verification values, that is, one field of information corresponds to one verification value.

[0093] In this embodiment, the hash information is hashId. Multiple hash information items correspond one-to-one with multiple check values; that is, one hash information item corresponds to one check value.

[0094] In this embodiment of the application, taking DCP as an example, the CRC32 check value of the user_id field is extracted and modulo operation is performed on the parallelism of the operator 100, that is, hashId = mod(crc32('xxxxxxxxxxxxxxxxx001'), 100) + 1, resulting in the value hashId = 91.

[0095] In this embodiment, the complex event detection device can use multiple hash information as input parameters of Flink's keyBy operator to generate multiple sets of operation events.

[0096] In this embodiment of the application, taking DCP as an example, the keyBy operator groups the operation event streams bound to multiple event rules according to hashId and generates multiple groups of operation events grouped by key. For example, the operation event with user information user_id=xxxxxxxxxxxxxxxxx00 and the operation event with user information user_id=simulator_dcp_delta4_sit631 have the same hashId, that is, the operation events of these two users will be assigned to the same group. In this way, even if the user user_id=xxxxxxxxxxxxxxxxx001 does not have any subsequent operation events, as long as other users with hashId=91 have subsequent operation events, the water level of the current group can be raised, thereby triggering the calculation of the operation event data of the user user_id=xxxxxxxxxxxxxxxxx001.

[0097] S104. In the case of matching, filter target events from the operation event stream whose operation time is less than the target water level; the target water level is the time determined based on the operation times of multiple events corresponding to the operation event stream.

[0098] In this embodiment of the application, after the complex event detection device uses a computing engine to determine the matching between the operation event stream and the multiple bound event rules, the complex event detection device filters target events from the operation event stream whose operation time is less than the target water level when a match is determined.

[0099] In this embodiment of the application, if a match is found, the complex event detection device writes the operation event into the status list; if a mismatch is found, the complex event detection device deletes the operation event.

[0100] In this embodiment of the application, the process of the complex event detection device filtering target events whose operation time is less than the target water level from the operation event stream can be to obtain the first operation event in the status list for the complex event detection device, and then filter the target events whose operation time is less than the target water level from the first operation event.

[0101] It should be noted that the first operation event is all operation events added to the status list.

[0102] In this embodiment of the application, taking DCP as an example, it is determined whether the rule start condition, rule intermediate condition, and rule end condition are satisfied. For the begin and end events in the following "event flow of binding rules and calculation logic", the Aviator expression engine will determine that the result of RULE_BEGIN and RULE_END is true and write these two events into the status.

[0103] Begin event:

[0104]

[0105]

[0106] end event:

[0107]

[0108]

[0109] In this embodiment of the application, before the complex event detection device filters the target events whose operation time is less than the target water level from the operation event stream, the complex event detection device also obtains multiple event operation times corresponding to the operation event stream and generates multiple water level lines based on the multiple event operation times; the complex event detection device merges the multiple water level lines to obtain merged water level lines; the complex event detection device filters the target water level line with the smallest water level line value from the merged water level line and the preset water level line.

[0110] In this embodiment, Flink extracts the event operation time field from the operation event stream to generate the corresponding watermark.

[0111] In this embodiment, the complex event detection device generates multiple water level lines based on the operation time of multiple events. This can be achieved by using a preset time for tolerating disorder as the water level line generation strategy, or by using the operation time of multiple events as the multiple water level lines. The specific method by which the complex event detection device generates multiple water level lines based on the operation time of multiple events can be determined according to the actual situation, and this embodiment does not limit this method.

[0112] In this embodiment, the preset duration can be the duration configured in the complex event detection device; the preset duration can also be the duration transmitted from other devices to the complex event detection device; the specific way in which the complex event detection device obtains the preset duration can be determined according to the actual situation, and this embodiment does not limit it.

[0113] It should be noted that the preset duration can be 5 seconds, 3 seconds, or 1 second. The specific preset duration can be determined according to the actual situation, and this application embodiment does not limit it.

[0114] In this embodiment, taking DCP as an example, Flink extracts the timestamp field from the real-time event stream as the input parameter of the watermark generator, and uses a 5-second tolerance for out-of-order events as the watermark generation strategy to generate the event stream watermark.

[0115] The water level line shape is as follows:

[0116] Watermark: 1646877595000

[0117] The corresponding event data format is as follows:

[0118]

[0119] In this embodiment, the preset water level can be a water level configured in the complex event detection device; the preset water level can also be a water level transmitted to the complex event detection device by other devices; the specific way the complex event detection device obtains the preset water level can be determined according to the actual situation, and this embodiment does not limit it.

[0120] It should be noted that the preset water level can be the maximum value of the Long type, or it can be other values; the specific preset water level can be determined according to the actual situation, and this application embodiment does not limit it in this way.

[0121] In this embodiment of the application, taking DCP as an example, 2 to the power of 63 - 1 is used as the input parameter of the water level generator to generate a preset water level. The preset water level can be: watermark:0x7fffffffffffffffL.

[0122] In this embodiment of the application, taking DCP as an example, the operation event stream and multiple event rules and multiple event determination logic are connected by the connect operator to generate an operation event stream that binds multiple event rules and multiple event determination logic. The user id field in the operation event stream is extracted and the CRC32 check value is calculated. Then, the operator parallelism is moduloed to generate hashId. The smaller value between the water level of the event stream (item operation time) and the water level of the rule stream (preset water level) is taken as the target water level.

[0123] In this embodiment of the application, the process of merging multiple water level lines by the complex event detection device to obtain a merged water level line includes: the complex event detection device obtaining the first water level line with the largest water level line value from the multiple water level lines; and the complex event detection device using the first water level line as the merged water level line.

[0124] S105. Determine the determination result of the target event based on the determination logic of multiple events, and define the target event based on the determination result as a complex event.

[0125] In this embodiment of the application, after the complex event detection device filters target events whose operation time is less than the target water level from the operation event stream, the complex event detection device can determine the determination result of the target event according to multiple event determination logic, and determine the target event as a complex event according to the determination result.

[0126] In this embodiment of the application, the process by which the complex event detection device determines the determination result of the target event based on multiple event determination logics can be that the complex event detection device calls the Aviator expression engine to determine the determination result of the target event based on multiple event determination logics.

[0127] In this embodiment of the application, the complex event detection device reads all the state data of the current hashId (corresponding to the same hashId) in stateFunc. The operation event corresponding to the state data is the first operation event. The complex event detection device filters the target events whose operation time is less than the target water level from the first operation events, performs complex event detection on the target event, and calls the Aviator expression engine to parse and execute the calculation logic to generate the determination result of the target event.

[0128] In this embodiment, taking DCP as an example, all state data of the current hashId=91 group is read, and then target events with operation times less than the target watermark are filtered from the first operation events corresponding to the state data. Events with the same user_id and the same RULE_ID are grouped together for calculation. The event rules are determined according to the complex event category, complex event subclass, and event determination logic. For example, CEP_TYPE=deltaTime and CEP_SUBTYPE=1 and CEP_CALC_EXPRESS=END.timestamp-BEGIN.timestamp represent the time difference between two non-strictly consecutive events. The event determination logic after complex event detection is equal to the event time of the end event minus the event time of the start event. The Aviator expression engine is called to parse and execute the event determination logic, i.e., calc_grant_delta_time=1646877700000-1646877600000=1000000 milliseconds.

[0129] In this embodiment, the complex event detection device clears the states of operation events corresponding to already calculated complex events, as well as the states of operation events that do not match event rules, from the stateFunc. Taking DCP as an example, this means deleting the states of operation events corresponding to already calculated complex events, as well as the states of operation events that do not match event rules.

[0130] In this application embodiment, an exemplary complex event detection method is as follows: Figure 3 As shown:

[0131] S30. The complex event detection device receives multiple configured event rules and multiple event determination logics, and broadcasts the multiple event rules and multiple event determination logics to the computing engine.

[0132] S31. The complex event detection device acquires the operation time of multiple events corresponding to the operation event stream, and generates multiple water level lines based on the multiple event operation times.

[0133] S32. The complex event detection device merges multiple water level lines to obtain a merged water level line.

[0134] S33. The complex event detection device selects the target water level line with the smallest water level line value from the merged water level line and the preset water level line.

[0135] S34. The complex event detection device acquires the operation event stream and binds multiple event rules and multiple event determination logic to each event in the operation event stream.

[0136] It should be noted that there are multiple events in the operation event stream.

[0137] S35. The complex event detection device obtains preset fields in the operation event stream to get multiple field information; and verifies the multiple field information to get multiple verification values.

[0138] S36. The complex event detection device performs hash processing on multiple verification values ​​to obtain multiple hash information; and groups the operation event stream according to the multiple hash information to obtain multiple groups of operation events.

[0139] S37. The complex event detection device determines the matching between each set of operation events and the multiple event rules bound to it, until the matching between the multiple sets of operation events and the multiple event rules bound to it is determined.

[0140] S38. In the case of matching, the complex event detection device filters target events from the operation event stream whose operation time is less than the target water level.

[0141] It should be noted that the target water level is the time determined based on the operation times of multiple events corresponding to the operation event flow.

[0142] S39. The complex event detection device determines the determination result of the target event based on multiple event determination logic, and determines the target event as a complex event based on the determination result.

[0143] For example, such as Figure 4As shown, taking DCP as an example, the complex event detection device detects the operation event "DCP user authorization behavior time consumption" as follows: The event tracking system reports operation events to the Kafka message queue in real time. Flink consumes the real-time operation events in Kafka in real time to form a real-time operation event stream. Flink extracts the timestamp field from the real-time event stream as the input parameter of the watermark generator, using a 5-second tolerance for out-of-order behavior as the watermark generation strategy to generate the watermark corresponding to the event stream. Multiple event rules and multiple event determination logics are defined using Aviator expressions. Flink reads the Aviator expressions stored in the database every n seconds to form "multiple event rules and multiple event determination logics". 2^63 - 1 is used as the input parameter of the watermark generator to generate the preset watermark. A broadcast operator is used to broadcast "multiple event rules and multiple event determination logics (rules and calculation logic streams)" to form a broadcast stream. The CRC32 checksum of the user_id field is extracted and moduloed with the operator's parallelism of 100 to obtain hashId = 91. The `connect` operator connects (binds) the operation event stream and multiple event rules and multiple event determination logics, generating an operation event stream that binds multiple event rules and multiple event determination logics (binding the operation event stream with the rule and calculation logic stream). The user ID field in the operation event stream is extracted and the CRC32 check value is calculated. Then, a modulo operation is performed on the operator parallelism to generate a hashId. The smaller value between the waterline of the event stream (item operation time) and the waterline of the rule stream (preset waterline) is taken as the target waterline. The `keyBy` operator groups operation event streams bound to multiple event rules based on their hashId, generating multiple groups of operation events grouped by key. For example, operation events for user_id=xxxxxxxxxxxxxxxxx00 and user_id=simulator_dcp_delta4_sit631 share the same hashId, meaning these two users' operation events will be grouped together. Even if user_id=xxxxxxxxxxxxxxxxx001 has no subsequent operation events, as long as other users with hashId=91 have subsequent operation events, the watermark of the current group will be raised, triggering the calculation of the operation event data for user_id=xxxxxxxxxxxxxxxxx001. Each operation event in the multiple groups of operation events bound to multiple event rules and event determination logic iterates through its own bound event rules and event determination logic.Taking DCP as an example, it determines whether the rule start condition, rule intermediate condition, and rule end condition are met. For the begin and end events in the following "event flow of binding rules and calculation logic", the Aviator expression engine will determine that the result of RULE_BEGIN and RULE_END is true and write these two events into the state. It reads all the state data of the group with current hashId=91, then filters the target events whose operation time is less than the target watermark from the first operation event corresponding to this state data. Events with the same user_id and the same RULE_ID are then grouped together for calculation. The event rules are determined based on the complex event category, complex event subclass, and event determination logic. For example, CEP_TYPE=deltaTime and CEP_SUBTYPE=1 and CEP_CALC_EXPRESS=END.timestamp-BEGIN.timestamp represent the time difference between two non-strictly consecutive events. The event determination logic after complex event detection is: event time of the end event - event time of the start event. The Aviator expression engine is then called to parse and execute the event determination logic (to obtain the determination result of the target event). Delete the state of operation events corresponding to complex events that have already been calculated, as well as the state of operation events that have not matched the event rules.

[0144] Understandably, since the complex event detection device uses multiple event rules and multiple event determination logics in the form of expressions, when adding new complex event detection rules and event determination logic, it can directly broadcast the new complex event detection rules and event determination logic to the calculation engine. The calculation engine can then parse and calculate the new rules in the form of expressions, without needing to add new rule calculation unit code or compile and run the code, thus improving the speed of adding complex event detection rules.

[0145] Based on the same inventive concept as the above-mentioned complex event detection method, this application provides a complex event detection device 1, corresponding to a complex event detection method; Figure 5 A schematic diagram of the composition structure of a complex event detection device provided in this application embodiment. Figure 1 The complex event detection device 1 may include:

[0146] Acquisition unit 11 is used to acquire an operation event stream, wherein the number of events in the operation event stream is multiple;

[0147] Binding unit 12 is used to bind multiple event rules and multiple event determination logic to each event in the operation event stream;

[0148] The determining unit 13 is used to use a computing engine to determine the matching between the operation event stream and the multiple bound event rules; and to determine the determination result of the target event according to the multiple event determination logic, so as to determine the target event as a complex event according to the determination result.

[0149] The filtering unit 14 is used to filter target events whose operation time is less than a target water level from the operation event stream when a match is found; the target water level is a time determined based on the operation times of multiple events corresponding to the operation event stream.

[0150] In some embodiments of this application, the apparatus further includes a receiving unit and a broadcasting unit;

[0151] The receiving unit is used to receive multiple configured event rules and multiple event determination logic;

[0152] The broadcast unit is used to broadcast the multiple event rules and multiple event determination logic to the computing engine.

[0153] In some embodiments of this application, the apparatus further includes a grouping unit;

[0154] The acquisition unit 11 is used to acquire preset fields from the operation event stream to obtain multiple field information;

[0155] The grouping unit is used to group the operation event stream according to the multiple field information to obtain multiple groups of operation events;

[0156] The determining unit 13 is used to determine the matching between each group of operation events in the multiple groups of operation events and the multiple bound event rules, until the matching between the multiple groups of operation events and the multiple bound event rules is determined.

[0157] In some embodiments of this application, the apparatus further includes a verification unit and a processing unit;

[0158] The verification unit is used to verify the multiple field information respectively to obtain multiple verification values;

[0159] The processing unit is used to perform hash processing on the multiple verification values ​​to obtain multiple hash information;

[0160] The grouping unit is used to group the operation event stream according to the multiple hash information to obtain the multiple groups of operation events.

[0161] In some embodiments of this application, the acquisition unit 11 is used to acquire multiple event operation times corresponding to the operation event stream, and generate multiple water level lines based on the multiple event operation times;

[0162] The processing unit is used to merge the multiple water level lines to obtain a merged water level line;

[0163] The filtering unit 14 is used to filter the target water level line with the smallest water level line value from the merged water level line and the preset water level line.

[0164] In some embodiments of this application, the acquisition unit 11 is used to acquire the first water level line with the largest water level value from the plurality of water level lines; and to use the first water level line as the merged water level line.

[0165] It should be noted that, in practical applications, the aforementioned acquisition unit 11, binding unit 12, determination unit 13, and filtering unit 14 can be implemented by the processor 15 on the complex event detection device 1, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor), or Field Programmable Gate Array (FPGA), etc.; the aforementioned data storage can be implemented by the memory 16 on the complex event detection device 1.

[0166] This application also provides a complex event detection device 1, such as... Figure 6 As shown, the complex event detection device 1 includes a processor 15, a memory 16, and a communication bus 17. The memory 16 communicates with the processor 15 through the communication bus 17. The memory 16 stores programs executable by the processor 15. When the program is executed, the processor 15 executes the complex event detection method as described above.

[0167] In practical applications, the aforementioned memory 16 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 15.

[0168] This application provides a computer-readable storage medium having a computer program thereon, which, when executed by a processor 15, implements the complex event detection method as described above.

[0169] Understandably, since the complex event detection device uses multiple event rules and multiple event determination logics in the form of expressions, when adding new complex event detection rules and event determination logic, it can directly broadcast the new complex event detection rules and event determination logic to the calculation engine. The calculation engine can then parse and calculate the new rules in the form of expressions, without needing to add new rule calculation unit code or compile and run the code, thus improving the speed of adding complex event detection rules.

[0170] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0171] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method for detecting complex events, characterized in that, The method includes: Obtain an operation event stream, wherein the number of events in the operation event stream is multiple; Receive multiple configured event rules and multiple event determination logics; broadcast the multiple event rules and multiple event determination logics to the computing engine; bind multiple event rules and multiple event determination logics to each event in the operation event stream; The matching between the operation event stream and the bound event rules is determined using a computing engine. Specifically, this includes: obtaining preset fields from the operation event stream to obtain multiple field information; grouping the operation event stream according to the multiple field information to obtain multiple groups of operation events, specifically including: obtaining preset fields from the operation event stream to obtain multiple field information, where the multiple field information is user ID or device ID; validating each of the multiple field information to obtain multiple verification values; hashing the multiple verification values ​​to obtain multiple hash information; grouping the operation event stream according to the multiple hash information to obtain multiple groups of operation events; and determining the matching between each group of operation events and the bound event rules, until the matching between the multiple groups of operation events and the bound event rules is determined. In the case of matching, multiple event operation times corresponding to the operation event stream are obtained, and multiple water level lines are generated based on the multiple event operation times; the multiple water level lines are merged to obtain a merged water level line; from the merged water level line and the preset water level line, the target water level line with the smallest water level line value is selected; target events with event operation times less than the target water level line are selected from the operation event stream; the target water level line is the time determined based on the multiple event operation times corresponding to the operation event stream. The determination result of the target event is determined based on the multiple event determination logic, and the target event is determined to be a complex event based on the determination result.

2. The method according to claim 1, characterized in that, The process of merging the multiple water level lines to obtain merged water level lines includes: From the plurality of water level lines, select the first water level line with the largest water level value; The first water level line is used as the merged water level line.

3. A complex event detection device, characterized in that, The device includes: An acquisition unit is used to acquire an operation event stream, wherein the operation event stream contains multiple events; The receiving unit is used to receive multiple configured event rules and multiple event determination logic; The broadcast unit is used to broadcast the multiple event rules and multiple event determination logic to the computing engine; A binding unit is used to bind multiple event rules and multiple event determination logic to each event in the operation event stream; The determining unit, used to determine the matching between the operation event stream and the bound multiple event rules using a computing engine, specifically includes: obtaining preset fields in the operation event stream to obtain multiple field information; grouping the operation event stream according to the multiple field information to obtain multiple groups of operation events, specifically including: obtaining preset fields in the operation event stream to obtain multiple field information, wherein the multiple field information is user ID or device ID; verifying the multiple field information to obtain multiple verification values; hashing the multiple verification values ​​to obtain multiple hash information; grouping the operation event stream according to the multiple hash information to obtain multiple groups of operation events; determining the matching between each group of operation events in the multiple groups of operation events and the bound multiple event rules, until the matching between the multiple groups of operation events and the bound multiple event rules is determined; determining the determination result of the target event according to the multiple event determination logic, so as to determine the target event as a complex event according to the determination result; A filtering unit is configured to, in the case of matching, acquire multiple event operation times corresponding to the operation event stream, and generate multiple water level lines based on the multiple event operation times; merge the multiple water level lines to obtain a merged water level line; filter the target water level line with the smallest water level line value from the merged water level line and the preset water level line; and filter the target events whose event operation times are less than the target water level line from the operation event stream; wherein the target water level line is the time determined based on the multiple event operation times corresponding to the operation event stream.

4. A complex event detection device, characterized in that, The device includes: The system includes a memory, a processor, and a communication bus, wherein the memory communicates with the processor via the communication bus, and the memory stores a complex event detection program executable by the processor, wherein when the complex event detection program is executed, the processor performs the method as described in any one of claims 1 to 2.

5. A storage medium storing a computer program thereon, used in a complex event detection device, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1 to 2.

Images