Real-time rule engine control method and system based on Flink and medium
A control method and rule technology, applied in the field of big data-real-time computing, can solve problems such as poor stability and affecting the performance of frequent query of business data, and achieve the effect of improving stability, timeliness and accuracy
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0043] Such as figure 1 , figure 2 As shown, the present invention is based on Flink's real-time rule engine control method, such as figure 1 As shown, the method includes:
[0044] S1, receiving an event source in real time and configuring a real-time rule for the event source;
[0045] S2. After configuring the real-time rules, use the Flink streaming real-time computing engine to store the event source internally, and store the event stream and rule stream data in the form of key-value;
[0046] S3 uses the Flink streaming real-time computing engine to connect event streams and rule streams for dynamic grouping; for each event stream: according to the received rule stream and the fields that need to be grouped in the rule stream, use the Java reflection mechanism to obtain a specific value , to form a new event and send it to the downstream operator for processing; the downstream receives the new event, and performs dynamic keyBy grouping according to this value. After ...
example 1
[0064]
[0065] Table 1 is converted into rule flow input: 1,Active,Name,Amount,Sum,>,5000,20, translated into Chinese means: group by name, accumulate by amount field, and send an alert if the amount exceeds 5,000 yuan within 20 minutes.
[0066] (5) The other supported input format of rule flow is Json format: {"rule ID":"1","rule status":"Active","grouping field":"Name","calculation field":"Amount" ,"symbol":">","threshold":"5000","window size":"20"}
[0067] Specifically, when the rules are read, the fields that need to be grouped are parsed out, and each rule flow will be applied to each event flow data. At the same time, the time span that needs to be aggregated is defined in the rule flow, such as salary requirements within 20 minutes and. Because the rule flow may be added in real time, the whole process achieves real-time aggregation calculation.
[0068] For example: 2021-11-30 10:00:00 received event data, {"id":"1","amount":"200"}
[0069] 2021-11-30 10:01:00...
Embodiment 2
[0075] Such as figure 1 , figure 2 As shown, the difference between this embodiment and Embodiment 1 is that the method further includes: S5, taking a snapshot of the event flow and rule flow data status in real time, and automatically recovering from the specified snapshot if the task is abnormal.
[0076] Specifically, after the above-mentioned steps S1 to S4 are established, start the program automatic save point, and take a snapshot of the data status at regular intervals to prevent data loss when the task is abnormal. After taking a snapshot, if the task is abnormal, it can automatically restore from the specified snapshot.
[0077] The system of the present invention has an automatic save point function, and the snapshot technology supports restoring data from a specified snapshot. If the system is abnormal, it can recover from the last failed location, and no business data that should be alarmed is missed.
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More