An event rule matching method based on a dual unbiased function

Abstract

The present invention relates to the technical field of data processing, and discloses an event rule matching strategy based on dual unbiased functions. A large number of rules are preprocessed, and two different unbiased functions are respectively used to calculate their Hash values. The corresponding primary Hash value and secondary Hash value are respectively stored in the corresponding sequential lists. The above two different unbiased functions are used to calculate the Hash value of the event to be matched, and this Hash value is compared with the above Hash values. The rules are filtered three times. The first and second filtrations utilize two Hash tables and two unbiased Hash functions. This event rule matching strategy based on dual unbiased functions can reduce the potentially matching rules to a small number when facing a large number of data processing rules, which not only ensures the stability of the service but also speeds up the data processing speed, can better help the system meet the real-time requirements, and improves the user experience while ensuring the service quality.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an event rule matching method based on a dual unbiased function. Background Art

[0002] With the development of Internet finance, the risks faced by online financial transactions have become diversified and intelligent, requiring Internet finance companies to be able to quickly identify risks, improve risk management capabilities, and make corresponding early warnings and adjustments in a timely manner.

[0003] With the complication of financial operations, events in streaming data processing are also becoming diverse, and the corresponding early warning event matching rules are also becoming more numerous. When the number of matching rules is too large, in the process of processing streaming data, it may take a lot of time to judge and find the corresponding matching rules when an event arrives, and it is impossible to achieve real-time early warning well.

[0004] In existing data processing systems, there are several common event matching rule strategies:

[0005] Sequential Matching, the simplest rule matching strategy, sequentially matches each rule in the order of the rule set. When a data event matches a certain rule successfully, the matching stops and the corresponding operation is executed. This strategy is applicable to scenarios where there is no interdependence between rules.

[0006] Parallel Matching, a strategy that simultaneously matches multiple rules. Data events are concurrently matched with multiple rules, and the matching efficiency can be improved through parallel computing or distributed processing. Parallel matching is applicable to scenarios where rules are independent of each other and can be processed simultaneously.

[0007] Priority-based Matching, matches according to the priority order of rules. Each rule has a priority, and higher-priority rules are matched before lower-priority rules. When a data event matches multiple rules, only the matching rule with the highest priority is executed. This strategy is applicable to scenarios where there is a priority relationship between rules.

[0008] The current event rule matching strategies cannot well adapt to business scenarios with overly numerous rules. However, in today's situation, as the functions of Internet finance products become increasingly complete and complex, the possible risk events are increasing, and thus the corresponding risk event matching rules are also increasing. When an event occurs, in the face of too many risk event matching rules, the streaming data processing engine will consume more time for rule matching, thus consuming a large amount of time and it is difficult to meet the real-time requirements of the system. Summary of the Invention

[0009] The present invention provides an event rule matching method based on a dual unbiased function, which solves the problem that the current event rule matching strategy in the above-mentioned background technology cannot well adapt to business scenarios with too many rules. When an event occurs, in the face of too many risk event matching rules, the streaming data processing engine will consume a lot of time for rule matching, thus consuming a large amount of time and it is difficult to meet the real-time requirements of the system again.

[0010] The present invention provides the following technical solution: an event rule matching method based on a dual unbiased function, which preprocesses a large number of set rules, applies two different unbiased functions to calculate their Hash values (main Hash value and secondary Hash value) respectively, and stores the corresponding main Hash value and secondary Hash value in the corresponding sequential lists (divided into a main Hash table and a secondary Hash table) respectively, facilitating subsequent sequential search to accelerate the subsequent filtering and matching of events to rules.

[0011] An event rule matching method based on a dual unbiased function includes a rule processing process and an event matching process. The specific steps of the rule processing process are as follows:

[0012] S1. Receive n rules for event processing;

[0013] S2. Initialize two sequential lists with a length of kn and denote them as the main Hash table and the secondary Hash table, which are used to store the linked lists constructed by rule elements with the same Hash value;

[0014] S3. Extract the keywords or attribute names included in the events corresponding to each rule, and encode all the extracted keywords using a fixed encoding method to generate corresponding keyword encoding values;

[0015] S4. Calculate the main Hash value corresponding to the i-th rule using the main Hash formula, and store the relevant information of this rule in the position with the serial number hash in the corresponding main Hash table; calculate the secondary Hash value corresponding to the i-th rule using the secondary Hash formula, and store the relevant information of this rule in the position with the serial number hash in the corresponding secondary Hash table; until all n rules received in S1 are processed; 主-i The position; calculate the secondary Hash value corresponding to the i-th rule using the secondary Hash formula, and store the relevant information of this rule in the position with the serial number hash in the corresponding secondary Hash table; until all n rules received in S1 are processed; 次-i The position; until all n rules received in S1 are processed;

[0016] The steps of the event matching process are as follows:

[0017] S01. Receive an event, extract the keywords in the event, and encode the extracted keywords using a fixed encoding method to generate corresponding encoding values (x1, x2... x m )

[0018] S02. Calculate the main Hash value hash corresponding to the event according to the main Hash formula 主 ;

[0019] S03. Check whether the corresponding position in the main Hash table stores the relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step;

[0020] S04. Calculate the secondary Hash value hash corresponding to the event according to the secondary Hash formula 次 ;

[0021] S05. Check whether the corresponding position in the secondary Hash table stores the relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step;

[0022] S06. Compare the event rules stored in the main Hash table with the serial number Hash 主 and the event rules stored in the secondary Hash table with the serial number Hash 次 to check whether they store the same event rules. If not, it can be determined that there is no matching rule corresponding to the currently occurring event. If there is, define the same event rules as the possible matching rules for the currently occurring event and proceed to the next step;

[0023] S07. Process the currently occurring event using the possible matching rules;

[0024] Among them: The keyword encoding values in rule processing or event matching are: x1, x2…x m ;

[0025] The calculation formula for the main Hash value is:

[0026] ;

[0027] The calculation formula for the secondary Hash value is:

[0028] .

[0029] Preferably, in the rule processing, each of the n received matching rules is calculated and stored one by one; and each rule corresponds to the occurring event.

[0030] Preferably, the method for extracting event keywords in the event matching process is the same as the method for extracting keywords in the rule processing process.

[0031] Preferably, in the main Hash value calculation formula and the secondary Hash value calculation formula, k is a constant. The larger k is, the better the hashing property, but the more storage space is required. Conversely, the required space is smaller, but the number of conflicts increases.

[0032] Preferably, in the rule processing flow, the main Hash is used to calculate the main Hash value hash of the i-th rule value ranging from 0 to kn - 1 主-i ; the secondary Hash is used to calculate the secondary Hash value hash of the i-th rule value ranging from 0 to kn - 1 次-i .

[0033] Preferably, the rule processing flow preprocesses a large number of set rules; the event matching flow filters the rules three times, reducing the possibly matching rules to a small number of several.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The event rule matching method based on the dual unbiased function realizes three - stage filtering of the rules. The first and second filtering utilize two Hash tables and two unbiased Hash functions, while the third filtering utilizes the comparison results of the two Hash tables. In this way, when dealing with a large number of data processing rules, the possibly matching rules can be reduced to a small number of several. This not only ensures the stability of the service but also speeds up the data processing speed.

[0036] 2. The event rule matching method based on the dual unbiased function can better help the system meet the real - time requirements in the scenario of Internet financial risk prevention and control, ensuring the service quality while improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of the rule processing flow of the present invention;

[0038] Figure 2 is a schematic diagram of the event matching flow of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] This application proposes an embodiment: an event rule matching method based on dual unbiased functions, which preprocesses a large number of set rules, applies two different unbiased functions to calculate their Hash values (primary Hash value and secondary Hash value) respectively, and stores the corresponding primary Hash value and secondary Hash value in the corresponding sequential lists (divided into a primary Hash table and a secondary Hash table) respectively, facilitating subsequent sequential search to accelerate the subsequent filtering and matching of events to rules; in the subsequent event rule matching, the rules are filtered three times. The first and second filters utilize the two Hash tables and two unbiased Hash functions, while the third filter utilizes the comparison results of the two Hash tables.

[0041] The above settings can reduce the potentially matching rules to a small number of several when dealing with a large number of data processing rules during the event rule matching process, ensuring both the stability of the service and accelerating the data processing speed. In the scenario of Internet financial risk prevention and control, this method can better help the system meet the real-time requirements, ensuring the service quality while improving the user experience.

[0042] The above event rule matching method based on dual unbiased functions can be roughly divided into two processes when specifically implemented, namely the rule processing process and the event matching process. The rule processing process is used to preprocess a large number of rules, and the event matching process is used to filter the rules three times during the event matching process, reducing the potentially matching rules to a small number of several.

[0043] The specific steps of the rule processing process are as follows:

[0044] S1. Receive n rules for event processing;

[0045] S2. Initialize two sequential lists with a length of kn and denote them as the primary Hash table and the secondary Hash table, which are used to store the linked lists constructed by rule elements with the same Hash value (where k is a constant. The larger k is, the better the hashability, but the more storage space is required. Conversely, the required space is smaller, but the conflicts increase);

[0046] S3. Each rule corresponds to an event that occurs. Extract the keywords or attribute names contained in the event that occurs corresponding to each rule, and encode all the extracted keywords using a fixed encoding method to generate corresponding keyword encoding values (x1, x2…x m ) The encoding methods are common encoding methods in the prior art such as GB2312, GBK, Big5, etc.;

[0047] S4. Calculate and store each of the n matching rules one by one using the following method until all n rules are processed: Calculate the main Hash value corresponding to the i-th rule using the main Hash formula. According to this unbiased function formula, the main Hash value hash of the i-th rule value in the range of 0 to kn - 1 can be obtained. 主-i Store the relevant information of this rule in the main Hash table at the position with the serial number hash. 主-i Calculate the secondary Hash value corresponding to the i-th rule using the secondary Hash formula. According to this unbiased function formula, the secondary Hash value hash of the i-th rule value in the range of 0 to kn - 1 can be obtained. 次-i Store the relevant information of this rule in the secondary Hash table at the position with the serial number hash. 次-i The position.

[0048] The event matching process steps are as follows:

[0049] S01. Receive an event, extract the keywords in the event, and the method for extracting event keywords in this step is the same as the keyword extraction method in the above rule processing process. Also, encode the extracted keywords using a fixed encoding method to generate corresponding encoded numerical values (x1, x2... x m );

[0050] S02. Calculate the main Hash value hash corresponding to the event according to the main Hash formula. 主 ;

[0051] S03. Check whether the corresponding position in the main Hash table stores the relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step;

[0052] S04. Calculate the secondary Hash value hash corresponding to the event according to the secondary Hash formula. 次 ;

[0053] S05. Check whether the corresponding position in the secondary Hash table stores the relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step;

[0054] S06. Compare the event rules stored in the main Hash table with the serial number Hash 主 and the event rules stored in the secondary Hash table with the serial number Hash 次 to see if they store the same event rules. If not, it can be determined that there is no matching rule corresponding to the currently occurring event. If there is, define the same event rules as the possible matching rules for the currently occurring event and proceed to the next step;

[0055] S07. Process the currently occurring event using the available matching rules.

[0056] The main Hash value calculation formula in the rule processing flow and the main Hash value calculation formula in the event matching flow are both:

[0057] ;

[0058] The secondary Hash value calculation formula in the rule processing flow and the secondary Hash value calculation formula in the event matching flow are both:

[0059] .

[0060] From the above description, it can be seen that the event rule matching method based on dual unbiased functions proposed in this application pre - processes a large number of set rules in advance, converts the events represented by the rules and the events to be matched into Hash values using the same two different unbiased functions, improves the matching rate between events and rules, and filters the rules three times during the event rule matching process, reducing the possibly matching rules to a small number of several. This not only ensures the stability of the service but also speeds up the data processing speed.

[0061] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An event rule matching method based on a dual unbiased function, including a rule processing flow and an event matching flow, characterized in that: The specific steps of the above-mentioned rule processing flow are as follows: S1. Receive n rules for event processing; S2. Initialize two sequential lists with a length of kn, denoted as the main Hash table and the secondary Hash table, for storing linked lists constructed by rule elements with the same Hash value; S3. Extract the keywords or attribute names included in the events corresponding to each rule, and encode all the extracted keywords using a fixed encoding method to generate corresponding keyword encoding values; S4. Calculate the main Hash value corresponding to the i-th rule using the main Hash formula, and store the relevant information of this rule at the position with the serial number hash in the corresponding main Hash table; calculate the secondary Hash value corresponding to the i-th rule using the secondary Hash formula, and store the relevant information of this rule at the position with the serial number hash in the corresponding secondary Hash table; until all n rules received in S1 are completely processed; 主-i The position; until all n rules received in S1 are completely processed; 次-i The position; until all n rules received in S1 are completely processed; The steps of the above-mentioned event matching flow are as follows: S01. Receive an event, extract keywords in the event, and encode the extracted keywords using a fixed encoding method to generate corresponding encoded values (x1, x2... x m ); S02. Calculate the main Hash value hash corresponding to the event according to the main Hash formula 主 ; S03. Check whether the corresponding position in the main Hash table stores relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step; S04. Calculate the secondary Hash value hash corresponding to the event according to the secondary Hash formula 次 ; S05. Check whether the corresponding position in the secondary Hash table stores relevant information of the rule. If not, it can be determined that there is no matching rule corresponding to the event. If it is not empty, proceed to the next step; S06. Compare the event rules stored in the main Hash table with the serial number Hash 主 and the event rules stored in the secondary Hash table with the serial number Hash 次 to check if they store the same event rules. If not, it can be determined that there is no matching rule corresponding to the currently occurring event. If there is, define the same event rules as the possible matching rules for the currently occurring event and proceed to the next step; S07. Process the currently occurring event using the possible matching rules; Among them: The keyword encoding values in rule processing or event matching are: x1, x2…x m ; The calculation formula for the main Hash value is: ； The calculation formula for the secondary Hash value is: 。 2. The event rule matching method based on a dual unbiased function according to claim 1, characterized in that: In the above-mentioned rule processing, each of the n received matching rules is calculated and stored one by one; and each rule corresponds to an event that occurs.

3. The event rule matching method based on a dual unbiased function according to claim 1, characterized in that: In the above-mentioned event matching flow, the method for extracting event keywords is the same as the method for extracting keywords in the rule processing flow.

4. The event rule matching method based on a dual unbiased function according to claim 1, characterized in that: In the above-mentioned calculation formulas for the main Hash value and the secondary Hash value, k is a constant. The larger k is, the better the hashability, but the more storage space is required. Conversely, the required space is smaller, but the number of conflicts increases.

5. The event rule matching method based on a dual unbiased function according to claim 4, wherein: In the above-mentioned rule processing process, the main Hash value hash of the i-th rule value ranging from 0 to kn-1 is obtained by using the main Hash 主-i ; the secondary Hash value hash of the i-th rule value ranging from 0 to kn-1 is obtained by using the secondary Hash 次-i .

Images