Automatic data accounting method and system based on rule engine

Abstract

The application relates to the technical field of data accounting, in particular to an automatic data accounting method and system based on a rule engine, which comprises the following steps: obtaining a historical database based on a merchant, constructing a rule sample database for the rule engine through the historical database, and establishing rules and constructing a rule library according to the requirements of the merchant; evaluating the similarity of two rules in the rule library, and dividing the rule library into multiple rule sets; constructing an accounting database through the historical database and the rule sample database, obtaining a new rule through the accounting database, analyzing the new rule to determine a rule set, judging the data delay state of the new rule entering the rule engine and joining the rule library according to the rule set, and setting a data delay reaction window; evaluating the data delay performance degree based on the data delay reaction window, adjusting and iterating the rule set through the data delay performance degree until the data accounting is completed. The distributed accounting process of data is realized, and the automatic efficiency of the final data accounting is improved.

Description

Technical Field

[0001] This invention relates to the field of data accounting technology, specifically to an automated data accounting method and system based on a rule engine. Background Technology

[0002] Faced with the complexity, variability, and scale of modern business demands, and the sheer volume of data requiring accounting, traditional verification methods typically involve directly embedding data into code, resulting in extremely low data processing efficiency. Furthermore, the variety and volume of data in existing merchant data accounting platforms make efficient automated accounting difficult through traditional methods such as manual verification and hard coding. To effectively improve the automation efficiency of data accounting, existing technologies often incorporate rule engines. These rule engines are components embedded in the application, separating business decisions from the application code. They use predefined semantic modules to write business decisions, allowing the data accounting platform to operate without manual settings during accounting, achieving better automation results.

[0003] A rule engine is an integrated component whose rule logic is based on relevant pattern matching algorithms. Among them, the RETE algorithm is the most common. It is a forward rule fast matching algorithm that creates multiple rules by building a rule base, constructs nodes and allocates storage areas for each rule, and allows different rules with the same pattern to share nodes and storage areas without repeating the judgments that have been made. It is a typical space-for-time approach, which allows only the changed facts to be calculated when the factual part changes, thus improving matching efficiency and improving the efficiency of data accounting.

[0004] However, in real-world scenarios, the amount of data on existing data accounting platforms is extremely large. If the data is still centrally input into the rule engine for matching in the traditional way, the RETE algorithm will be unable to process the massive amount of data in a short time. It will spend a lot of time searching and matching each rule in turn, resulting in low automation efficiency of the final data accounting. Summary of the Invention

[0005] To address the technical problem of low automation efficiency in existing data accounting methods when dealing with massive amounts of data, the present invention aims to provide an automated data accounting method based on a rule engine. The specific technical solution adopted is as follows:

[0006] Based on the historical database obtained from merchants, a rule sample database is built from the historical database as the rule engine, and rules are established according to the needs of merchants to build a rule library;

[0007] Evaluate the similarity between pairs of rules in the rule base and divide the rule base into multiple rule sets;

[0008] An accounting database is constructed using historical databases and rule sample databases. New rules are obtained from the accounting database. The new rules are analyzed to determine the rule set to which they belong. Based on the rule set to which they belong, the data delay status of the new rules entering the rule engine and joining the rule base is determined, and a data delay response window is set.

[0009] The data latency performance is evaluated based on the data latency response window. The rule set is then adjusted and iterated based on the data latency performance until the data accounting is completed.

[0010] Preferably, the rules include multiple types of nodes, alpha nodes, and beta nodes, wherein the alpha nodes construct the alpha network to form rules, and the beta nodes construct the beta network to form rules.

[0011] Preferably, the similarity between pairs of rules in the rule base is evaluated, and the rule base is divided into multiple rule sets, including:

[0012] Analyze the type nodes and alpha nodes of rules in the rule base to determine the similarity between pairs of rules;

[0013] The similarity is defined as a distance metric, and clustering is performed using the distance metric to divide the data into multiple clusters, with each cluster corresponding to a set of rules.

[0014] Preferably, the similarity between rule type nodes and alpha nodes in the rule base is determined by analyzing the similarity between pairs of rules, including:

[0015] The number of type nodes and alpha nodes for each rule in the statistical rule base is counted, and the similarity ratio of each pair of rules is determined based on the number of type nodes and alpha nodes of each pair of rules.

[0016] By forming connection paths between type nodes and alpha nodes, the number of identical connection paths is counted to determine the path universality between pairs of rules;

[0017] The similarity of pairwise rules is obtained by combining the similarity ratio and the generality of the paths.

[0018] Preferably, an accounting database is constructed using a historical database and a rule sample database. New rules are obtained from the accounting database, and the new rules are analyzed to determine their associated rule set. Based on the associated rule set, the data delay status of the new rules entering the rule engine and being added to the rule base is determined, and a data delay response window is set, including:

[0019] Remove the data from the rule sample database from the historical database and integrate the remaining data from the historical database to establish the accounting database;

[0020] When data from the accounting database enters the rule engine, new rules are generated based on the merchant's needs. The new rules are then analyzed in conjunction with the rule set to determine the rule set to which they belong.

[0021] Analyze the data corresponding to the new rules, combine it with the attribution rule set, determine the data delay status, and set the data delay response window.

[0022] Preferably, the new rule is analyzed in conjunction with the rule set to determine the rule set to which the new rule belongs, including:

[0023] Obtain the similarity between the new rule and each rule in each rule set, and count the number of rules in each rule set to obtain the rule incorporation property;

[0024] A preset evaluation threshold is set, and the evaluation threshold is compared with the rule incorporation to determine the corresponding rule set to which the new rule belongs.

[0025] Preferably, the data corresponding to the new rule is analyzed, and combined with the attribution rule set, the data delay status is determined, and a data delay response window is set, including:

[0026] A sliding window is preset, and a dataset is built by integrating the data corresponding to the new rules. Combined with the set of attribution rules, the time delay window of the alpha network is determined.

[0027] The triggering time of the alpha node is determined based on the time delay window of the alpha network, and the beta delay level of the dataset execution to the beta network is determined.

[0028] Obtain the search results, and determine the data delay response window duration from the execution of data in the dataset to the search results based on the beta latency level and the sliding window.

[0029] The data delay response window is set by specifying the data delay response window duration.

[0030] Preferably, the alpha network time delay window is determined as follows:

[0031] The timestamps of the new rule entering the corresponding rule set and the dataset traveling to the alpha node are counted respectively. Several initial time matching windows of the new rule and dataset traveling to the alpha node are obtained by sliding window. The same initial time matching windows are filtered to obtain the alpha network time delay window.

[0032] Preferably, determining the beta latency level of the dataset execution to the beta network includes:

[0033] Based on the alpha network time delay window, the alpha nodes triggered by the data corresponding to the new rule are arranged to form the actual alpha arrangement structure sequence.

[0034] The initial predetermined alpha arrangement structure sequence is obtained from the alpha network, and the beta delay level is obtained by combining the initial predetermined alpha arrangement structure sequence and the actual alpha arrangement structure sequence.

[0035] To address the aforementioned issues, this invention also provides an automated data accounting system based on a rule engine. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions stored in the memory to execute the automated data accounting method based on a rule engine as described in any of the preceding claims.

[0036] The present invention has the following beneficial effects:

[0037] 1. This application proposes an intelligent distributed scheme for rules. It obtains a historical database from merchants and constructs corresponding rule sample databases as a rule engine and rule base. Then, by evaluating the similarity between pairs of rules, the rule base built into the rule engine intelligently divides the rule sets. After data from the constructed accounting database enters the rule engine, it analyzes new rules to determine their corresponding rule sets, automatically assigning them to the appropriate rule sets within the rule base. Next, it determines the data latency state when a new rule is added to the rule base to set a data latency response window. Based on this window, it determines the data latency performance to adjust the rule sets for data accounting operations. This method only filters out a small number of rule sets that need matching, saving significant time spent matching other unnecessary rules, realizing a distributed data accounting process, and improving the automation efficiency of the final data accounting.

[0038] 2. The automated data accounting system based on a rule engine provided by this invention has the same beneficial effects as the automated data accounting method based on a rule engine provided by this invention, and will not be described in detail here. Attached Figure Description

[0039] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating the steps of an automated data accounting method based on a rule engine, as provided in one embodiment of the present invention;

[0041] Figure 2This is a schematic diagram of rule creation code for an automated data accounting method based on a rule engine, provided in one embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the rule set allocation code for an automated data accounting method based on a rule engine, provided in one embodiment of the present invention.

[0043] Figure 4 This is a code diagram illustrating a data delay state in an automated data accounting method based on a rule engine, provided as an embodiment of the present invention. Detailed Implementation

[0044] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automated data accounting method based on a rule engine proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] The following description, in conjunction with the accompanying drawings, details a specific scheme for an automated data accounting method based on a rule engine provided by the present invention.

[0047] For data accounting platforms with massive data volumes, traditional accounting methods require a significant amount of time to sequentially search and match each rule, resulting in low automation efficiency in the final data accounting process. Therefore, an intelligent distributed scheme based on rules is adopted to complete the corresponding rule accounting division, selecting only a small set of rules that need to be matched, saving a lot of time in matching other unnecessary rules, realizing a distributed data accounting process, and improving the automation efficiency of the final data accounting process.

[0048] Please see Figure 1 The diagram illustrates a flowchart of the steps of an automated data accounting method based on a rule engine, provided in the first embodiment of the present invention. The method includes:

[0049] Step S1: Obtain historical databases from merchants, build a rule sample database as a rule engine from the historical databases, and establish rules according to the needs of merchants to build a rule library;

[0050] Step S2: Evaluate the similarity between pairs of rules in the rule base and divide the rule base into multiple rule sets;

[0051] Step S3: Construct an accounting database using historical databases and rule sample databases, obtain new rules from the accounting database, analyze the new rules to determine the rule set to which they belong, determine the data delay status of the new rules entering the rule engine and adding them to the rule base based on the rule set to which they belong, and set the data delay response window.

[0052] Step S4: Evaluate the data latency performance based on the data latency response window, adjust and iterate the rule set based on the data latency performance until the data accounting is completed.

[0053] To better illustrate this, in real-world scenarios, various database accounting operations are required. Typically, well-defined and executable business rules serve as a rule engine, ensuring that each data accounting operation adheres to the same standards, enabling data matching, verification, calculation, and result generation. However, traditional accounting methods require manual verification of large amounts of data line by line according to preset rules. This is not only resource-intensive but also prone to errors due to human negligence or fatigue, affecting the reliability of decisions. To address this, the RETE algorithm has been used to automate the accounting process. However, this algorithm is inefficient for processing large volumes of data and cannot handle large amounts of data in a short time. Therefore, a rule engine-based automated data accounting method is proposed to automate data accounting, improve data processing efficiency and accuracy, and enhance the reliability of decisions.

[0054] As an optional implementation method, historical databases are obtained from merchants who need to perform accounting tasks such as financial reconciliation, inventory counting, transaction settlement, tax declaration, and audit verification. Preferably, in this embodiment, a database of bank customer information is used as the historical database, which has a large amount of data, including basic account information of bank customers, such as customer name, account number, and bank, as well as detailed information such as customer transaction records, account balance changes, historical transfer details, loan repayment records, and credit card consumption data. The specific implementation scenario can be adjusted according to the actual situation.

[0055] As explained, in step S1, a certain amount of sample data is selected from the obtained historical database according to a certain proportion to establish a rule sample database, which is used as the rule engine. Typically, 10% or other proportions of sample data are randomly selected to build the rule engine. In the rule sample database, multiple rules are built according to the merchant's needs, and all rules are stored to establish a rule library.

[0056] Please see Figure 2 Furthermore, the rules include multiple types of nodes, alpha nodes, and beta nodes. Alpha nodes construct alpha networks to form rules, and beta nodes construct beta networks to form rules.

[0057] It can be explained that a rule is a statement used for business decisions in a rule engine. It is composed of multiple nodes that collaborate to achieve specific logical judgments and event processing. The type node refers to the data type or entity category involved in the rule, i.e., object attributes or characteristics, providing the object basis for the operation and ensuring that subsequent nodes can process the correct data type. Rules typically use the format IF ... THEN ... or WHEN ... THEN ..., containing two parts: condition and result. The condition is the IF or WHEN part, i.e., the LHS (Left Hand Side), formed by an alpha network built from alpha nodes. It is a series of pattern matches used to determine whether the current fact meets the condition, i.e., using the alpha network for fact matching and condition filtering. The result is the THEN part, i.e., the RHS (Right Hand Side), formed by a beta network built from beta nodes. It is the action to be executed when all conditions are met, i.e., the beta network associates the facts that meet the conditions with the corresponding result actions.

[0058] The alpha network, built by alpha nodes, and the beta network, built by beta nodes, are two different types of node network architectures in blockchain technology. The alpha network mainly reflects the differences in rules, i.e., conditions; the beta network mainly reflects the execution of rules, i.e., results. In other words, type nodes are used to define the data object categories of rule operations, alpha nodes are responsible for filtering the conditions of facts, and beta nodes handle the association and combination judgment between multiple facts. The three work together to promote accurate accounting between rules and data.

[0059] To better illustrate this, let's take a database of bank customer information as an example. Suppose the rule is "frequent failed transaction detection". In this rule, the type node is the business processing status of individual or corporate customers, i.e., business entities such as orders and products; the judgment condition is the set factual attribute constraint, such as the password error reaching the maximum number of times, insufficient account balance, abnormal transaction mode, abnormal account association, etc.; the final result is "triggering risk control alarm".

[0060] Under normal circumstances, the rule sample database usually contains data that needs to be calculated so that the data can be retrieved for calculation. Therefore, the rule sample database usually contains a large number of different types and quantities of data, which to a certain extent reflects the relatively large number of rule setting requirements. Existing rule engines usually use the RETE algorithm to formulate multiple rules for processing, so as to guide the data in the rule sample database to the reserved cache space for subsequent data calculation.

[0061] While existing rules may differ to some extent, they are not entirely different and often use similar judgment conditions. For example, in the rule base for bank customer information, two or more rules may use the judgment condition of "users with deposits greater than 10,000 yuan". This indicates that there are certain similarities between different rules. Therefore, by analyzing the similarity between different rules in the rule base and dividing the rule base into several rule sets, the rule engine can frequently retrieve data from the corresponding cache space within the corresponding rule set when new data is retrieved. This can effectively avoid conflicts between multiple rules when new data is retrieved, which could interfere with the efficiency of data calculation results.

[0062] In real-world scenarios, different rules require different alpha and beta nodes. However, similar rules may reuse alpha and beta nodes. Specifically, in node relationships, rules are established based on merchant needs. When corresponding operations are issued based on type nodes and transmitted to the corresponding alpha nodes, the judgment conditions set in the alpha nodes are used to filter and transmit the data to the matching beta nodes to obtain the corresponding execution results. Therefore, by specifically analyzing the number of alpha and beta nodes and the path they form, rules with similar processing situations can be effectively identified, thus dividing the data into several rule sets.

[0063] Further, step S2 includes:

[0064] Step S21: Analyze the type node and alpha node of the rules in the rule base to determine the similarity between pairs of rules.

[0065] To clarify, among the multiple rules formed based on the existing RETE algorithm, each rule can be regarded as a path. The main difference between the rules is the judgment condition, that is, the constructed alpha network.

[0066] Further, step S21 includes:

[0067] Step S211: Count the number of type nodes and alpha nodes for each rule in the rule base, and determine the similarity ratio of each pair of rules based on the number of type nodes and alpha nodes of each pair of rules.

[0068] Optionally, in this embodiment, rules are adopted. and rules Please provide relevant explanations.

[0069] Specifically, the number of type nodes and alpha nodes occupied by each rule is counted to obtain the rule. and rules The total number of type nodes and alpha nodes contained in each rule is denoted as . Then statistical rules and rules The number of type nodes in both rules that are the same as the number of alpha nodes is denoted as . The similarity percentage of each pairwise rule is obtained, and the corresponding calculation formula is:

[0070]

[0071] in, This indicates the proportion of similarity between pairs of rules; This indicates the number of type nodes in each pairwise rule that are the same as the alpha node; This represents the total number of type nodes and alpha nodes contained in each pair of rules.

[0072] Step S212: Form connection paths between type nodes and alpha nodes, count the number of identical connection paths, and determine the path universality between pairs of rules.

[0073] It can be explained that, in the rules and rules In these two rules, the way nodes are combined with alpha nodes of the same type will form different connection paths depending on the rules. For example, suppose the rule is... and rules There are two nodes of the same type, namely the number of people and the number of times. There are also two nodes with the same alpha value, namely "greater than 1" and "greater than 2". The connection path could be a rule. For more than 1 person, more than 2 times, or rules For a number of participants greater than 2 and a number of times greater than 1; rule For more than 2 people, more than 1 time, or rules The number of participants is greater than 1 and the number of occurrences is greater than 2. Therefore, even if two rules have the same type of node and alpha node, they may produce the same or completely different judgment results due to different connection paths. Therefore, it is necessary to consider the similarity between the two rules in order to comprehensively judge the degree of similarity between them.

[0074] Specifically, the number of identical connection paths formed by the type node and the alpha node is denoted as . This data is used as the path universality between pairwise rules, denoted as... In particular, in practical applications, when there is no single instance of identical connection paths, path universality becomes crucial. The default value is 0.1; where path universality refers to the repeated occurrence of connection paths between nodes of the same type and alpha nodes throughout the entire alpha network, which is used to reflect the structural similarity or the same connection logic between rules.

[0075] Step S213: Combine similarity percentage and path generality to obtain the similarity degree of each pair of rules.

[0076] Specifically, the corresponding calculation formula is:

[0077]

[0078] in, Indicates the degree of similarity between pairwise rules; This indicates the proportion of similarity between pairs of rules; This indicates the path universality of pairwise rules.

[0079] It can be explained that when the similarity between pairwise rules... The larger the value, the more similar the structural paths of the two rules in the alpha network constructed by the RETE algorithm. This indicates that the node structures in the actual application of the two rules are more similar, and that the two rules will go through similar processing procedures when matching facts, thus reflecting a strong correlation between the two rules in business logic or application scenarios.

[0080] Step S22: Define similarity as a distance metric, perform clustering using the distance metric, divide into multiple clusters, and each cluster corresponds to a set of rules.

[0081] Preferably, in this embodiment, the K-means clustering method is used for clustering, with the default value of K being 5. The specific value can be adjusted according to the actual situation. By determining the number of clusters K, K initial centroids are randomly selected as the initial center points of each cluster, so that each sample data is assigned to the cluster containing the nearest centroid. This clustering method has high computational efficiency and is simple to implement, making it suitable for clustering analysis of large-scale datasets.

[0082] Specifically, similarity is used as a distance metric to perform K-means clustering on the rules in the rule base, resulting in several clusters. Each cluster is then used as a rule set, meaning that the rules in each rule set have a certain logical consistency, which facilitates subsequent data verification.

[0083] Understandably, based on the relevant operations in step S2, the rule classification in the constructed rule base is initially clarified. This rule base, as a rule engine, can continuously calculate the remaining data in the historical database. In actual data calculation scenarios, the remaining data in the historical database still has a large amount of data. Therefore, when calculating it, it usually arrives at the rule engine's processor in batches, that is, it enters the rule sample database in batches. When the rule engine is actually calculating the remaining data in the historical database, it may be interfered with by factors such as network latency, causing the data that should have entered the rule engine at the predetermined timestamp to be delayed. This results in the data entering the rule engine at a certain time after the predetermined timestamp. At this time, the data that enters late should be matched with the data within the time range adjacent to the predetermined timestamp. However, the data matching process between the rule sample database and the rule base is continuous. This causes the data that enters late to have already completed the matching process or entered the irreversible matching node by the time it enters the rule engine, resulting in an error state. Therefore, it is necessary to perform data delay state analysis on the data newly added to the rule engine based on the rule set in order to intelligently set the data delay response window.

[0084] Furthermore, step S3 includes:

[0085] Step S31: Remove the data from the rule sample database from the historical database and integrate the remaining data in the historical database to establish the accounting database.

[0086] To clarify, when the remaining data in the historical database used to build the rule sample database enters the rule sample database in the aforementioned step S1, new rules may be generated due to the needs of merchants. Therefore, it is necessary to traverse and exclude the data in the new input rule engine, i.e. the rule sample database, to obtain the accounting database. This database is used to analyze whether the new rules generated when entering the rule engine from the accounting database belong to any rule set in the existing rule base, and then to assess whether it is necessary to create a new rule set to place the new rules.

[0087] Step S32: When the data in the accounting database enters the rule engine, new rules are obtained according to the merchant's needs. The new rules are analyzed in combination with the rule set to determine the rule set to which the new rules belong.

[0088] It is explained that when any data in the accounting database enters the rule engine, i.e. the rule sample database, the merchant generates new rules based on the data in the corresponding rule sample database according to their needs.

[0089] Further, in step S32, the new rule is analyzed in conjunction with the rule set to determine the rule set to which the new rule belongs, including:

[0090] Step S321: Obtain the similarity between the new rule and each rule in each rule set, and count the number of rules in each rule set to obtain the rule incorporation property.

[0091] Specifically, based on step S2, the similarity between the new rule and each rule in each rule set is determined, and the rule incorporation between the new rule and the rule set is calculated. That is, the new rule is adapted to the rule set by comprehensively analyzing the similarity ratio and path generality between the new rule and any rule in the rule set. The corresponding calculation formula is:

[0092]

[0093] in, Indicates rule incorporation; Represents the normalization function; Indicates the number of rules in the current rule set; Indicates the number of rules in the current rule set. One rule; Indicates the new rule and the first The degree of similarity between the rules.

[0094] It can be explained that, similarly, the rule incorporation property between the new rule and all rule sets in the rule base can be obtained. The greater the rule incorporation property, the more suitable the new rule added to the rule sample database is to be incorporated into the rule set of the current analysis.

[0095] Step S322: Preset the evaluation threshold, compare the evaluation threshold with the rule incorporation, and determine the corresponding rule set to which the new rule belongs.

[0096] Preferably, in this embodiment, the evaluation threshold is set to 0.65, and the specific value can be adjusted according to the actual situation.

[0097] Specifically, the maximum value is selected based on the rule incorporation among the new rule and all rule sets in the rule base. When the maximum rule incorporation is greater than the evaluation threshold, i.e. When the maximum rule inclusion property is less than or equal to the evaluation threshold, the rule set corresponding to the maximum rule inclusion property is taken as the ideal rule set for the new rule, and the new rule is included in the ideal rule set, indicating that the ideal rule set is the corresponding rule set to which the new rule belongs; otherwise, when the maximum rule inclusion property is less than or equal to the evaluation threshold, i.e. When a new computing node is established, a new rule set is constructed based on that node, and the new rule is incorporated into this new rule set. This indicates that the new rule set is the rule set to which the new rule belongs.

[0098] Please see Figure 3To better illustrate, based on database analysis of bank customer information, the computing node refers to the alpha node. Each node corresponds to multiple different rule sets. If the maximum rule incorporation value between the new rule and all rule sets in the rule base is greater than the evaluation threshold, and the rule set corresponding to the maximum rule incorporation value is risk control rule 1, then the new rule is incorporated into risk control rule 1. Conversely, if the maximum rule incorporation value is less than or equal to the evaluation threshold, a new computing node is created outside the current three computing nodes, and a new rule set is established, and the new rule is incorporated into the new rule set.

[0099] Step S33: Analyze the data corresponding to the new rule, combine it with the attribution rule set, determine the data delay status, and set the data delay response window.

[0100] It can be explained that, compared to the original RETE algorithm which concentrates operations on the same server, the operation of distributed data accounting by dividing the rule set and allocating computing nodes distributes the data processing tasks that were originally concentrated on a single server to other servers, and also changes the data access process of the entire data accounting project. While the use of distributed accounting can effectively reduce the pressure of data processing, it will also amplify the latency caused by the original data delay to a certain extent. Therefore, it is necessary to reserve a certain delay response window to reduce latency interference.

[0101] Further, step S33 includes:

[0102] Step S331: Set a sliding window and integrate the data corresponding to the new rules to build a dataset. Combine the set of rules to determine the alpha network time delay window.

[0103] Optionally, the duration of the sliding window is recorded as In this embodiment, Each slide takes 1 second; the specific value can be adjusted according to the actual situation.

[0104] Further, in step S331, the alpha network time delay window is determined, specifically as follows:

[0105] The timestamps of the new rule entering the corresponding rule set and the dataset traveling to the alpha node are counted respectively. Several initial time matching windows of the new rule and dataset traveling to the alpha node are obtained by sliding window. The same initial time matching windows are filtered to obtain the alpha network time delay window.

[0106] Specifically, the timestamps of new rules entering their corresponding rule sets are counted. These timestamps are then combined with sliding windows to obtain several initial time-matching windows for the new rule. For example, assuming the timestamp of a new rule entering its corresponding rule set is 10:00:00, the left end of the window slides from 9:55:00 until the right end slides to 10:05:00. The time range of 9:55:00-10:05:00 is the buffer period for the new rule. Within this buffer period, the window slides for 1 second at a time, forming multiple sliding windows. Each sliding window serves as the corresponding initial time-matching window. Next, multiple datasets corresponding to the new rule are integrated to create a dataset. The timestamps of the dataset reaching the alpha node are counted. Based on the initial time-matching windows of the new rule, several initial time-matching windows for the dataset reaching the alpha node are obtained similarly. Then, the two initial time-matching windows are filtered to obtain the initial time-matching windows that are the same for both the new rule and the dataset reaching the alpha node. This window is used as the alpha network time delay window for the dataset reaching the alpha node.

[0107] Step S332: Determine the triggering time of the alpha node based on the determined alpha network time delay window, and determine the beta delay level of the dataset execution to the beta network.

[0108] It can be explained that after the dataset passes through the alpha node, it enters the beta network and then enters the rule feedback stage. In real-world scenarios, a single beta node may connect to multiple alpha nodes. Since the same rule may impose restrictions on the order of alpha node combinations, some results can only be generated if the preconditions are met. This means that the dataset will only enter the beta node if the judgment conditions of the alpha node are met. Therefore, within the alpha network time delay window, that is, within a time delay window that reserves a certain amount of time to cope with network fluctuations and transmission time within the normal range, the triggering time of the alpha node is considered to determine the beta delay level of the dataset from execution to the beta network.

[0109] Further, in step S332, determining the beta latency of the dataset being executed to the beta network includes:

[0110] Step S3321: Based on the alpha network time delay window, arrange the alpha nodes triggered by the data corresponding to the new rule to form the actual alpha arrangement structure sequence.

[0111] Specifically, within the alpha network time delay window, the sequence of alpha nodes triggered by the new rule is recursively arranged according to the time sequence. That is, the corresponding alpha nodes triggered by the new rule are recursively arranged according to the order of events, forming the actual alpha arrangement structure sequence of the new rule.

[0112] Step S3322: Obtain the initial predetermined alpha arrangement structure sequence based on the alpha network, and combine the initial predetermined alpha arrangement structure sequence and the actual alpha arrangement structure sequence to obtain the beta delay level.

[0113] Specifically, the initial pre-defined alpha layout sequence is obtained based on the core logic and application scenario of the rule and is deployed on the alpha network to reflect the deployment status of the rule. Each rule has a corresponding initial pre-defined alpha layout sequence by default, which is stored in the rule base and can be used directly without additional separate settings.

[0114] Specifically, the number of triggers of the actual alpha arrangement structure sequence within the initial predetermined alpha arrangement structure sequence is counted, denoted as . And determine the trigger timestamp corresponding to each trigger, denoted as . The formula for calculating beta latency is as follows:

[0115]

[0116] in, Indicates the degree of beta delay; Indicated by An exponential function with base 0; This indicates the number of times the actual alpha arrangement structure sequence is triggered within the initial predetermined alpha arrangement structure sequence; This indicates the actual alpha arrangement structure sequence in the initial predetermined alpha arrangement structure sequence. This trigger; , They represent the first Second and third The trigger timestamp corresponding to each trigger; Indicates the absolute value symbol; This represents the coefficient to prevent the denominator from being 0.

[0117] What can be explained is the degree of beta latency. The larger the value, the more delays occur within the reserved alpha network time delay window, and the more significant the data delay effect. In other words, within the alpha network time delay window, factors such as network congestion, route switching, packet retransmission, and excessive load on processing nodes may cause the delay to accumulate and increase, resulting in end-to-end delays exceeding expectations.

[0118] To better understand the trigger count, let's take an example. Suppose the initial predetermined alpha arrangement structure sequence is... The actual alpha arrangement structure sequence formed by the alpha nodes triggered by the new rule is as follows: The two are compared to obtain the trigger condition marked as A total of three triggers were performed. This indicates that within the reserved alpha network time delay window, the interval between each trigger can be shorter or longer depending on the number of triggers. For example, when the number of triggers is 2, the interval between two triggers is 3 seconds, while when the number of triggers increases to 4, the sum of the intervals for the four triggers is 2.5 seconds. Alternatively, when the number of triggers is 2, the interval between two triggers is shorter, at 0.5 seconds, while when the number of triggers increases to 4, the corresponding sum of the trigger intervals remains 2.5 seconds. Regarding the different intervals, the beta delay corresponding to 4 triggers is greater than that corresponding to 2 triggers, meaning that within this window, the beta delay is greater. The larger the value, the more delays will occur.

[0119] Specifically, when the number of triggers is 0 or 1, There are cases where the value is 0 or the formula is meaningless, thus requiring the coefficients to be considered. To ensure that the overall value is not zero, preferably, in practical applications, the coefficient... The value can be 1 for calculation; when the number of triggers is 0, it means that no delay has occurred, indicating that the data calculation is running well within the time delay window of the alpha network.

[0120] Step S333: Obtain the search results and determine the data delay response window duration from data execution to search results based on the beta delay level and the sliding window.

[0121] Specifically, in this embodiment, the retrieval result is the agenda, which is the time point at which the data delay response window duration analysis algorithm logic reaches the node where the retrieval result is to be output. The corresponding calculation formula is:

[0122]

[0123] in, This indicates the duration of the data delay response window when data is processed and retrieved as a result. Indicates the duration of the sliding window; This indicates the degree of beta delay.

[0124] Step S334: Set the data delay response window duration; that is, when the data execution reaches the search result, a time limit is reserved. The data delay response window.

[0125] Please see Figure 4 This can be explained as follows: When the rule engine performs calculations on the data in the calculation database, after running the aforementioned steps, a data delay response window is obtained. If a misalignment occurs within the data delay response window, that is, in the scenario of "network delay causing abnormal data arrival order", an error state is generated due to network problems, and this situation is then assessed as a delay in the data calculation of the dataset called by the new rule. Conversely, if no misalignment occurs within the data delay response window, or if a misalignment occurs after the data delay response window, it indicates that there is no delay in the data calculation of the dataset called by the new rule.

[0126] Understandably, if the data delay phenomenon triggered by any rule set is more frequent when the new rule calls the dataset for data calculation, it may indicate that the data type combination in the rule set itself is more complex, and the interference caused by it during time matching is more obvious. Therefore, based on the data delay response window, the distribution content of rules in the corresponding rule set is analyzed, and the existing rule set is intelligently adjusted to iterate the rule set division.

[0127] As explained, in step S4, within the data delay response window, the number of times the misalignment state occurs is counted and recorded as... Then, count the total number of nodes contained in the rule set corresponding to the new analysis rule, i.e., the number of type nodes, alpha nodes, and beta nodes, denoted as . The formula for evaluating data latency performance is as follows:

[0128]

[0129] in, This indicates the data latency performance after the data latency response window for the application of the new rule; Represents the normalization function; This indicates the number of times the misalignment occurs within the data delay response window; Indicates the first The situation where a secondary misalignment occurs; Indicates the first When a misalignment occurs, the beta delay level of the new rule's corresponding dataset being executed in the beta network; This indicates the number of type nodes, alpha nodes, and beta nodes within the rule set corresponding to the new rule.

[0130] This can be explained by searching for results with the same conditions within the data delay response window. If they exist, the number of different occurrences of the results is counted as the number of times the misalignment state occurs. In particular, if they do not exist, The default value is 1; each occurrence of a misalignment corresponds to a complete execution process of a rule. A higher data latency indicates a more complex network structure built by the alpha and beta networks within the rule set to which the new rule belongs, suggesting that the resulting data latency may be caused by this complex network structure.

[0131] Next, a preset judgment threshold is set. In this embodiment, the judgment threshold is set to 0.5, and the specific value can be adjusted according to the actual situation; when the data delay performance When the new rule's corresponding rule set and the original rule set are subjected to another K-means clustering, all the newly redefined rule sets under the current conditions are taken as the latest intelligent rule set after the new rule is applied, and then a cleanup operation is triggered. Preferably, when performing K-means clustering, the K value is 2 by default to minimize the data computation pressure; conversely, when the data latency is high, the K value is lower than 2. When the current data latency is good and there are no frequent delays, it means that the current rule set can perform data accounting. Similarly, the rule set is continuously iterated until the data accounting process in the accounting database is completed.

[0132] To better illustrate, the data cleaning operation occurs after the data delay response window. When the data delay response window intelligently set for a new rule triggers the cleaning operation, theoretically, the rule engine should continue to import subsequent new rules or new data. However, frequent data delays cause the rule set to be updated to trigger the cleaning operation. Specifically, after the cleaning operation is triggered, all states related to the data delay response window are removed from the RETE network. That is, all facts in the alpha node that belong to the data delay response window and all partial matches in the beta node that belong to the data delay response window are cleaned up, effectively removing outdated or invalid state information caused by data delays to reduce data interference.

[0133] As an optional implementation, in the current distributed environment, a unified coordinator or clock- or watermark-based signals are usually used to trigger cleanup operations on each node to ensure the stable operation of the entire data accounting process and the reasonable allocation of resources. The coordinator includes JobManager, etc. In the stream processing framework Flink (coordinator), JobManager (job manager) acts as the coordination center of the current node cluster, responsible for receiving status updates from each node and issuing corresponding cleanup trigger instructions to release redundant data in a timely manner.

[0134] Understandably, this application proposes an intelligent distributed scheme for rules. It obtains a historical database from merchants and constructs corresponding rule sample databases as a rule engine and rule base. Then, by evaluating the similarity between pairs of rules, the rule base built into the rule engine intelligently divides the rule sets. After data from the constructed accounting database enters the rule engine, it analyzes new rules to determine their corresponding rule sets, automatically assigning them to the appropriate rule sets within the rule base. Next, it determines the data latency state when a new rule is added to the rule base to set a data latency response window. Based on this window, it determines the data latency performance to adjust the rule sets for data accounting operations. This method only filters out a small number of rule sets that need matching, saving significant time spent matching other unnecessary rules, realizing a distributed data accounting process, and improving the automation efficiency of the final data accounting.

[0135] The second embodiment of the present invention provides an automated data accounting system based on a rule engine. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the automated data accounting method based on a rule engine as described in any embodiment of the present invention. This system has the same beneficial effects as the aforementioned automated data accounting method based on a rule engine, and will not be described in detail here.

[0136] Understandably, when the rule engine-based automated data accounting system is in operation, it needs to utilize the rule engine-based automated data accounting method provided in any of the foregoing embodiments. Therefore, whether the system and program data are integrated or different hardware is configured to produce functions with similar effects to those achieved by the present invention, they all fall within the protection scope of the present invention.

[0137] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for automated data accounting based on a rule engine, characterized in that, The method comprises: Based on the merchant acquisition history database, a rule sample database is constructed from the history database for the rule engine, and rules are established according to the merchant demand to construct a rule library; The similarity of each two rules in the rule library is evaluated, and the rule library is divided into multiple rule sets; An accounting database is constructed from the history database and the rule sample database, a new rule is obtained from the accounting database, the attribution rule set of the new rule is determined by analyzing the new rule, the data delay state of the new rule entering the rule engine to join the rule library is judged according to the attribution rule set, and a data delay reaction window is set; The data delay performance degree is evaluated based on the data delay reaction window, and the rule set is adjusted and iterated through the data delay performance degree until the data accounting is completed.

2. The rule engine based automated data accounting method as claimed in claim 1, wherein, The rule comprises multiple type nodes, alpha nodes and beta nodes, the alpha node constructs an alpha network to form the condition of the rule, and the beta node constructs a beta network to form the result of the rule.

3. The rule engine based automated data accounting method as claimed in claim 2, wherein, The similarity of each two rules in the rule library is evaluated, and the rule library is divided into multiple rule sets, which comprises: The type node and the alpha node of the rule in the rule library are analyzed to determine the similarity of each two rules; The similarity degree is defined as a distance metric, clustering is performed through the distance metric, and multiple clustering clusters are divided, each clustering cluster corresponding to each rule set.

4. The rule engine based automated data accounting method as claimed in claim 3, wherein, The type node and the alpha node of the rule in the rule library are analyzed to determine the similarity of each two rules, which comprises: The number of type nodes and alpha nodes of each rule in the rule library is counted, the similarity proportion content of each two rules is determined based on the number of type nodes and alpha nodes of each two rules, the connection path is formed through the type node and the alpha node, the number of identical connection paths is counted to determine the path universality between each two rules, and the similarity of each two rules is obtained by comprehensively considering the similarity proportion content and the path universality. An accounting database is constructed from the history database and the rule sample database, a new rule is obtained from the accounting database, the attribution rule set of the new rule is determined by analyzing the new rule, the data delay state of the new rule entering the rule engine to join the rule library is judged according to the attribution rule set, and a data delay reaction window is set, which comprises: The data of the rule sample database is excluded from the history database, and the remaining data in the history database is integrated to establish the accounting database; 5. The rule engine based automated data accounting method as claimed in claim 3, wherein, When the data in the accounting database enters the rule engine, a new rule is obtained according to the merchant demand, the new rule is analyzed in combination with the rule set to determine the attribution rule set of the new rule; The data corresponding to the new rule is analyzed, the data delay state is determined in combination with the attribution rule set, and the data delay reaction window is set. The attribution rule set of the new rule is determined by analyzing the new rule in combination with the rule set, which comprises: The similarity of the new rule to each rule in each rule set is obtained, the number of rules in each rule set is counted, and the rule incorporation is obtained; 6. The rule engine based automated data accounting method as claimed in claim 5, wherein, An evaluation threshold is preset, the evaluation threshold and the rule incorporation are compared, and the corresponding attribution rule set of the new rule is determined. The data corresponding to the new rule is analyzed, the data delay state is determined in combination with the attribution rule set, and the data delay reaction window is set. ​ 7. The rule engine based automated data accounting method as claimed in claim 5, wherein, ​ A preset sliding window is set, and a data set is established by integrating data corresponding to the new rule, and the alpha network time delay window is determined by combining the rule set; The trigger time of the alpha node is determined based on the alpha network time delay window, and the beta delay degree of the data set executing to the beta network is determined; The search result is obtained, and the data delay reaction window length of the data in the data set executing to the search result is determined according to the beta delay degree and the sliding window; The data delay reaction window is set through the data delay reaction window length.

8. The rule engine based automated data accounting method as claimed in claim 7, wherein, The alpha network time delay window is determined, specifically: The timestamps of the new rule entering the corresponding rule set and the data set traveling to the alpha node are counted respectively, and a plurality of initial time matching windows of the new rule and the data set traveling to the alpha node are obtained through the sliding window, the same initial time matching windows are screened, and the alpha network time delay window is obtained.

9. The rule engine based automated data accounting method as claimed in claim 7, wherein, The beta delay degree of the data set executing to the beta network includes: Based on the alpha network time delay window, the alpha nodes triggered by the data corresponding to the new rule are arranged to form an actual alpha arrangement structure sequence; According to the alpha network, an initial predetermined alpha arrangement structure sequence is obtained, and the beta delay degree is obtained by combining the initial predetermined alpha arrangement structure sequence and the actual alpha arrangement structure sequence.

10. An automated data accounting system based on a rules engine, characterized in that, The system includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus, and the processor calls the logic instructions in the memory to execute the automatic data accounting method based on the rule engine in any one of claims 1-9.

Images