Mining method and apparatus for entity enhancing rule, device, and medium
By setting upper and lower bounds for revenue scores and using credibility to filter candidate rules in rule mining, and employing a greedy algorithm for iterative search, the problem of low rule discovery efficiency in existing technologies is solved, and efficient rule discovery is achieved.
Patent Information
- Application Number
- PCT/CN2024/105908
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-07-12
- Filing Date
- 2024-07-17
- Publication Date
- 2026-01-15
AI Technical Summary
Existing rule discovery methods suffer from problems such as too many irrelevant rules, low computational efficiency, and redundant processing, resulting in low efficiency for users in rule mining.
By initializing the initial mining rule set and the used predicate set as empty sets, and combining preset relevance and diversity measures, the candidate rules are scored for their benefits. Upper and lower bounds of the benefit scores and credibility are set to filter the best subset of candidate rules. A greedy algorithm is used for iterative search to reduce unnecessary rule interference and redundant processing.
It effectively reduces the amount of rule processing, improves the efficiency of rule discovery, reduces unnecessary iterations and redundant processing, and improves the speed and accuracy of rule discovery.
Smart Images

Figure CN2024105908_15012026_PF_FP_ABST
Abstract
Description
Methods, apparatus, equipment and media for mining entity augmentation rules
[0001] This application is based on and claims priority to Chinese Invention Application No. 202410938865.6, filed on July 12, 2024, entitled “Method, Apparatus, Device and Medium for Mining Entity Enhancement Rules”. Technical Field
[0002] This application relates to the field of big data mining technology, and in particular to a method, apparatus, equipment and medium for mining entity augmentation rules. Background Technology
[0003] Logical rules have been widely applied in fields such as data cleaning, association analysis, knowledge discovery, online recommendation, drug discovery, and manufacturing. To practically use rules, we must discover high-quality rules from real-world data. Many rule discovery methods have been researched, which typically take a dataset D as input and mine or learn rules from D, ensuring that the rules have support and confidence scores above predefined thresholds. Support measures the frequency with which a rule can be applied, while confidence assesses the degree of correlation between the preconditions and the outcome. However, industry practitioners have raised the following questions about previous rule discovery methods:
[0004] (1) Too many irrelevant rules. Previous methods typically returned the entire set of rules satisfied on D, where each rule had sufficiently high support and confidence, and this set was usually quite large. For example, 128,726 functional dependency rules were found from a small dataset with 27 attributes and 368 tuples. Users are often overwhelmed by the sheer number of rules and have to spend a lot of time manually checking and selecting rules that meet their needs. Moreover, practitioners often have prior knowledge of their applications and have even accumulated a certain number of rules through years of practice. Therefore, practitioners want unknown / novel rules that meet their needs, rather than those that are already known or common sense.
[0005] (2) High cost. Mining the entire rule set from D is very expensive. Previous methods enumerate all candidate rules and verify the support and confidence of each rule by applying it to the entire dataset. For example, even with 20 machines, it would take more than 3 hours on a dataset with 1.68 million tuples. In addition, applying a large number of mined rules to the entire dataset is usually also expensive.
[0006] (3) Redundant rules. To reduce the number of irrelevant rules and the cost of rule discovery, previous methods have studied top-k rule discovery, finding top-ranked rules based on objective metrics (support and confidence) and subjective metrics (relevance to user needs). However, since the returned rules are often too "homogeneous" with each other, they are generally considered redundant.
[0007] Typical rule discovery algorithms are based on depth-first or breadth-first search and can be roughly divided into the following two steps: (1) Discover all REE rules that meet the support / confidence threshold in the relational data; (2) Select k rules from them to form a rule subset so that the objective function F() is maximized.
[0008] Step 1: For each REE result *e*, these methods store two predicate sets: *Psel* and *Pre*. *Psel* stores the predicates already selected to form REE conditions, while *Pre* stores the candidate predicates. Initially, *Psel* is empty, and *Pre* is the set of all possible predicates. Then, the rule discovery algorithm traverses the search space using either depth-first or breadth-first search, iteratively adding predicates from *Pre* to *Psel* until all rules have been checked. Essentially, it's a process of enumerating all predicate permutations and combinations on the data and verifying them one by one. For all possible predicates, any one or more extracted predicates can form a valid rule with the REE result *e*. Therefore, for rule discovery, existing methods need to try all permutations and combinations of predicates to obtain all REE rules that satisfy the support / confidence threshold.
[0009] The second step is to enumerate all the subsets of the k rules again, and for each rule subset, calculate their ranking score F(), and finally return the k rules with the highest scores.
[0010] The main drawback of top-k rule discovery methods based on brute-force enumeration is their low computational efficiency. To obtain the subset of the k highest-scoring rules, existing techniques first need to enumerate all possible rules that satisfy the data and calculate their corresponding support / confidence. However, in practice, users are often only interested in the top-ranking rules and shouldn't incur the cost of traversing all rules. In practice, the scale of datasets is often in the millions. Secondly, to obtain the subset of the k highest-scoring rules, existing techniques need to enumerate all k rule subsets again and calculate the ranking score F() for each subset. When the data scale is large, the efficiency of this brute-force approach is undoubtedly very low. Even with only 100 rules, selecting 10 rules requires enumerating 1.7 * 10¹³ rule combinations. Even if each subset takes only 1 millisecond to process, processing all subset combinations would take 548 years.
[0011] Therefore, how to reduce unnecessary interference in rule mining and thus improve the efficiency of rule discovery has become an urgent problem to be solved.
[0012] Summary of the Invention
[0013] In view of this, embodiments of this application provide a method, apparatus, device, and medium for mining entity augmentation rules to address the problem of how to reduce unnecessary interference in rule mining, thereby improving the efficiency of rule discovery.
[0014] In a first aspect, embodiments of this application provide a method for mining entity enhancement rules, the mining method comprising:
[0015] Initialize the initial mining rule set and the used predicate set as empty sets. Merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate. Use all predicates in the used predicate set of each candidate predicate and combine them with the rule objectives of the known rules to construct the corresponding candidate rule.
[0016] Based on the preset relevance and diversity metrics, each candidate rule and its extended rules are scored for their returns, resulting in an upper bound and a lower bound for the return score of each candidate rule.
[0017] Calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, determine the subset of the best candidate rules and the subset of candidate rules for the next round of iteration. Add each rule in the subset of the best candidate rules to the initial mining rule set to obtain the updated rule set corresponding to each rule.
[0018] Calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration.
[0019] In the next iteration, an extended search is performed using the updated subset of candidate rules for the next iteration to obtain extended rules. These extended rules are then used as candidate rules, and the updated set of mining rules is used as the initial set of mining rules. The credibility of each candidate rule is calculated and the process continues until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining the updated set of mining rules.
[0020] Secondly, embodiments of this application provide an entity enhancement rule mining apparatus, the mining apparatus comprising:
[0021] The candidate rule construction module is used to initialize the initial mining rule set and the used predicate set as empty sets, merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate, and use all predicates in the used predicate set of each candidate predicate, combined with the rule objectives of the known rules, to construct the corresponding candidate rule;
[0022] The upper and lower bound calculation module is used to score the revenue of each candidate rule and its extended rules based on the preset relevance and diversity metrics, and to obtain the upper bound and lower bound of the revenue score for each candidate rule.
[0023] The rule filtering module is used to calculate the credibility of each candidate rule, and determine the best candidate rule subset and the next round of iteration candidate rule subset based on the upper bound of the profit score, the lower bound of the profit score and the credibility of each candidate rule. Each rule in the best candidate rule subset is added to the initial mining rule set to obtain the updated rule set corresponding to each rule.
[0024] The rule mining and update module is used to calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration.
[0025] The iterative loop module is used to perform an expanded search on the updated subset of candidate rules for the next iteration in the next iteration to obtain expanded rules, use the expanded rules as candidate rules, and use the updated set of mining rules as the initial set of mining rules. It then returns to calculate the credibility of each candidate rule until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining an updated set of mining rules.
[0026] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the mining method as described in the first aspect.
[0027] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the mining method as described in the first aspect.
[0028] The beneficial effects of this application embodiment compared with the prior art are as follows: This application initializes the initial mining rule set and the used predicate set as empty sets, merges each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate, uses all predicates in the used predicate set of each candidate predicate, and combines them with the rule objectives of known rules to construct the corresponding candidate rule; according to the preset relevance metric and the preset diversity metric, the benefit score of each candidate rule and its extended rules is calculated to obtain the upper bound and lower bound of the benefit score for each candidate rule; the credibility of each candidate rule is calculated, and based on the upper bound, lower bound, and credibility of the benefit score of each candidate rule, the optimal candidate rule subset and the next round of iteration candidate rule subset are determined, and each rule in the optimal candidate rule subset is added to the initial mining rule set to obtain each rule. The corresponding updated rule set is used; the profit improvement of each updated rule set compared to the initial mining rule set is calculated, and the rule corresponding to the updated rule set with the highest profit improvement is determined as the best rule for this iteration. The best rule is added to the initial mining rule set to obtain an updated mining rule set. The rules other than the best rule in the candidate best rule subset are added to the candidate rule subset for the next iteration to obtain an updated candidate rule subset for the next iteration. In the next iteration, an extended search is performed using the updated candidate rule subset for the next iteration to obtain extended rules. The extended rules are used as candidate rules, and the updated mining rule set is used as the initial mining rule set. The calculation of the credibility of each candidate rule is performed until the number of rules in the updated mining rule set reaches a preset number or a preset number of iterations, thus obtaining an updated mining rule set.
[0029] By setting upper and lower bounds for the profit score and credibility, candidate rules are filtered out. Effective rules are determined by improving profits, and rules that have already been processed and cannot be optimized for future expansion are excluded. This limits the rules for the next iteration, reduces unnecessary interference, and avoids redundant rule processing in multiple iterations. The reduction in the number of rules processed effectively improves processing efficiency. Attached Figure Description
[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 is a schematic diagram of an application environment for a method for mining entity enhancement rules provided in Embodiment 1 of this application;
[0032] Figure 2 is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 2 of this application;
[0033] Figure 3 is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 3 of this application;
[0034] Figure 4 is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 4 of this application;
[0035] Figure 5 is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 5 of this application;
[0036] Figure 6 is a schematic diagram of the structure of a mining device for entity enhancement rules provided in Embodiment 6 of this application;
[0037] Figure 7 is a schematic diagram of the structure of a computer device provided in Embodiment 7 of this application. Detailed Implementation
[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0039] To illustrate the technical solution of this application, specific embodiments are described below.
[0040] The entity augmentation rule mining method provided in Embodiment 1 of this application can be applied in the application environment shown in Figure 1. The server connects to the client and a corresponding database. The client configures corresponding metrics (i.e., parameters for pruning operations) to the server. The server obtains data stored in the database and performs mining based on the above method. The client can be, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs), etc. The server can be a standalone server or an integrated server. The database can be a relational database such as MySQL or SQL Server, or a non-relational database.
[0041] Referring to Figure 2, which is a flowchart illustrating a method for mining entity augmentation rules according to Embodiment 2 of this application, the above mining method is applied to the server in Figure 1. The client configures the corresponding metrics (i.e., parameters used for pruning operations) to the server. The server obtains the data stored in the database and executes subsequent method steps to mine entity augmentation rules. As shown in Figure 2, the method for mining entity augmentation rules may include the following steps:
[0042] Step S201: Initialize the initial mining rule set and the used predicate set as empty sets. Merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate. Use all predicates in the used predicate set of each candidate predicate and combine them with the rule objectives of the known rules to construct the corresponding candidate rule.
[0043] In this embodiment, the rule used is the Entity Enhancing Rules (REE). The basic component of an REE is the predicate p, defined as follows:
[0044] p=R(t)|tA◎c|tA◎sB|M(tA,sB)
[0045] Here, ◎ is an operator that can be equal to or not equal to; R(t) indicates that t is a tuple variable in the relation table R; tA represents the A attribute of variable t; M is a machine learning model that returns true if tA and sB are related, otherwise it returns false. tA◎c has a constant and is a constant predicate; tA◎sB does not have a constant and is a variable predicate; M(tA,sB) is a machine learning predicate.
[0046] Based on predicates, the definition of REE is: X→e. Wherein, (1) X is a combination of multiple predicates, which is the precondition of REE; (2) e is a predicate, which is the result of REE.
[0047] A specific REE Examples are as follows:
[0048] Express delivery (t) ∧ Express delivery (s) ∧t.Recipient = s.Recipient ∧t.Address = "City S" → s.Postal Code = "000000"
[0049] This REE The scenario described is that if two packages, t and s, are addressed to the same person, and t's address is in "City S", then s's postal code must be "000000".
[0050] The mining rule set is a set that stores the mined rules. The initial mining rule set is set to an empty set at the beginning of step S201 so that it can be used to store the rules mined later.
[0051] The candidate predicate set is a collection that stores all candidate predicates, and this candidate predicate set can be obtained directly from the database.
[0052] The used predicate set is a collection of predicates that have been used to construct rules. At the beginning of step S201, the used predicate set is empty. After the current iteration, it contains one predicate. In subsequent iterations, corresponding predicates are stored in the used predicate set. After extracting a candidate predicate from the candidate predicate set, the candidate predicate set becomes the remaining candidate predicate set (after removing the extracted candidate predicates).
[0053] The initial set of used predicates is empty. After merging with a selected candidate predicate, the set of used predicates now contains the candidate predicate. Combining this with the known rule objectives, a candidate rule is constructed that enables the candidate predicate to achieve the rule objective. The known rules are rules derived from the user's prior knowledge or interests, and they express the goal of the rule.
[0054] Each candidate rule corresponds to a set of used predicates and a set of remaining candidate predicates. These are used in subsequent iterations to expand the candidate rule using predicates from the set of remaining candidate predicates. The number of candidate rules formed in step S201 is equal to the number of predicates in the candidate predicate set.
[0055] Step S202: Based on the preset relevance metric and the preset diversity metric, score the benefit of each candidate rule and the rules derived from its extension, and obtain the upper bound and lower bound of the benefit score for each candidate rule.
[0056] In this embodiment, both the preset relevance metric and the preset diversity metric are selected and set by the user of the aforementioned operating client. They can also be used to form the pruning strategy used subsequently. Specifically, to discover rules that meet user needs, a relevance metric is defined to capture the user's prior knowledge or interests. Given a REE... use Representation rules The correlation, thus The higher, The more relevant it is to the user's needs. Similarly, given a set of REE rules called Σ, δ div (Σ) represents the diversity among the rules in Σ, i.e., δ div The larger (Σ) is, the more diverse the rules in Σ become. The above are open to any correlation and diversity metrics and can be set according to needs.
[0057] Multiple pruning and optimization strategies based on ranking scores are employed, and additional upper and lower bounds are maintained for each rule. It is worth noting that the specific calculation method for the upper and lower bounds is related to the correlation δ. rel () and diversity δ div () Regarding metrics, different metrics are allowed to have their own calculation methods, but the upper and lower bounds of the score calculated based on any metric must meet the following conditions:
[0058] 1) Upper bound of payoff score UB. The idea behind the pruning strategy is to provide an upper bound for the payoff score for any given REE. Compute an upper bound UB such that any element from UB... The extended REE's payoff score does not exceed UB, meaning that for all REEs that can be obtained... Extended
[0059] 2) Lower bound of the payout score (LB). The value of LB is always lower than any score from... The exact payoff score for the extended REE rules, i.e., for all rules that can be obtained from... Extended
[0060] Optionally, based on preset relevance and diversity metrics, a benefit score is assigned to each candidate rule and its extended rules to obtain an upper bound and a lower bound for the benefit score of each candidate rule, including:
[0061] For any candidate rule, the benefit score of the candidate rule is calculated based on the preset relevance measure and the preset diversity measure.
[0062] Based on the remaining candidate predicates in the candidate predicate set excluding the candidate predicates corresponding to the candidate rules, the candidate rules are expanded to obtain the expanded rules;
[0063] Based on the preset relevance and diversity metrics, the benefit score for each extended rule is calculated.
[0064] Based on the revenue scores of the candidate rules and their extensions, we obtain the upper and lower bounds of the revenue scores for each candidate rule. By iterating through all candidate rules, we obtain the upper and lower bounds of the revenue scores for each candidate rule.
[0065] For a candidate rule, there is a set of used predicates and a set of remaining predicates. A predicate is selected from the set of remaining predicates and merged with the set of used predicates to form a new rule. This rule is an extension of the candidate rule. By traversing all the predicates in the set of remaining predicates, all rules that can be extended are obtained. It can be seen that the extended rule includes at least two predicates.
[0066] For a candidate rule, calculate the payout scores of all rules derived from it and the rule itself, thereby determining a maximum payout score and a minimum payout score. The maximum payout score is the upper bound of the payout score, and the minimum payout score is the lower bound of the payout score.
[0067] Step S203: Calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, determine the subset of the best candidate rules and the subset of candidate rules for the next round of iteration. Add each rule in the subset of the best candidate rules to the initial mining rule set to obtain the updated rule set corresponding to each rule.
[0068] The credibility can be either support or confidence, which is used to characterize whether a rule is reasonable. Only rules with support greater than the support threshold or confidence greater than the confidence threshold can be considered valid rules. Valid rules are a necessary condition for being written into the mining rule set to ensure that the mined rules are usable.
[0069] Credibility can be used to filter candidate rules to a certain extent to see if they can enter the mining rule set. Similarly, the upper and lower bounds of the profit score can be combined with the aforementioned credibility to filter candidate rules, thereby determining the optimal subset of candidate rules that can be selected into the mining rule set (i.e., the optimal subset of candidate rules includes at least one rule). In this process, the subset of candidate rules to be used in the next iteration can also be determined to achieve looping. In this embodiment, an iterative greedy algorithm is used as the basis to achieve looping iteration.
[0070] Maintain the best REE seen so far in this iteration round if Earnings score ratio Big (i.e.) ), for all that can be obtained Extended In other words, Therefore, this can be skipped within the current greed cycle. The extension is written into the next iteration.
[0071] For each rule in the candidate best rule subset, an updated rule set can be formed with the initial mining rule set. That is, the number of updated rule sets is the same as the number of rules in the candidate best rule subset.
[0072] Step S204: Calculate the profit improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest profit improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration.
[0073] In this embodiment, the rule that improves the benefit score of the initial mining rule set the most is selected from the candidate best rule subset and is taken as the best rule in this round, thereby forming an updated mining rule set with the initial mining rule set. If the iteration number is 1, there is 1 rule in the updated mining rule set, and so on.
[0074] For example, using the iterative greedy algorithm TopKDivMine, Σ is initially initialized as an empty set. Then, it iteratively finds the best next REE based on the current set Σ and adds this REE to Σ, until Σ contains k REE rules. The best REE added each time... All of these are obtained through calculating the payout score based on the rules. The payout score is determined when... Adding Σ improves the score of the objective function F(), mathematically represented as a REE. The score for the profit is Each time, the algorithm greedily adds the REE with the highest reward score to the current Σ, until Σ contains k REEs. At this point, the algorithm stops, and Σ is returned.
[0075] Of course, the remaining rules in the above subset of candidate best rules, other than the best rule, can be included in the next iteration, that is, these rules are written into the subset of candidate rules for the next iteration.
[0076] Step S205: In the next iteration, expand the search using the updated subset of candidate rules for the next iteration to obtain expanded rules. Use the expanded rules as candidate rules and the updated mining rule set as the initial mining rule set. Return to calculate the credibility of each candidate rule until the number of rules in the updated mining rule set reaches a preset number or a preset number of iterations, and obtain the updated mining rule set.
[0077] In this embodiment, after the previous iteration ends, the current iteration begins. Based on the rules in the updated subset of candidate rules obtained from the previous iteration, an extended search is performed on these rules. This extension can be executed according to the extended rule method described above: extension is performed for each candidate rule to obtain extended rules. These extended rules are then used as candidate rules in step S203, and the calculation is repeated until a preset number (k) of rules are found in the updated set of mining rules. Alternatively, the number of iterations can be set to avoid unlimited iteration.
[0078] To find the optimal next REE, candidate rules are generated iteratively. For each REE result e, starting from an empty premise P... sel We begin by gradually expanding it by adding predicates one after another. More specifically, suppose the candidate REE under consideration is P. sel →e, and P re It is the remaining set of candidate predicates. From P re Choose a predicate p to generate a new REE: P sel ∪{p}→e. Here, P re Predicates in the algorithm can be processed in different orders. For example, predicates that can form high-scoring rules (i.e., those with high support) can be processed first to obtain the highest-ranking REE possible. However, it is worth noting that a rule only needs to be expanded if it passes the pruning condition (i.e., the rule in the updated subset of candidate rules for the next round of iterations mentioned above).
[0079] Instead of searching the entire rule space in each iteration, TopKDivMiner maintains a search boundary, and each rule is expanded at most once. Intuitively, the search boundary is a set B of rules such that for any given rule... One of the following three conditions must be met:
[0080] 1) On the boundary, that is
[0081] 2) Within the boundary, that is, from We can extend this to obtain a rule. Make At the boundary, this indicates It has already been visited before, so it does not need to be visited again;
[0082] 3) Outside the boundary, that is, there exists a rule on the boundary. So that φ can be from This expansion indicates that φ has not been visited yet.
[0083] Specifically, the search boundary B consists of four subsets of rules: (a) Qpast, rules that have the potential to become the best rule but do not require further expansion; (b) Qrm, rules that can be completely excluded and do not require further expansion; (c) Qnext, rules that can only be excluded in the current iteration but may be expanded into the best rule in future iterations; and (d) Q, rules that have not yet been tested. In the current iteration, all rules in Q will be checked until Q becomes empty. It is worth noting that the rules within the boundary are invalid rules (i.e., they do not meet the support threshold or confidence threshold), and these rules do not need to be checked a second time. Therefore, the next iteration can allow Q = Qnext to continue searching outward from the current boundary.
[0084] Optionally, after expanding the search with the updated subset of candidate rules for the next iteration to obtain expanded rules, the following steps are also included:
[0085] For any extended rule, determine the candidate rules for the extended rule, and use the upper and lower bounds of the revenue score corresponding to the candidate rules as the upper and lower bounds of the revenue score of the extended rule.
[0086] For the extended rule, the profit score has already been calculated in step S202, and the upper and lower bounds of the profit score are consistent with the upper and lower bounds of the candidate rule for extending the extended rule, thereby avoiding repeated calculations and improving the efficiency of mining.
[0087] For example, the working principle of TopKDivMiner is as follows:
[0088] Assume that initially k = 2 and P re ={pcname, ptype, porg, pest, pUSCC, pcid} is the initial set of candidate predicates. Here, pcname, ptype, porg, pest, pUSCC, and pcid compare whether the company name, type, organization code, establishment date, unified social credit code, and company ID are equal, respectively. For example, pcid: t.cid = s.cid compares whether the IDs of company t and company s are equal. For ease of demonstration, we treat pcid as a fixed rule result.
[0089] Assuming we know the rules pcname ∧ ptype → pcid. Assuming a user wants to discover unknown rules, a relevance metric is designed, and the relevance of each candidate rule is calculated. and known rules The repeatability, if and If they are similar or equal, then The lower the relevance to the user, the lower the relevance. Furthermore, the diversity measure is the attribute diversity of the rules within Σ; that is, the fewer attributes are reused in the rules within Σ, the higher the diversity of Σ.
[0090] Initially, P sel For an empty set, from P re Choose a predicate p to generate a new REE: Psel∪{p}→pcid. Therefore, five new rules are obtained in Q: pcname→pcid, ptype→pcid, porg→pcid, pest→pcid, and pUSCC→pcid. Assuming only porg→pcid satisfies the support / confidence threshold (i.e., it doesn't need further expansion), it is added to Qpast. Assuming the upper bound of the scores of other REEs is lower than the payoff score of porg→pcid, then porg→pcid is the best REE in the first iteration, while other REEs are excluded and added to Qnext in this iteration. In the current iteration, all rules in Q are checked until Q becomes empty, the first iteration ends, and Σ={porg→pcid} is returned. Next, the second iteration continues the search outward from the current boundary using Q=Qnext. To find the second optimal REE, we need to consider not only the relevance of the candidate rules but also their differences from the first optimal REE, porg→pcid. This ensures that each rule in Σ satisfies the support / confidence threshold, and that these rules are as similar as possible to known rules. The rules of Σ are different, and the attributes used are as diverse as possible.
[0091] The efficiency of our proposed method and traditional rule discovery algorithms was compared across multiple publicly available datasets. The results show that the rule discovery method aimed at optimizing rule relevance and diversity is effective. For all k ≤ 40, PTopKDivMiner's average speed is faster than mining the entire Σ dataset. allThe rules are 62.4 times faster. This is feasible in practice: with 20 processors, finding 10 REEs on a large dataset of 1,681,617 records takes less than 818 seconds. On average, the speedup increases by 3.05 times when the number of processors increases from 4 to 20. The method also exhibits good scalability as the parameter k changes. Furthermore, the pruning strategy is effective; when k=1, PTopKDivMiner reduces runtime by 51.66 times compared to not using pruning. When the number of errors detected by the rules is used as a relevance metric, the rules returned by PTopKDivMiner are relevance and diversity. For example, when k=10, its average F1 score for true error detection is 0.74, which is 39.62% higher than the best existing method; simultaneously, the 10 rules returned by PTopKDivMiner can capture Σ all 77.5% of the errors were detected.
[0092] This application embodiment filters candidate rules by setting upper and lower bounds for profit scores and credibility. Valid rules are determined by improving profit, and rules that have already been processed and cannot be optimized for future expansion are excluded. This limits the rules for the next iteration, reduces unnecessary interference, and avoids redundant rule processing in multiple iterations. The reduction in the amount of rule processing effectively improves processing efficiency.
[0093] Referring to Figure 3, which is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 3 of this application, as shown in Figure 3, step S203 above determines the subset of candidate best rules and the subset of candidate rules for the next round of iteration based on the upper bound of the revenue score, the lower bound of the revenue score, and the credibility of each candidate rule. This includes the following steps:
[0094] Step S301: Detect whether the credibility of each candidate rule is greater than the credibility threshold. If the credibility of a candidate rule is not greater than the credibility threshold, then classify the candidate rule into the subset of candidate rules for the next round of iteration.
[0095] Step S302: If the credibility of a candidate rule is detected to be greater than the credibility threshold, the candidate rule is taken as the candidate best rule. For the first and second candidate best rules among all candidate best rules, if the lower bound of the profit score corresponding to the first candidate best rule is detected to be greater than the upper bound of the profit score corresponding to the second candidate best rule, the second candidate best rule is assigned to the subset of candidate rules for the next round of iteration.
[0096] Step S303: If an intersection is detected between the upper and lower bounds of the profit score corresponding to the first candidate best rule and the upper and lower bounds of the profit score corresponding to the second candidate best rule, then both the first candidate best rule and the second candidate best rule are classified as subsets of candidate best rules.
[0097] In this embodiment, the confidence level is compared with a confidence threshold. Rules that are not greater than the confidence threshold are directly excluded from the current iteration and added to the subset of candidate rules for the next iteration. The confidence level can be categorized as support or confidence level as described above.
[0098] If the credibility is greater than the credibility threshold, the candidate rule is taken as the candidate best rule. If there are multiple candidate best rules, the upper and lower bounds of any two candidate best rules can be determined. If the lower bound of one candidate best rule is greater than the upper bound of another candidate best rule, the candidate best rule with the smaller benefit score cannot reach the best in this round of iteration and can be excluded from the next round of iteration or directly excluded from all iterations.
[0099] If the two candidate best rules cannot be determined in the above manner, both are written into the candidate best rule subset and determined in the manner described in step S204 above, thereby obtaining the best rule.
[0100] The embodiments of this application can effectively filter candidate rules, thereby reducing the number of rules, avoiding global rule search, and using upper and lower bounds and confidence level to achieve accurate filtering, thus improving the efficiency of the entire rule mining process.
[0101] Referring to Figure 4, which is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 4 of this application, as shown in Figure 4, after calculating the credibility of each candidate rule in step S20 above, the following steps may also be included:
[0102] Step S401: Obtain the remaining quota of the initial mining rule set. The remaining quota is the difference between the preset number of rules and the current number of rules in the initial mining rule set.
[0103] Step S402: Sort all candidate rules in descending order of the lower bound of the benefit score, and determine the candidate rule whose ranking corresponds to the value of the remaining quota as the lower bound rule.
[0104] Step S403: All candidate rules whose upper bound of the profit score is less than the lower bound of the profit score of the lower bound rule are classified into the exclusion rule subset. All rules in the exclusion rule subset are removed from all rounds of iteration.
[0105] Here, the rules on the current search boundary are sorted according to their lower bounds of score and reward. The lower bound of the reward score of the k-|Σ|th rule is recorded as LBlazy(Σ), where k-|Σ| represents the remaining spots in Σ. Through this definition of the lower bound of scores and the search boundary, LBlazy(Σ) represents the lower bound of the scores of all rules in the set Σ of optimal rules that can be formed. This is because any rule is either already visited or not yet visited, but can be extended from visited rules that are on the search boundary. Any upper bound of scores... Rules below this lower bound None of them can enter the final optimal rule subset Σ, thus realizing the greedy round-robin pairing rule. The exclusion of this lower bound LBlazy(Σ) is also used to skip the actual fraction calculation, also known as lazy evaluation.
[0106] The embodiments of this application can exclude rules that are unlikely to be optimal in subsequent iterations based on the remaining quota in the mining rule set, thereby reducing the number of rule searches and helping to improve mining efficiency.
[0107] Referring to Figure 5, which is a flowchart illustrating a method for mining entity enhancement rules according to Embodiment 5 of this application, as shown in Figure 5, the calculation of the revenue improvement of each updated rule set compared to the initial mined rule set in step S204 includes the following steps:
[0108] Step S501: Using the objective function, calculate the first profit score of the initial mining rule set and the second profit score of each updated rule set.
[0109] Step S502: Subtract each second profit score from the first profit score to determine the profit increase of each updated rule set compared to the initial mining rule set.
[0110] In this embodiment, the first profit score of the initial mining rule set and the second profit score of the updated rule set are calculated, and the difference between the two can be used to see the improvement in profit after adding the corresponding rules.
[0111] Alternatively, the objective function can be as follows:
[0112] In the formula, Σ represents the initial mining rule set or the update rule set, and λ∈[0,1]. Characterize the rules within the set Σ, δ div (Σ) represents the computation of a measure of the diversity of rules within set Σ. Representation of rules Calculate the correlation measure with known rules.
[0113] The embodiments of this application can effectively analyze the improvement of revenue, thereby obtaining an accurate amount of revenue improvement, so as to accurately determine the optimal rule and improve the accuracy of rule mining.
[0114] Corresponding to the entity augmentation rule mining method in the above embodiments, Figure 6 shows a structural block diagram of the entity augmentation rule mining apparatus provided in Embodiment Six of this application. The above mining apparatus is applied to the server in Figure 1. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0115] Referring to Figure 6, the excavation device includes:
[0116] The candidate rule construction module 61 is used to initialize the initial mining rule set and the used predicate set as empty sets, merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate, and use all predicates in the used predicate set of each candidate predicate, combined with the rule objectives of the known rules, to construct the corresponding candidate rule;
[0117] The upper and lower bound calculation module 62 is used to score the revenue of each candidate rule and its extended rules according to the preset relevance measure and the preset diversity measure, so as to obtain the upper bound and lower bound of the revenue score for each candidate rule.
[0118] The rule filtering module 63 is used to calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, it determines the best candidate rule subset and the next round of iteration candidate rule subset. Each rule in the best candidate rule subset is added to the initial mining rule set to obtain the updated rule set corresponding to each rule.
[0119] The rule mining and update module 64 is used to calculate the profit improvement of each updated rule set compared with the initial mining rule set, determine the rule corresponding to the updated rule set with the highest profit improvement as the best rule in this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule in the candidate best rule subset to the candidate rule subset of the next iteration to obtain the updated candidate rule subset of the next iteration.
[0120] The iterative loop module 65 is used to perform an extended search with the updated subset of candidate rules for the next iteration in the next iteration, obtain extended rules, use the extended rules as candidate rules, and use the updated mining rule set as the initial mining rule set. It then returns to calculate the credibility of each candidate rule until the number of rules in the updated mining rule set reaches a preset number or a preset number of iterations, thus obtaining the updated mining rule set.
[0121] Optionally, the rule filtering module 63 includes:
[0122] The first classification unit is used to detect whether the credibility of each candidate rule is greater than the credibility threshold. If the credibility of a candidate rule is not greater than the credibility threshold, the candidate rule is classified into the subset of candidate rules for the next round of iteration.
[0123] The second classification unit is used to classify a candidate rule as a candidate best rule if the confidence of a candidate rule is detected to be greater than the confidence threshold. For the first and second candidate best rules among all candidate best rules, if the lower bound of the revenue score corresponding to the first candidate best rule is detected to be greater than the upper bound of the revenue score corresponding to the second candidate best rule, the second candidate best rule is classified into the subset of candidate rules for the next round of iteration.
[0124] The third classification unit is used to classify both the first and second candidate best rules into a subset of candidate best rules if an intersection is detected between the upper and lower bounds of the profit score corresponding to the first candidate best rule and the upper and lower bounds of the profit score corresponding to the second candidate best rule.
[0125] Optionally, the excavation device also includes:
[0126] The remaining quota acquisition module is used to obtain the remaining quota of the initial mining rule set after calculating the credibility of each candidate rule. The remaining quota is the difference between the preset number of rules and the current number of rules in the initial mining rule set.
[0127] The sorting module is used to sort all candidate rules from high to low according to the lower bound of the benefit score, and determine the candidate rule whose ranking corresponds to the value of the remaining quota as the lower bound rule;
[0128] The rule exclusion module is used to classify candidate rules whose upper bound of the profit score is less than the lower bound of the profit score of the lower bound rule into an exclusion rule subset. All rules in the exclusion rule subset are removed from all rounds of iteration.
[0129] Optionally, the rule mining and update module 64 includes:
[0130] The profit score calculation unit is used to calculate the first profit score of the initial mining rule set and the second profit score of each updated rule set using the objective function.
[0131] The revenue improvement calculation unit is used to subtract each second revenue score from the first revenue score to determine the revenue improvement of each updated rule set compared to the initial mining rule set.
[0132] Alternatively, the objective function can be as follows:
[0133] In the formula, Σ represents the initial mining rule set or the update rule set, and λ∈[0,1]. Characterizing the rules within the set Σ, δ div (Σ) represents the computation of a measure of the diversity of rules within set Σ. Representation of rules Calculate the correlation measure with known rules.
[0134] Optionally, the upper and lower bound calculation module 62 includes:
[0135] The candidate score calculation unit is used to calculate the benefit score of any candidate rule based on preset relevance and diversity metrics.
[0136] The rule expansion unit is used to expand the candidate rules based on the remaining candidate predicates in the candidate predicate set other than the candidate predicates corresponding to the candidate rules, so as to obtain the expanded rules.
[0137] An extended score calculation unit is used to calculate the benefit score of each extended rule based on preset relevance and diversity metrics.
[0138] The upper and lower bound calculation unit is used to obtain the upper bound and lower bound of the revenue score of the candidate rule based on the revenue score of the candidate rule and the rules derived from its extension. It then iterates through all candidate rules to obtain the upper bound and lower bound of the revenue score for each candidate rule.
[0139] Optionally, the excavation device also includes:
[0140] The extended revenue score determination module is used to perform an extended search on the updated subset of candidate rules for the next round of iterations. After obtaining extended rules, for any extended rule, it determines the candidate rules for the extended rules and uses the upper and lower bounds of the revenue scores corresponding to the candidate rules as the upper and lower bounds of the revenue scores of the extended rules.
[0141] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.
[0142] In one embodiment, a computer device, which may be a server, is provided, and its internal structure can be as shown in Figure 7. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a readable storage medium, and the database. The internal memory provides an environment for the operation of the operating system and the readable storage medium. The database stores raw user data. The network interface communicates with external terminals via a network connection. When executed by the processor, the readable storage medium is used to implement a method for mining entity-enhanced rules.
[0143] In one embodiment, a computer device is provided, including a memory, a processor, and a readable storage medium stored on the memory and executable on the processor. When the processor executes the readable storage medium, it implements the steps of the entity augmentation rule mining method in the above embodiments, such as steps S201-S205 shown in FIG2, or the steps shown in FIG3 to FIG5. To avoid repetition, these steps will not be described again here. Alternatively, when the processor executes the readable storage medium, it implements the functions of each module / unit in this embodiment of the user data processing device, such as the functions of the candidate rule construction module 61, the upper and lower bound calculation module 62, the rule selection module 63, the rule mining and updating module 64, and the loop iteration module 65 shown in FIG6. To avoid repetition, these functions will not be described again here.
[0144] In one embodiment, one or more readable storage media storing computer-readable instructions are provided. When executed by one or more processors, these computer-readable instructions cause the processors to perform the steps of the entity augmentation rule mining method described in the above embodiments, such as steps S201-S205 shown in FIG2, or the steps shown in FIG3 to 5. To avoid repetition, these steps will not be described again here. Alternatively, when the processor executes the readable storage medium, it implements the functions of various modules / units in this embodiment of the user data processing device, such as the candidate rule construction module 61, upper and lower bound calculation module 62, rule selection module 63, rule mining and updating module 64, and loop iteration module 65 shown in FIG6. To avoid repetition, these modules will not be described again here. The readable storage medium in this embodiment includes non-volatile readable storage media and volatile readable storage media.
[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by instructing related hardware through a readable storage medium. The readable storage medium can be stored in a non-volatile computer-readable storage medium, which, when executed, can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0147] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for mining entity augmentation rules, wherein, The mining method includes: Initialize the initial mining rule set and the used predicate set as empty sets. Merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate. Use all predicates in the used predicate set of each candidate predicate and combine them with the rule objectives of the known rules to construct the corresponding candidate rule. Based on the preset relevance and diversity metrics, each candidate rule and its extended rules are scored for their returns, resulting in an upper bound and a lower bound for the return score of each candidate rule. Calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, determine the subset of the best candidate rules and the subset of candidate rules for the next round of iteration. Add each rule in the subset of the best candidate rules to the initial mining rule set to obtain the updated rule set corresponding to each rule. Calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration. In the next iteration, an extended search is performed using the updated subset of candidate rules for the next iteration to obtain extended rules. These extended rules are then used as candidate rules, and the updated set of mining rules is used as the initial set of mining rules. The credibility of each candidate rule is calculated and the process continues until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining the updated set of mining rules.
2. The excavation method according to claim 1, wherein, The process of determining the optimal subset of candidate rules and the subset of candidate rules for the next iteration based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule includes: Detect whether the credibility of each candidate rule is greater than the credibility threshold. If the credibility of a candidate rule is not greater than the credibility threshold, then the candidate rule is classified into the subset of candidate rules for the next round of iteration. If the credibility of a candidate rule is detected to be greater than the credibility threshold, then the candidate rule is taken as the candidate best rule. Among all the candidate best rules, if the lower bound of the revenue score corresponding to the first candidate best rule is detected to be greater than the upper bound of the revenue score corresponding to the second candidate best rule, then the second candidate best rule is assigned to the subset of candidate rules for the next round of iteration. If an intersection is detected between the upper and lower bounds of the profit score corresponding to the first candidate best rule and the upper and lower bounds of the profit score corresponding to the second candidate best rule, then both the first candidate best rule and the second candidate best rule are classified as subsets of candidate best rules.
3. The excavation method according to claim 1, wherein, After calculating the confidence level of each candidate rule, the following is also included: Obtain the remaining quota of the initial mining rule set, wherein the remaining quota is the difference between the preset number of rules and the current number of rules in the initial mining rule set; All candidate rules are sorted from high to low according to the lower bound of the benefit score, and the candidate rule whose ranking corresponds to the value of the remaining quota is determined as the lower bound rule. Candidate rules whose upper bound of profit score is less than the lower bound of the profit score of the lower bound rule are classified into an exclusion rule subset, and all rules in the exclusion rule subset are removed from all rounds of iteration.
4. The excavation method according to claim 1, wherein, The calculation of the revenue improvement of each updated rule set compared to the initial mining rule set includes: Using the objective function, the first revenue score of the initial mining rule set and the second revenue score of each updated rule set are calculated. The difference between each second profit score and the first profit score is calculated to determine the profit increase of each updated rule set compared to the initial mining rule set.
5. The excavation method according to claim 4, wherein, The objective function is as follows: In the formula, Σ represents the initial mining rule set or the update rule set, and λ∈[0,1]. Characterizing the rules within the set Σ, δ div (Σ) represents the computation of a measure of the diversity of rules within set Σ. Representation of rules The correlation metric with the known rules is calculated.
6. The excavation method according to claim 1, wherein, The step involves scoring the revenue of each candidate rule and its extended rules based on preset relevance and diversity metrics, thereby obtaining an upper bound and a lower bound for the revenue score of each candidate rule, including: For any candidate rule, the benefit score of the candidate rule is calculated based on the preset relevance metric and the preset diversity metric. Based on the remaining candidate predicates in the candidate predicate set other than the candidate predicates corresponding to the candidate rules, the candidate rules are expanded to obtain expanded rules; Based on the preset relevance and diversity metrics, the benefit score for each extended rule is calculated. Based on the revenue scores of the candidate rules and the rules derived from their extensions, the upper and lower bounds of the revenue scores of the candidate rules are obtained. By traversing all candidate rules, the upper and lower bounds of the revenue scores corresponding to each candidate rule are obtained.
7. The excavation method according to any one of claims 1 to 6, wherein, After performing an expanded search using the updated subset of candidate rules for the next iteration to obtain expanded rules, the process further includes: For any extended rule, determine the candidate rules for the extended rule, and use the upper and lower bounds of the revenue score corresponding to the candidate rules as the upper and lower bounds of the revenue score of the extended rule.
8. A mining device for entity augmentation rules, wherein, The excavation device includes: The candidate rule construction module is used to initialize the initial mining rule set and the used predicate set as empty sets, merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate, and use all predicates in the used predicate set of each candidate predicate, combined with the rule objectives of the known rules, to construct the corresponding candidate rule; The upper and lower bound calculation module is used to score the revenue of each candidate rule and its extended rules based on the preset relevance and diversity metrics, and to obtain the upper bound and lower bound of the revenue score for each candidate rule. The rule filtering module is used to calculate the credibility of each candidate rule, and determine the best candidate rule subset and the next round of iteration candidate rule subset based on the upper bound of the profit score, the lower bound of the profit score and the credibility of each candidate rule. Each rule in the best candidate rule subset is added to the initial mining rule set to obtain the updated rule set corresponding to each rule. The rule mining and update module is used to calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration. The iterative loop module is used to perform an expanded search on the updated subset of candidate rules for the next iteration in the next iteration to obtain expanded rules, use the expanded rules as candidate rules, and use the updated set of mining rules as the initial set of mining rules. It then returns to calculate the credibility of each candidate rule until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining an updated set of mining rules.
9. A computer device comprising a memory, a processor, and a readable storage medium stored in the memory and operable on the processor, wherein, When the processor executes the readable storage medium, it performs the following steps: Initialize the initial mining rule set and the used predicate set as empty sets. Merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate. Use all predicates in the used predicate set of each candidate predicate and combine them with the rule objectives of the known rules to construct the corresponding candidate rule. Based on the preset relevance and diversity metrics, each candidate rule and its extended rules are scored for their returns, resulting in an upper bound and a lower bound for the return score of each candidate rule. Calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, determine the subset of the best candidate rules and the subset of candidate rules for the next round of iteration. Add each rule in the subset of the best candidate rules to the initial mining rule set to obtain the updated rule set corresponding to each rule. Calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration. In the next iteration, an extended search is performed using the updated subset of candidate rules for the next iteration to obtain extended rules. These extended rules are then used as candidate rules, and the updated set of mining rules is used as the initial set of mining rules. The credibility of each candidate rule is calculated and the process continues until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining the updated set of mining rules.
10. The computer device according to claim 9, wherein, The process of determining the optimal subset of candidate rules and the subset of candidate rules for the next iteration based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule includes: Detect whether the credibility of each candidate rule is greater than the credibility threshold. If the credibility of a candidate rule is not greater than the credibility threshold, then the candidate rule is classified into the subset of candidate rules for the next round of iteration. If the credibility of a candidate rule is detected to be greater than the credibility threshold, then the candidate rule is taken as the candidate best rule. Among all the candidate best rules, if the lower bound of the revenue score corresponding to the first candidate best rule is detected to be greater than the upper bound of the revenue score corresponding to the second candidate best rule, then the second candidate best rule is assigned to the subset of candidate rules for the next round of iteration. If an intersection is detected between the upper and lower bounds of the profit score corresponding to the first candidate best rule and the upper and lower bounds of the profit score corresponding to the second candidate best rule, then both the first candidate best rule and the second candidate best rule are classified as subsets of candidate best rules.
11. The computer device according to claim 9, wherein, After calculating the confidence level of each candidate rule, the following is also included: Obtain the remaining quota of the initial mining rule set, wherein the remaining quota is the difference between the preset number of rules and the current number of rules in the initial mining rule set; All candidate rules are sorted from high to low according to the lower bound of the benefit score, and the candidate rule whose ranking corresponds to the value of the remaining quota is determined as the lower bound rule. Candidate rules whose upper bound of profit score is less than the lower bound of the profit score of the lower bound rule are classified into an exclusion rule subset, and all rules in the exclusion rule subset are removed from all rounds of iteration.
12. The computer device according to claim 9, wherein, The calculation of the revenue improvement of each updated rule set compared to the initial mining rule set includes: Using the objective function, the first revenue score of the initial mining rule set and the second revenue score of each updated rule set are calculated. The difference between each second profit score and the first profit score is calculated to determine the profit increase of each updated rule set compared to the initial mining rule set.
13. The computer device according to claim 12, wherein, The objective function is as follows: In the formula, Σ represents the initial mining rule set or the update rule set, and λ∈[0,1]. Representation set Σ Internal rules, δ div (Σ) represents the computation of a measure of the diversity of rules within set Σ. Representation of rules The correlation metric with the known rules is calculated.
14. The computer device according to claim 9, wherein, The step involves scoring the revenue of each candidate rule and its extended rules based on preset relevance and diversity metrics, thereby obtaining an upper bound and a lower bound for the revenue score of each candidate rule, including: For any candidate rule, the benefit score of the candidate rule is calculated based on the preset relevance metric and the preset diversity metric. Based on the remaining candidate predicates in the candidate predicate set other than the candidate predicates corresponding to the candidate rules, the candidate rules are expanded to obtain expanded rules; Based on the preset relevance and diversity metrics, the benefit score for each extended rule is calculated. Based on the revenue scores of the candidate rules and the rules derived from their extensions, the upper and lower bounds of the revenue scores of the candidate rules are obtained. By traversing all candidate rules, the upper and lower bounds of the revenue scores corresponding to each candidate rule are obtained.
15. The computer device according to any one of claims 9 to 14, wherein, After performing an expanded search using the updated subset of candidate rules for the next iteration to obtain expanded rules, the process further includes: For any extended rule, determine the candidate rules for the extended rule, and use the upper and lower bounds of the revenue score corresponding to the candidate rules as the upper and lower bounds of the revenue score of the extended rule.
16. One or more readable storage media storing computer-readable instructions, wherein, When the computer-readable instructions are executed by one or more processors, the one or more processors cause the processors to perform the following steps: Initialize the initial mining rule set and the used predicate set as empty sets. Merge each candidate predicate in the candidate predicate set with the used predicate set to obtain the used predicate set of the corresponding candidate predicate. Use all predicates in the used predicate set of each candidate predicate and combine them with the rule objectives of the known rules to construct the corresponding candidate rule. Based on the preset relevance and diversity metrics, each candidate rule and its extended rules are scored for their returns, resulting in an upper bound and a lower bound for the return score of each candidate rule. Calculate the credibility of each candidate rule. Based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule, determine the subset of the best candidate rules and the subset of candidate rules for the next round of iteration. Add each rule in the subset of the best candidate rules to the initial mining rule set to obtain the updated rule set corresponding to each rule. Calculate the revenue improvement of each updated rule set compared to the initial mining rule set, determine the rule corresponding to the updated rule set with the highest revenue improvement as the best rule for this iteration, add the best rule to the initial mining rule set to obtain the updated mining rule set, and add the rules other than the best rule from the candidate best rule subset to the candidate rule subset for the next iteration to obtain the updated candidate rule subset for the next iteration. In the next iteration, an extended search is performed using the updated subset of candidate rules for the next iteration to obtain extended rules. These extended rules are then used as candidate rules, and the updated set of mining rules is used as the initial set of mining rules. The credibility of each candidate rule is calculated and the process continues until the number of rules in the updated set of mining rules reaches a preset number or a preset number of iterations, thus obtaining the updated set of mining rules.
17. The readable storage medium according to claim 16, wherein, The process of determining the optimal subset of candidate rules and the subset of candidate rules for the next iteration based on the upper bound of the profit score, the lower bound of the profit score, and the credibility of each candidate rule includes: Detect whether the credibility of each candidate rule is greater than the credibility threshold. If the credibility of a candidate rule is not greater than the credibility threshold, then the candidate rule is classified into the subset of candidate rules for the next round of iteration. If the confidence level of a candidate rule is detected to be greater than the confidence threshold, then the candidate rule is selected as the candidate best rule. Among all candidate best rules, if the lower bound of the revenue score corresponding to the first candidate best rule is greater than the upper bound of the revenue score corresponding to the second candidate best rule, then... The second candidate best rule is assigned to the subset of candidate rules for the next round of iteration; If an intersection is detected between the upper and lower bounds of the profit score corresponding to the first candidate best rule and the upper and lower bounds of the profit score corresponding to the second candidate best rule, then both the first candidate best rule and the second candidate best rule are classified as subsets of candidate best rules.
18. The readable storage medium according to claim 16, wherein, After calculating the confidence level of each candidate rule, the following is also included: Obtain the remaining quota of the initial mining rule set, wherein the remaining quota is the difference between the preset number of rules and the current number of rules in the initial mining rule set; All candidate rules are sorted from high to low according to the lower bound of the benefit score, and the candidate rule whose ranking corresponds to the value of the remaining quota is determined as the lower bound rule. Candidate rules whose upper bound of profit score is less than the lower bound of the profit score of the lower bound rule are classified into an exclusion rule subset, and all rules in the exclusion rule subset are removed from all rounds of iteration.
19. The readable storage medium according to claim 16, wherein, The calculation of the revenue improvement of each updated rule set compared to the initial mining rule set includes: Using the objective function, the first revenue score of the initial mining rule set and the second revenue score of each updated rule set are calculated. The difference between each second profit score and the first profit score is calculated to determine the profit increase of each updated rule set compared to the initial mining rule set.
20. The readable storage medium according to claim 19, wherein, The objective function is as follows: In the formula, Σ represents the initial mining rule set or the update rule set, and λ∈[0,1]. Characterizing the rules within the set Σ, δ div (Σ) represents the computation of a measure of the diversity of rules within set Σ. Representation of rules The correlation metric with the known rules is calculated.
Citation Information
Patent Citations
Redundancy rule screening method and device for association rule mining
CN112434104A
Data processing method and device based on relational data
CN115033650A
Entity enhancement rule mining method and device, computer equipment and medium
CN116663663A
Data screening method and apparatus based on reinforcement learning
WO2023236240A1