A Method and System for Mining Rules for Electricity Pricing Consistency Verification

Abstract

This invention belongs to the field of power system technology, specifically relating to a method and system for mining rules for electricity price and fee consistency verification. The method includes: acquiring a verification dataset and calculating the equivalence partitions of quantity, price, and fee data; using a depth-first spatial search strategy to mine data consistency relationships, obtaining a set of consistency relationships in the data; using the obtained set of consistency relationships to perform rule parsing on the quantity, price, and fee consistency relationships; and identifying rare data in the quantity, price, and fee data through rule parsing. This invention applies a novel consistency rule mining method for categorized data to the verification of electricity price and fee data. It can proactively mine potential rules in the data, effectively overcoming the shortcomings of previous manual rule definition methods, such as high workload, long time consumption, insufficient rule coverage, and the ability to only discover abnormal business data covered by defined rules. This assists business personnel in completing the verification of electricity marketing quantity, price, and fee data more intelligently and comprehensively.

Description

Technical Field

[0001] This invention belongs to the field of power system technology, specifically relating to a method and system for mining rules for verifying the consistency of electricity pricing and fees. Background Technology

[0002] Electricity customer service is one of the core businesses of power operations and a key system for power companies to serve their customers. The core business of electricity volume and price (referred to as volume-based pricing and fees) is crucial to the safe operation of power companies and the vital interests of a large number of electricity customers. To improve the efficiency of volume-based pricing and fees operations, an important task of the electricity customer service system is to verify the core data of these transactions. A key foundational task of this work is to define verification rules based on the characteristics of volume-based pricing and fees operations, and then, based on these rules, conduct compliance verification of the business logic constraints (business consistency) of electricity data. This identifies data that violates consistency rules and allows for manual review and correction, preventing volume-based pricing and fees-related issues from occurring.

[0003] In traditional quantity, price, and fee verification, data consistency checks are currently mainly conducted by manually defining rules to identify business data that violates business consistency rules. This method is difficult to achieve full rule coverage and suffers from problems such as a large workload in rule definition and delayed response to abnormal data.

[0004] To address these issues, the business has previously employed several machine learning-based association rule mining algorithms, such as the commonly used Apriori or FP-Growth frequent set mining algorithms. However, these algorithms generate redundant rule results, posing challenges for organizing, reviewing, and utilizing these results. Furthermore, they can only identify association relationships, not the dependencies between left and right items.

[0005] In the past, some AI-based outlier detection algorithms have been used in power data verification systems. While these algorithms can detect abnormal data that deviates from group behavior, their results often lack interpretability. This makes it difficult for business personnel to verify whether the "abnormal data" identified by the algorithm is truly abnormal, posing potential risks. Therefore, they are hesitant to apply them on a large scale in power data compliance checks. Furthermore, AI-based anomaly detection algorithms often rely on manually defined thresholds, which carries the risk of inappropriate parameter settings. If the threshold is set too low, it may fail to detect or miss too many abnormal data points; if it is set too high, it may lead to an increase in false alarms. Summary of the Invention

[0006] To overcome the problems in the prior art, this invention proposes a method and system for mining rules for verifying the consistency of electricity pricing.

[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0008] A method for mining rules for electricity pricing consistency verification includes the following steps:

[0009] Step 1. Obtain the verification dataset and calculate the equivalent partitions of quantity, price, and fee data;

[0010] Step 2. Use a depth-first spatial search strategy to mine the consistency relationships between quantity, price, and fee data to obtain the minimum complete set of consistency relationships in the data;

[0011] Step 3. Use the obtained minimal complete consistency relation set to perform rule parsing on the quantity-price-fee consistency relation;

[0012] Step 4. Based on the rule parsing process, identify rare data in the overall quantity, price, and fee data.

[0013] Further, the calculation of the equivalent partitioning of quantity, price, and fee data in step 1 includes:

[0014] For a given set of verification data R, let attr(R) be the set of attributes of its categorical attributes;

[0015] Verify the consistency between field combinations X and individual fields Y in a dataset R. Notation: X→Y;

[0016] in, This is called a consistency relation. The left-hand side (LHS) is the condition part of the consistency rule; Y∈attr(R) is called the consistency relation. The right-hand side (RHS) is the result part of the consistency rule;

[0017] For a combination of fields X on a dataset R, let t∈R be a data record on R;

[0018] Define [t] X Let [t] be an equivalence class on X. X It contains the indexes of all data in the dataset R that have the same value as t in field X; the set of all equivalence classes in X is denoted as π. X ={[t] X |t∈R} is an equivalent partition on X.

[0019] Furthermore, step 2 uses a depth-first spatial search strategy to mine potential consistency relationships between quantity, price, and fee data, specifically including the following sub-steps:

[0020] For the search layer Perform initialization and define The "-" symbol represents any possible value for a field A in R;

[0021] For any combination of fields Calculate its equivalent partition π X and π X∪A ;

[0022] Constructing a search tree For any given verification dataset R, compute its difference set. Where D(t1, t2) = A∈attr(R)|t1[A]≠t2[A]}, the difference set Add to search tree In the leaf nodes;

[0023] when When not empty, for the search tree Given a consistency relation, the left-hand side LHS = X and the right-hand side RHS = A, determine whether the consistency relation is valid. The determination criteria are as follows:

[0024] If |π X |=|π X∪A |, then consistency relation X→A holds true;

[0025] In the formula, |π X | represents the equivalent partition π X The number of elements in a set, that is, when the number of equivalence classes of field combination X and X∪A in quantity, price and fee data are equal, there is a valid data consistency relationship between field X→A;

[0026] If the current search layer Stop the spatial search algorithm and check that all consistent relations on dataset R have been found; if the current search layer After pruning redundant conditions, the next search layer is calculated. Where l represents the current search layer. The depth of X is a combination of fields in R. Methods for pruning redundant conditions will be given in subsequent steps.

[0027] Furthermore, step 2 uses a depth-first spatial search strategy to mine potential consistency relationships between quantity, price, and fee data, and specifically includes the following sub-steps:

[0028] In the rule mining and detection process, support and confidence parameters are introduced. If the support and confidence threshold conditions are met, then the business consistency relationship on the verification dataset is approximately established.

[0029] Redundant conditions in the aforementioned business consistency relationship are pruned;

[0030] Verify all consistency relations on the dataset to obtain the minimum complete set of consistency relations on the dataset.

[0031] Furthermore, in step 2, during the rule mining and detection process, support and confidence parameters are introduced. If the support and confidence threshold conditions are met, then an approximate business consistency relationship is established on the verification dataset. Specifically, this includes:

[0032] The classification error rate of the consistency relationship is denoted as e(X→A). The formula for calculating the classification error rate of the consistency relationship between quantity, price, and fee data is defined as follows:

[0033]

[0034] Let the confidence level of a consistency relation be . Based on the calculation result of e(X→A), the confidence level of this consistency relationship can be obtained as follows:

[0035]

[0036] Among them, the equivalent partition π of field combination X X ={c1,c2,…,c p}, where c1, c2, ..., c p Let X be the p equivalence classes. Similarly, define the equivalence partition π of field A. X∪A ={c1′,c2′,…,c p Let the total amount of data verified be |r|.

[0037] Record the consistency relationship between field combination X and field A in dataset R. The support rate is Its value is defined as all values ​​on R that satisfy this relation. The number of data points;

[0038] Presetting minimum support δ and minimum confidence ε will satisfy the threshold condition. and A relationship is called an approximately true consistency relation on the dataset R, satisfying this consistency relation. The quantity, price, and fee data are positive examples, while data that does not satisfy this consistency relationship are negative examples.

[0039] Furthermore, step 2 involves pruning redundant conditions in the business consistency relationship, including:

[0040] For each field combination X, construct a candidate generation set C of the right-hand side RHS of the consistent relation. + (X):

[0041]

[0042] Where B is any field in the field combination X, and X\{A, B} represents the difference set obtained by removing fields A and B from set X;

[0043] If the consistency relation X ∈ {A} → A holds, then A and all B ∈ C will be included. + (X)\X from C + Remove from (X);

[0044] like Then field combination X is selected from the current search layer. The branches were pruned in the middle;

[0045] If field combination X is a key in the verification dataset R, then field combination X is selected from the current search layer. The branches were pruned.

[0046] Furthermore, obtaining the minimum complete and consistent relation set includes:

[0047] If in the current search layer Once the depth-first search is complete, all consistency relations on the verification dataset R have been identified, resulting in the minimum complete set of consistency relations on the verification dataset R.

[0048] Minimal complete consistent relation set Satisfy: For any set of consistency relations A certain consistency relation If X→A, removing any field from X will invalidate the consistency relationship; from the set of consistency relationships... In this process, it is possible to derive all other consistency relations on the verification dataset R, that is... closure For any consistency relation Remove the set of relationships Incomplete.

[0049] Furthermore, the parsing content in step 3 includes: the value of the condition part, the positive / negative example value of the right-hand side of the rule, the proportion of the rule with the value in the overall data, the confidence of the rule, and the number of negative examples of the rule.

[0050] Furthermore, in step 4, based on the rule parsing process, rare data in the overall quantity, price, and fee data are identified, including: based on the proportion of positive / negative examples of various values ​​of quantity, price, and fee data in the overall data in the rule parsing table, as well as parameter information such as the confidence level of specific values ​​of the rules, rare data with extremely small quantity and extremely low proportion in the overall quantity, price, and fee data are automatically parsed through parameter sorting and other methods.

[0051] This invention also discloses an electricity pricing and fee consistency verification rule mining system, including: a quantity pricing and fee data management module, a quantity pricing and fee consistency rule mining module, an intelligent rule mining result display module, a rule mining result analysis and management module, a rare business data identification and analysis module, a rule execution module, and a quantity pricing and fee verification result management module;

[0052] The quantity, price, and fee data management module is used to manage the sources and standards of quantity, price, and fee data required by the data verification system, and to obtain the required business data from the corresponding data sources.

[0053] The consistency rule mining module for volume, price, and fee is used to execute the consistency rule mining algorithm in the volume, price, and fee archive data according to the algorithm configuration and relevant input from business personnel, output relevant rule results, and automatically parse the rules. According to the distribution pattern of volume, price, and fee business data, it provides information such as the value, error rate, positive and negative examples, and their respective quantities and proportions for each specific rule.

[0054] The rule mining result analysis and management module allows experts to determine the usability and effectiveness of a rule based on the detailed analysis of quantity, price, and fee rule examples and the consistency relationships between different field values ​​displayed in the rule display module. This is achieved through an understanding of electricity pricing and fee business and the interpretation of positive and negative examples of the rule. This module provides unified management of effective rules, allowing business experts to set rules to be valid or invalid according to business needs, thereby generating a rule model.

[0055] The rare business data identification and analysis module is used to highlight rare business data that account for a very small proportion and are very few in number in the volume, price and fee marketing archive data according to the results obtained from rule parsing, so that business experts can review and analyze them.

[0056] The rule execution module is used to execute rules on the same field but different data in the quantity, price and fee business dataset based on the rule model modified by experts. The system will highlight abnormal data that violates the existing consistency rule model to assist business personnel in data consistency verification.

[0057] The quantity, price, and fee verification result management module is used to perform management operations such as adding, deleting, modifying, and querying the quantity, price, and fee data verification results, as well as storing the relevant verification results in the database.

[0058] Compared with the prior art, the present invention has the following technical effects:

[0059] (1) This invention applies a new consistency rule mining method for categorized data to the verification of electricity price and fee data. It can proactively mine potential rules in the data, effectively making up for the shortcomings of previous manual rule definition, such as large workload, long time consumption, insufficient rule coverage, and only being able to discover abnormal business data covered by the defined rules. It assists business personnel to complete the verification of electricity marketing volume, price and fee data more intelligently and comprehensively.

[0060] (2) In the process of mining the consistency rules of quantity, price and fee, this invention can not only effectively mine general patterns, but also identify rare data in the process of rule mining. These rare data are likely to be anomalies or rare phenomena in business, which are worth paying attention to; this is something that cannot be done in the previous process of manually defining rules.

[0061] (3) The present invention provides automatic parsing of rule instances, and provides information such as the positive / negative example values ​​of the rule, the number of positive / negative example values, the proportion of positive / negative example values ​​in the overall data, and the rule confidence, so that the rule results have good interpretability. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the method for mining rules to verify the consistency of electricity pricing and fees according to the present invention.

[0063] Figure 2 This is a schematic diagram of the overall structure of the automatic mining system for electricity price and fee consistency verification rules of the present invention. Detailed Implementation

[0064] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0065] In the traditional quantity, price, and fee verification rules, business experts define the following rule: if the "electricity type" is "urban and rural residential electricity use", then the "transformer loss charge mark" should be 0 (no charge).

[0066] For the above rules: (If the "Electricity Type" is "Urban and Rural Residential Electricity") is the condition part of the rule, also known as the left-hand side of the rule; (Then the "Transformer Loss Charging Identifier" should be 0 (no charge)) is the conclusion part of the rule, also known as the right-hand side of the rule; "Electricity Type" and "Transformer Loss Charging Identifier" are category-specific quantity-price-fee related attributes (fields), and "Urban and Rural Residential Electricity" and "0" are the values ​​corresponding to the respective category attributes; changes in the value of the "Transformer Loss Charging Identifier" directly affect the calculation results of quantity-price-fee for electricity users and the compliance of the business: for the left-hand side of the rule "Electricity Type" being "Urban and Rural Residential Electricity," if its "Transformer Loss Charging Identifier" is not 0, then the current user's quantity-price-fee calculation result does not comply with the business regulations. Therefore, it is necessary to identify rules that violate consistency constraints during the verification process.

[0067] The data related to electricity and customer service involves many classification attribute fields in the quantity, price and fee data. For example, there is a relationship between the classification attribute "electricity industry category" and the classification attribute "electricity price category". There is also a rule as follows: if the "electricity industry category" is "religious activity places such as temples, palaces, mosques, etc". then the "electricity price category" should be "residential electricity price".

[0068] Due to insufficient operational experience, inadequate understanding of pricing and quantity-related policies, or operational errors, some unreasonable data exists in the current power system. For example, during a business investigation, a data point was found where the "Electricity Industry Category" was "Religious Activity Sites such as Temples, Palaces, and Mosques," while the "Electricity Price Category" was "Non-Industrial Electricity Price," clearly violating relevant policy regulations. To identify these issues, the common practice in the power system is to organize business experts to compile rules and then use information systems to investigate the problems.

[0069] However, the power system involves numerous categorized attribute fields in its quantity, price, and fee data, and each categorized attribute has many possible values. For example, the attribute "electricity industry category" has nearly a thousand possible values, while the categorized attribute "electricity price category," which has a business consistency relationship with it, has nearly a hundred possible values. This can result in tens of thousands of possible quantity, price, and fee verification rules. If business experts were to meticulously review every single one, on the one hand, manually defining the rules would consume a significant amount of their energy and time, leading to extremely low efficiency; on the other hand, business experts ultimately have limited energy and cannot possibly develop all the rules perfectly. Therefore, incomplete verification rules inevitably result in omissions and errors.

[0070] To improve the efficiency of rule formulation for consistency verification of categorized data related to electricity customer volume, price, and fees, and to enhance the data consistency verification capabilities for volume, price, and fees business, this invention proposes a consistency rule mining method for categorized data related to electricity volume, price, and fees verification. Utilizing unsupervised machine learning, this method automatically mines consistency verification rules for electricity volume, price, and fees data, proactively discovering potential rules from massive datasets and detecting data inconsistencies. This provides an intelligent decision-making support tool for electricity volume, price, and fees verification work, going beyond traditional manual rule definitions based on experience. Furthermore, in addition to discovering consistency rules between categorized data, this invention can also provide business personnel with detailed rule examples, data error rates, and positive and negative examples. Simultaneously, it can automatically identify rare data and output it to business experts for their interpretation of whether it conforms to business constraints, enabling timely detection and response to rare business information.

[0071] This invention proposes a method for mining electricity pricing and fee consistency verification rules. Based on business consistency relationships, it achieves field relationship analysis, credibility analysis, and verification rule generation among quantity, price, and fee-related data. The method includes the following steps:

[0072] Step 1. Obtain quantity, price, and fee data, and calculate the equivalent divisions on the quantity, price, and fee data;

[0073] For a given set of verification data R, let attr(R) be the set of attributes of its categorical attributes.

[0074] Business consistency relationships can be defined as the consistency relationships between a combination of fields X and a single field Y in R. To describe, denoted as: X→Y means that the value of field Y can be completely determined by the combination of fields X.

[0075] in, This is called a consistency relation. The left-hand side (LHS) is the condition part of the consistency rule; Y∈attr(R) is called the consistency relation. The right-hand side (RHS) is the result part of the consistency rule.

[0076] For any two tuples t α ,t β ∈R, denoted by the symbol t α [X] indicates that field X is in data tuple t α The specific values ​​that R can take. Then, if R satisfies the consistency relation... If and only if t holds α [X] = t β[X], then t must hold. α [Y] = t β [Y].

[0077] Obviously, This describes a consistency rule between field combination X and individual field Y in the verification dataset R. If the rule is approved by business personnel, it can be considered a valid rule. Therefore, if data in the quantity, price, and fee archive does not meet this consistency rule, it can be identified as abnormal data.

[0078] For a combination of fields X in a dataset R, let t ∈ R be a data record in R. Define [t]. X Let [t] be an equivalence class on X. X It contains the indexes of all data in the dataset R that have the same value as t in field X. The set of all equivalence classes in X is denoted as π. X ={[t] X |t∈R} is called an equivalence partition on X. Clearly, quantity, price, and fee data records in the same equivalence class have the same value in field X. An equivalence partition π is said to refine another equivalence partition π′ if and only if every equivalence class in π is a subset of some equivalence class in π′.

[0079] Table 1. Examples of partial data from the marketing archives on quantity, price, and cost.

[0080]

[0081] For example, Table 1 provides a partial data example from the quantity-price-fee marketing profile. The equivalence class for the field "Electricity Category Code" is:

[0082]

[500] [″用电类别代码″] ={1, 2, 3, 4},

[300] [″用电类别代码″] ={5},

[600] [″用电类别代码″]

[0083] ={6},

[260] [″用电类别代码″] ={7, 8}

[0084] Therefore, the equivalent partition on the "Electricity Category Code" field is calculated as follows:

[0085] π [″用电类别代码″] ={{1, 2, 3, 4}, {5}, {6}, {7, 8}}

[0086] Similarly, the equivalent partitions on the field combination ["electricity category code", "electricity price code"] can be calculated as follows:

[0087] π [″用电类别代码″，″电价代码″]= {{1,2},{3,4},{5},{6},{7},{8}}.

[0088] It is easy to see that, according to the above definition, π [″用电类别代码″，″电价代码″] It is π [″用电类别代码″] The details.

[0089] Step 2. Use a depth-first spatial search strategy to mine the consistency relationships between quantity, price, and fee data to obtain the minimum complete set of consistency relationships in the data;

[0090] First, for the search layer Perform initialization and define The "-" symbol represents any possible value for a field A in R;

[0091] For any combination of fields Calculate its equivalent partition π X and π X∪A .

[0092] Construct a prefix search tree For any given verification dataset R, compute its difference set. Where D(t1, t2)={A∈attr(R)|t1[A]≠t2[A]}, Add to search tree In the leaf nodes. When When not empty, for The consistency relationship between the LHS=X and RHS=A fields in the data is determined based on the following criteria:

[0093] If |π X |=|π X∪A |, then it holds true X→A.

[0094] In the formula, |π X | represents the equivalent partition π X The number of elements in a set, i.e., when the number of equivalence classes of field combination X and X∪A on quantity, price, and fee data are equal, there is a strictly valid data consistency relationship between field X and A.

[0095] For example, according to the description in step 1, let X = ["Electricity Category Code"] and A = ["Electricity Price Code"], then the equivalent partition on X is π. X ={{1, 2, 3, 4}, {5}, {6}, {7, 8}}, the equivalent partition on X∪A is π X∪A ={{1, 2}, {3, 4}, {5}, {6}, {7}, {8}}, |π X |≠|π X∪AThe consistency relationship between "electricity price code" and "electricity category code" is not strictly valid.

[0096] But in the same data example, π A = {{1, 2}, {3, 4}, {5}, {6}, {7}, {8}}, satisfying |π X |=|π X∪A This indicates that the consistency relationship between ("electricity price code" => "electricity category code") is strictly valid.

[0097] In the rule detection process, the parameters minimum support and minimum confidence are introduced. A relationship that satisfies the support and confidence threshold conditions is an approximately valid consistency relationship on the verification dataset.

[0098] Determine the consistency relationship between volume, price, and fee-related approximate businesses. This consistency relationship includes the number of data points in a certain consistency relationship and the classification accuracy of that relationship. The consistency relationship must satisfy the aforementioned approximate business consistency criteria. The quantity, price, and fee data are positive examples, while data that does not satisfy the aforementioned consistency relationship of similar businesses are negative examples.

[0099] Due to operator unfamiliarity with the business, lack of understanding of the consistency of quantity, price, and fee related business relationships, inadequate understanding of relevant policies and regulations, or operational errors, a small amount of non-compliant data exists in the basic business data related to quantity, price, and fees. For example, users classified as "agriculture, forestry, animal husbandry, and fishery" are charged non-residential electricity prices, and users classified as "temples" are charged non-residential or non-industrial electricity prices.

[0100] Secondly, the proposed method for mining consistency rules of quantity, price, and fee allows for a certain degree of fault tolerance. During the rule detection process, the parameters minimum support δ and minimum confidence ε are introduced.

[0101] Support represents the degree of consistency achieved on the dataset R. The number of data points from X to A is denoted as Confidence describes the consistency relationship The classification accuracy of X→A is denoted as .

[0102] Only when both conditions are met and Only when X→A is it considered a valid consistency relationship.

[0103] Suppose the equivalent partition π of field combination X X ={c1,c2,…,c p}, where c1, c2, ..., c p Let X be the p equivalence classes. Similarly, define the equivalence partition π of field A.X∪A ={c1′,c2′,…,c p Let the total amount of data to be checked be |r|, and the classification error rate of the consistency relationship X→A be denoted as e(X→A). Then, the formula for calculating the classification error rate of this consistency relationship of quantity, price, and fee data is defined as follows:

[0104]

[0105] Let the confidence level of the consistency relation X→A be . Based on the calculation result of e(X→A), the confidence level of this consistency relationship can be obtained as follows (this formula also defines the relationship between confidence level and classification error rate):

[0106]

[0107] Therefore, based on the above theory, and using artificially set threshold parameters, namely the minimum support δ and the minimum confidence ε, the threshold condition is defined. and Relationship To verify a consistency relation that is approximately true on a dataset R, we say that the consistency relation is satisfied. The quantity, price, and fee data are positive examples, while data that does not satisfy this consistency relationship are negative examples.

[0108] For a specific dataset, support is set by the number or percentage of records that meet specific conditions; confidence is the ratio of records that meet the left-hand condition and the right-hand condition to all records that meet the left-hand condition, and it is a decimal between 0 and 1, generally taken as 0.95 based on statistical experience (a higher value can be used if more stringent rules are required).

[0109] Suppose that out of 1 million data points, there are 200,000 records with electricity price code = 5001039, and 199,800 records with force regulation calculation method = 2; then for the rule: if electricity price code = 5001039, then force regulation calculation method = 2, the support is 200,000 / 1,000,000 = 20%, and the confidence level is = 0.999.

[0110] Then, redundant conditions in the business consistency relationship are pruned.

[0111] Since only the minimum regular expression coverage of the business consistency relationship is needed to cover the complete consistency constraints, redundant conditions need to be pruned during algorithm execution to improve algorithm efficiency while simplifying rule results.

[0112] To address this, we first construct an RHS candidate generator set C for each field combination X. + (X):

[0113]

[0114] Where B is any field in the field combination X, and X\{A, B} represents the difference set obtained by removing fields A and B from set X.

[0115] The specific pruning criteria are as follows:

[0116] Minimal pruning: If the consistency relation X ∈ {A} → A holds, then A and all B ∈ C can be pruned. + (X)\X from C + Remove from (X). In other words, if there exists X\{A}∈X such that the rule X\{A}→A holds, then any rule X→B cannot be the minimum consistency relation;

[0117] RHS pruning: If Then X can be searched from the current search layer. The RHS was pruned because there were no more RHS options available at that point.

[0118] Key pruning: If field X is a key in the verification dataset R, then X can be pruned from the current search layer. The data is pruned. Clearly, according to the definition of a key, a single field X can determine all values ​​in the dataset R that do not contain the field X. Therefore, all supersets of X are superkeys of R, which does not satisfy the principle of minimumity.

[0119] If the current search layer After pruning redundant conditions, the next search layer is calculated. Where l represents the current search layer. The depth, where X is a combination of fields on R; if the current search layer Once the depth-first search is complete, all consistency relations on the verification dataset R have been identified, resulting in the minimum complete set of consistency relations on the verification dataset R.

[0120] The meaning of "minimum completeness" is as follows:

[0121] Left-hand simplicity: For any set of terms belonging to the consistency relation A certain consistency relation If X→A is reached by removing any field from X, the consistency relationship is no longer valid.

[0122] Completeness: From a set of consistent relations In this process, it is possible to derive all other consistency relations on the verification dataset R, that is... closure Such a set of consistency relations is called It is complete;

[0123] Minimalism: For any consistent relation Remove the set of relationships Incomplete.

[0124] This set of consistent relationships will serve as the basis for subsequent rule parsing.

[0125] Step 3. Using the obtained minimal complete consistency relation set, perform rule parsing on the consistency relation of quantity, price, and fee on the business data instance;

[0126] Using the minimum set of complete and consistent relations on the verification dataset R Rule parsing is performed on the consistency relationship between quantity, price, and fee in business data instances. Since the left-hand side of the rule is the condition part of the consistency relationship and the right-hand side of the rule is the result part of the consistency relationship, for a specific left-hand side value, there is a unique "positive example" (i.e., the right-hand side value that satisfies the consistency relationship) and several possible "negative examples" (i.e., the right-hand side value that violates the consistency relationship).

[0127] Therefore, the rule parsing section includes the following information: the left-hand side value of the rule, the positive / negative example values ​​of the right-hand side of the rule, the proportion of the rule with that value in the overall data, the confidence level of the rule, and the number of negative examples of the rule. This further reveals the business constraint relationships between fields and can also provide specific negative example sample results.

[0128] For example, given the rule that the electricity price code is 5001039 and the power regulation calculation method is 2, if a certain data has the electricity price code 5001039 and the power regulation calculation method is 2, then it is a positive example; however, if a certain data has the electricity price code 5001039 and the power regulation calculation method is 3, then it is a negative example because it does not conform to the rule.

[0129] Step 4. Based on the parsed content, extract rare data that is extremely rare and accounts for a very small proportion of the overall quantity, price, and fee data;

[0130] Based on the proportion of positive / negative examples of various values ​​in the quantity, price, and fee data within the overall data in the rule parsing table, as well as parameters such as the confidence level of specific rule values, the system automatically identifies rare data points that are extremely few in number and represent a very low proportion of the overall quantity, price, and fee data through parameter sorting and other methods. These rare data points are provided for business experts' reference. These rare data points may contain outliers or may also imply business characteristics that have not yet been discovered by business personnel, making them worthy of their attention.

[0131] Embodiments of the present invention also provide an electricity pricing and fee consistency verification rule mining system, including: a quantity, price, and fee data management module, a quantity, price, and fee consistency rule mining module, an intelligent rule mining result display module, a rule mining result analysis and management module, a rare business data identification and analysis module, a rule execution module, and a quantity, price, and fee verification result management module.

[0132] The quantity, price, and fee data management module is used to manage the sources and standards of quantity, price, and fee data required by the data verification system, and to obtain the required business data from the corresponding data sources.

[0133] The consistency rule mining module for volume, price, and fee is used to execute the consistency rule mining algorithm in the volume, price, and fee archive data according to the algorithm configuration and relevant input from business personnel, output relevant rule results, and automatically parse the rules. According to the distribution pattern of volume, price, and fee business data, it provides information such as the value, error rate, positive and negative examples, and their respective quantities and proportions for each specific rule.

[0134] The intelligent rule mining result display module is used to display the algorithm execution results from the previous step and the rule model obtained from rule parsing to experts on the system front end through relevant pages.

[0135] The rule mining result analysis and management module allows experts to determine the usability and effectiveness of a rule based on their understanding of electricity pricing and fee business and the interpretation of positive and negative examples. This module provides unified management of effective rules, allowing business experts to set rules as valid or invalid according to business needs, thereby generating a rule model. Within this module, business personnel can review and modify the algorithm's output rule results based on actual business conditions. For example, they can add fields ("electricity price code" => "electricity category code") or swap the left and right sides of the data consistency relationship ("electricity price code" => "electricity category code") to manually adjust the rule verification model, ensuring it better meets actual business needs.

[0136] The rare business data identification and analysis module is used to highlight rare business data that accounts for a very small percentage and is extremely limited in quantity within the volume, price, and fee marketing data, based on the results obtained from rule parsing. This data may be erroneous, which business personnel can mark as "abnormal," or it may be reasonable data that exists only in a very small amount within the volume, price, and fee marketing data and is applicable only to specific scenarios.

[0137] The rule execution module is used to execute rules on the same field but different data in the quantity, price and fee business dataset based on the rule model modified by experts. The system will highlight abnormal data that violates the existing consistency rule model to assist business personnel in data consistency verification.

[0138] The quantity, price, and fee verification result management module is used to perform management operations such as adding, deleting, modifying, and querying the quantity, price, and fee data verification results, as well as storing the relevant verification results in the database.

[0139] Experimental results

[0140] To verify the feasibility and usability of this solution, we extracted 500,000 data entries from six fields—"Electricity Category Code," "Line Loss Billing Indicator," "Transformer Loss Billing Indicator," "Time-of-Use Billing Indicator," "Power Adjustment Calculation Method," and "Electricity Price Code"—from the marketing records most closely related to electricity volume, price, and fees in the power system, and conducted an experimental verification. Partial data is as follows:

[0141] The following table shows some examples of data from the verification dataset:

[0142]

[0143]

[0144] In the experiment, the upper limit of the error rate threshold for the algorithm parameters was set to e = 0.1 (i.e., the minimum confidence level ∈ = 0.9). The results of the consistency relationship between volume, price, and fee services after sorting by error rate in ascending order are partially shown below:

[0145] Rule Number Left side of the rule Right-hand side of the rule Rule error rate Are the rules strictly valid? 1 Electricity Price Code Electricity category code 0.0 yes 2 Electricity Price Code Force adjustment calculation method 0.000742 no 3 Line loss billing indicator Force adjustment calculation method 0.000746 no 4 Loss metering sign Force adjustment calculation method 0.000746 no 5 Electricity category code Force adjustment calculation method 0.000778 no 6 Time-based billing indicator Time-based billing indicator 0.00078 no 7 Electricity Price Code Time-based billing indicator 0.002372 no 8 Electricity category code Time-based billing indicator 0.004968 no 9 Line loss billing indicator Time-based billing indicator 0.006016 no 10 Loss metering sign Time-based billing indicator 0.006016 no 11 Force adjustment calculation method Time-based billing indicator 0.006016 no 12 Electricity Price Code Loss metering sign 0.009044 no 13 Line loss billing indicator Loss metering sign 0.011606 no

[0146] The consistency relationship mining results obtained by this method do not contain redundant or duplicate relationships. For each consistency relationship (essentially a relationship between quantity, price, and fee-related attributes), specific rules under that consistency relationship can be obtained through rule parsing (essentially parsing the mapping of different attribute values ​​in the consistency relationship to form specific verification rules). Taking the electricity price code and the power regulation electricity fee calculation method code as an example:

[0147]

[0148] All positive examples of this business relevance rule account for 99.9292% of the total data. In other words, more than 99.9% of the data supports the following: there is a consistent relationship between the electricity price code and the power regulation calculation method, and the value of the electricity price code determines the value of the power regulation calculation method.

[0149] After the rule parsing module, the specific rules and the values ​​of positive and negative examples are as follows (partially shown):

[0150]

[0151] The parsing results table provides detailed information on rule fields, positive / negative example values, the number of negative examples, their percentage, and confidence levels, offering good interpretability. Business personnel can use these rule interpretation results for review and analysis, and add, modify, or delete rules based on actual needs and business experience, thereby achieving the goal of revising the rule model. The revised rule model will be applied in the subsequent rule execution module to perform data consistency checks on business datasets with the same attributes.

[0152] Examples of consistency rules obtained after rule parsing are as follows:

[0153] If the electricity price code is 500203C, then the power regulation calculation method is 2.0;

[0154] If the electricity price code is 26010231, then the power regulation calculation method is 2.0.

[0155] Based on this rule model, during the rule execution process, this system will automatically highlight the quantity, price, and fee data with the following values ​​(electricity price code = 500203C, power regulation calculation method = 3.0) as abnormal data, thereby assisting business personnel in verifying quantity, price, and fee data.

[0156] Meanwhile, for rare business data in the rule parsing table that constitutes a very small percentage and quantity in the verification dataset, this system will highlight them for business experts to analyze. The rare business data portion is shown below in the rule parsing table derived from the aforementioned consistency relationship ("electricity price code" => "electricity category code"):

[0157]

[0158] As shown in the table above, among the 500,000 data points for quantity, price, and fee verification, some data combinations exist with only a dozen or even just a few data points supporting them. These can be considered rare data. Whether the existence of these rare data is reasonable is worth the attention of business experts (rare data is often beyond the reach of human experience, but since these rare data exist, they are either indeed abnormal data, or have a special business background that requires special handling, or represent a potentially valuable business feature that has not yet been clearly recognized). At the same time, the number of positive and negative examples for certain data values ​​in the rule parsing table is the same, or their proportions are roughly equal, meaning that their positive / negative examples are random. Whether their existence is reasonable is also worth the attention of business experts.

[0159] This invention employs an unsupervised learning method to mine consistency rules from categorized power data. Without relying on prior business experience or anomaly tags, it proactively uncovers potential consistency rules and identifies anomalies from business data. It provides detailed rule analysis, including information such as positive / negative example values, the number of positive / negative examples, their proportion in the overall data, and rule confidence levels, for business personnel to review and assess, providing a reference for rule relationships. Business personnel can add, delete, or modify the rule model by changing the causal relationships of rules such as ("electricity category code" => "electricity price code"). The system automatically converts the rules reviewed by business personnel into a power data consistency rule verification model, which is then applied in the rule execution module. This effectively detects anomalies in the business data that violate the rules in the model and provides anomaly risk alerts.

[0160] This invention can significantly reduce the workload of business experts in formulating verification rules in quantity, price, and fee verification operations. It automatically mines rules based on data distribution, greatly improving the verification coverage of the rules. Furthermore, it can interactively verify and adjust the automatic mining results with the human experience of business experts, further enhancing the applicability of the invention. It can effectively achieve comprehensive verification of the consistency relationship of quantity, price, and fee related data. By effectively identifying problematic data involving quantity, price, and fee calculations, it safeguards the economic interests of power supply companies while improving the standardization level of marketing data, promoting the intelligentization of marketing verification operations, and is also a concrete application of the digital transformation of power marketing operations.

[0161] This invention, even without prior business experience, utilizes algorithms to analyze the relationships between categorized data related to quantity, price, and fees, focusing on fundamental data requiring verification. It not only identifies which data attributes (fields) have business relationships but also reveals the specific value constraints between different related fields, thereby uncovering business consistency relationships and ultimately forming a series of specific business consistency verification rules. This enables the automatic mining and interpretation of data-driven categorized data business consistency rules, displaying the data mining results in the system for expert review, evaluation, selection, and improvement. It effectively solves problems such as insufficient rule coverage and difficulties in rule management encountered in manual rule definition. Complementing existing related functions in the power system, it significantly reduces the workload of business experts, improving the efficiency of verification work, ensuring the accuracy of quantity, price, and fee calculations, and is particularly significant for advancing the digitalization of the power sector, especially in the current context of promoting full meter reading, full employee billing, and centralized accounting—the "two full coverage" goals.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for mining rules to verify the consistency of electricity pricing, characterized in that, Includes the following steps: Step 1. Obtain the verification dataset and calculate the equivalent partitions of quantity, price, and fee data; Step 2. Use a depth-first spatial search strategy to mine the consistency relationships between quantity, price, and fee data to obtain the minimum complete set of consistency relationships in the data; Step 3. Use the obtained minimal complete set of consistent relations to perform rule parsing on the consistency relationship between quantity, price, and fee; Step 4. Based on the rule parsing process, identify rare data in the overall quantity, price, and fee data; Step 1, which calculates the equivalent partitioning of quantity, price, and fee data, includes: For a given set of verification data R Let the set of its classification attributes be denoted as . attr(R) ; Verification data set R field combinations X and a single field Y Consistency relationship between , denoted as: ;in, attr(R) This is called a consistency relation. The left-hand side LHS is the condition part of the consistency rule; attr(R) This is called a consistency relation. The right-hand side RHS, which is the result part of the consistency rule; For the verification dataset R field combinations X ,make for R A data record above; definition for X An equivalence class on, It contains the verification dataset R All of the above t In the field X The index of data with the same value; X The set of all equivalence classes is denoted as , is called X Equivalent partitions on; Step 2 uses a depth-first spatial search strategy to uncover potential consistency relationships between quantity, price, and fee data, specifically including the following sub-steps: For the search layer Perform initialization and define The "-" symbol represents R A certain field A Any value that can be taken; For any combination of fields Calculate its equivalent partition and ; Constructing a search tree For any given verification dataset R Calculate their difference set ,in, , the difference set Add to search tree In the leaf nodes; when When not empty, for the search tree The left term of the consistency relation LHS = X The right-hand side of the consistency relation RHS = A， The following conditions are used to determine whether a consistency relationship holds: like Then the consistency relation Established; In the formula, Indicates equivalent partitions The number of elements in the set, i.e., when the fields are combined X and When the number of equivalence classes in quantity, price, and fee data is equal, the fields There exists a valid data consistency relationship; If the current search layer Stop the spatial search algorithm and check the dataset. R All consistency relationships have been found; if the current search layer After pruning redundant conditions, the next search layer is calculated. ,in For the current search layer depth, X yes R Field combinations on; The obtained minimum complete and consistent relation set includes: If in the current search layer Then the depth-first search is completed, and the dataset is checked. R All consistency relationships have been identified, and the verification dataset has been obtained. R The minimum set of complete and consistent relations on ; Minimal complete consistent relation set Satisfy: For any set of consistency relations A certain consistency relation Remove X If any field in the set of consistency relations is not specified, the consistency relation is invalid; from the set of consistency relations In this process, the verification dataset can be derived. R All other consistency relations, meaning closure For any consistency relation Remove the set of relationships. Incomplete.

2. The method for mining rules for electricity pricing consistency verification according to claim 1, characterized in that, Step 2 uses a depth-first spatial search strategy to mine potential consistency relationships between quantity, price, and fee data, obtaining a set of consistency relationships on the verification dataset. This step also includes the following sub-steps: In the rule mining and detection process, support and confidence parameters are introduced. If the support and confidence threshold conditions are met, then the business consistency relationship on the verification dataset is approximately established. Redundant conditions in the aforementioned business consistency relationship are pruned; Verify all consistency relations on the dataset to obtain the minimum complete set of consistency relations on the dataset.

3. The method for mining rules for electricity pricing consistency verification according to claim 2, characterized in that, In step 2, during the rule mining and detection process, support and confidence parameters are introduced. If the support and confidence threshold conditions are met, then an approximate business consistency relationship is established on the verification dataset. Specifically, this includes: The classification error rate of a consistency relation is denoted as Therefore, the formula for calculating the classification error rate of the consistency relationship between quantity, price, and fee data is defined as follows: Let the confidence level of a consistency relation be . ,according to The calculation results show that the confidence level of this consistency relationship is: Among them, field combination X Equivalent partitions ,in yes X of p Similarly, define fields using equivalence classes. A Equivalent partitions The total amount of data verified is recorded as follows: ; Record the verification dataset R upper field combination X With fields A Consistency relationship between The support rate is Its value is defined as in R All satisfying this relation The number of data points; Preset minimum support and minimum confidence This will satisfy the threshold condition. ≥ ,and ≥ The relationship is called the verification dataset. R An approximate consistency relation on the , satisfying this consistency relation The quantity, price, and fee data are positive examples, while data that does not satisfy this consistency relationship are negative examples.

4. The method for mining rules for electricity pricing consistency verification according to claim 3, characterized in that, Step 2 involves pruning redundant conditions in the business consistency relationship, including: For each field combination X Constructing the RHS candidate generation set for the right-hand side of the consistency relation : ， in, B It is a combination of fields X Any field in, Indicates from set X Remove from A,B The difference set obtained from the fields; If consistency relationship If established, then A and all from Remove from the middle; like Then the field combination X From the current search layer The branches were pruned in the middle; If field combination X To verify the dataset R A key, then the field combination X From the current search layer The branches were pruned.

5. The method for mining rules for electricity pricing consistency verification according to claim 4, characterized in that, The parsing content in step 3 includes: the value of the condition part, the positive / negative example value of the right-hand side of the rule, the proportion of the rule with the value in the total data, the confidence of the rule, and the number of negative examples of the rule.

6. The method for mining rules for electricity pricing consistency verification according to claim 5, characterized in that, In step 4, based on the rule parsing process, rare data in the overall quantity, price, and fee data are identified, including: based on the proportion of positive / negative examples of various values ​​of quantity, price, and fee data in the overall data in the rule parsing table, and the confidence parameter information of specific values ​​of the rules, rare data with extremely small quantity and extremely low proportion in the overall quantity, price, and fee data are automatically parsed through parameter sorting method.

7. A system for mining rules for electricity pricing and fee consistency verification, employing the electricity pricing and fee consistency verification rule mining method according to any one of claims 1-6, characterized in that, include: The module includes: a volume, price, and fee data management module; a volume, price, and fee consistency rule mining module; an intelligent rule mining result display module; a rule mining result analysis and management module; a rare business data identification and analysis module; a rule execution module; and a volume, price, and fee verification result management module. The quantity, price, and fee data management module is used to manage the sources and standards of quantity, price, and fee data required by the data verification system, and to obtain the required business data from the corresponding data sources. The consistency rule mining module for volume, price, and fee is used to execute the consistency rule mining algorithm in the volume, price, and fee archive data according to the algorithm configuration and relevant input from business personnel, output relevant rule results, and automatically parse the rules. According to the distribution pattern of volume, price, and fee business data, it provides the value, error rate, positive and negative examples, and their respective quantities and proportions for each specific rule. The rule mining result analysis and management module allows experts to determine the usability and effectiveness of a rule based on the detailed analysis of quantity, price, and fee rule examples and the consistency relationships between different field values ​​displayed in the rule display module. This is achieved through an understanding of electricity pricing and fee business and the interpretation of positive and negative examples of the rule. This module will uniformly manage the effective rules, and business experts can set the rules to be valid or invalid according to business needs, thereby generating a rule model. The rare business data identification and analysis module is used to highlight rare business data that account for a very small proportion and are very few in number in the volume, price and fee marketing archive data according to the results obtained from rule parsing, so that business experts can review and analyze them. The rule execution module is used to execute rules on the same field but different data in the quantity, price and fee business dataset based on the rule model modified by experts. The system will highlight abnormal data that violates the existing consistency rule model to assist business personnel in data consistency verification. The quantity, price, and fee verification result management module is used to perform add, delete, modify, and query management operations on the quantity, price, and fee data verification results, as well as to store the relevant verification results in the database.

Images