Electric power project data logic anomaly correction method, device and equipment and storage medium

Abstract

The invention provides an electric power project data logic exception correction method and device, equipment and a storage medium, and the method comprises the steps: constructing a data logic verification network based on an association rule mining algorithm, so as to express a constraint relation between fields; non-zero dependence of the data record is verified through a constraint propagation mechanism, and a logic anomaly result is generated when constraint violation is detected; triggering automatic correction: obtaining similar normal cases from the historical knowledge base to generate a correction scheme, and applying the correction scheme to the abnormal field to obtain a modified record; and performing secondary verification to ensure that the correction is effective. According to the method and the device, through construction of the verification network, verification dependence, automatic correction and secondary verification, rapid, accurate and dynamic correction of logic anomalies is realized, and the defects of low manual correction efficiency and lack of verification are overcome.

Description

Technical Field

[0001] This application belongs to the field of data repair, and in particular relates to a method, apparatus, equipment and storage medium for correcting logical anomalies in power project data. Background Technology

[0002] Data anomaly detection in power projects is used to identify logical anomalies in unstructured text data to improve data quality management. These methods include rule-based detection techniques and rule-enhanced techniques incorporating domain knowledge, aiming to improve anomaly detection efficiency by automating manual review.

[0003] In practical implementation, existing technologies construct data logic verification networks through association rule mining algorithms or ontology models to express the constraint relationships between fields, and utilize rule engines or inference mechanisms to verify the non-zero dependencies of data records. For example, by pre-setting professional terminology specifications or domain ontology, logical relationships between data fields are analyzed, and logical anomaly markers are generated when constraints are violated. This approach can achieve basic anomaly identification in structured or semi-structured data, reducing manual intervention.

[0004] However, existing technologies rely on manually defined rules and lack an automatic correction mechanism when verifying logical anomalies. When an anomaly is detected, manual intervention is required to correct the data, which is inefficient and prone to subjective errors. At the same time, the lack of a secondary verification process after correction makes it impossible to ensure the accuracy and consistency of the correction results, thus failing to meet the high-efficiency requirements of dynamic data management in power projects. Summary of the Invention

[0005] The purpose of this application is to overcome the deficiencies in the prior art and provide a method, apparatus, device and storage medium for correcting data logic anomalies in power projects.

[0006] This application provides a method for correcting data logic anomalies in power projects, including:

[0007] A data logic validation network is constructed based on an association rule mining algorithm. The data logic validation network is used to express the constraint relationships between fields.

[0008] Based on the data logic verification network, the non-zero dependencies of data records are verified through a constraint propagation mechanism, and a logic anomaly detection result is generated when a constraint violation is detected.

[0009] Based on the logical anomaly detection result, an automatic correction mechanism is triggered: normal cases similar to the feature combination of the data record are obtained from the historical knowledge base, and a correction scheme is generated based on the normal cases; the correction scheme is applied to the data record to modify the abnormal fields, and the modified data record is generated.

[0010] The modified data record is input into the constraint propagation mechanism for secondary verification, and a secondary verification result is generated.

[0011] Based on the secondary verification results, it is determined that the logical anomalies in the modified data records have been corrected and passed verification.

[0012] Optionally, based on the data logic verification network, the non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including:

[0013] Determine non-zero dependency rules, which specify that the loop number field must be non-zero when the path length field is non-zero;

[0014] The non-zero dependency relationships of the verification data records are based on the non-zero dependency relationship rules.

[0015] Optionally, an automatic correction mechanism is triggered based on the logical anomaly detection result, including:

[0016] The automatic correction mechanism is triggered when the logical anomaly detection result indicates that the anomaly type is a value range close to the allowable range or a minor deviation in the format.

[0017] The value range being close to the allowable range means that the data value is close to but does not exceed the preset limit. The slight deviation in format means that the data format has a difference from the standard format that can be automatically corrected.

[0018] Optionally, obtain normal cases similar to the combination of features of the data records from the historical knowledge base, and generate a correction scheme based on the normal cases, including:

[0019] The data record feature combination includes a material type field, a core number field, and a cross-sectional area field;

[0020] Similar case retrieval is performed based on the combination of the material type field, core number field, and cross-sectional area field.

[0021] Optionally, based on the data logic verification network, the non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including:

[0022] Before verification, the expected value of the loop number segment is derived from the path length field value in the data record through a forward reasoning mechanism;

[0023] The non-zero dependencies of the verification data records are based on the expected values.

[0024] Optionally, based on the data logic verification network, the non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including:

[0025] Verify the global consistency of all non-zero dependencies in the data records, where all dependencies are true simultaneously;

[0026] The global consistency refers to the simultaneous satisfaction of the constraints of the path length field, the loop number segment, and related fields.

[0027] Optionally, the modified data records are input into the constraint propagation mechanism for secondary verification to generate secondary verification results, including:

[0028] If the secondary verification result indicates an anomaly that is not corrected, the data record will be transferred to manual processing.

[0029] If the secondary verification result indicates that the anomaly has been corrected, then the data record is confirmed to have passed verification;

[0030] The phrase "transfer to manual processing" refers to marking abnormal data and sending it to a manual review process.

[0031] This application also provides a dynamic correction device for data logic anomalies in power projects, including:

[0032] The network module constructs a data logic verification network based on an association rule mining algorithm. This data logic verification network is used to express the constraint relationships between fields.

[0033] The detection module, based on the data logic verification network, verifies the non-zero dependencies of data records through a constraint propagation mechanism, and generates a logic anomaly detection result when a constraint violation is detected.

[0034] The repair module triggers an automatic correction mechanism based on the logical anomaly detection result: it retrieves normal cases similar to the feature combination of the data record from the historical knowledge base, generates a correction scheme based on the normal cases, applies the correction scheme to the data record to modify the abnormal fields, and generates a modified data record.

[0035] The verification module inputs the modified data records into the constraint propagation mechanism for secondary verification and generates secondary verification results.

[0036] The determination module determines, based on the secondary verification results, that the logical anomalies in the modified data records have been corrected and passed verification.

[0037] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0038] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0039] The beneficial effects of this application are:

[0040] This application provides a method for correcting logical anomalies in power project data, comprising: constructing a data logic verification network based on an association rule mining algorithm, wherein the data logic verification network is used to express the constraint relationships between fields; verifying the non-zero dependency relationships of data records through a constraint propagation mechanism based on the data logic verification network, and generating a logical anomaly detection result when a constraint violation is detected; triggering an automatic correction mechanism based on the logical anomaly detection result: obtaining normal cases similar to the feature combinations of the data records from a historical knowledge base, and generating a correction scheme based on the normal cases; applying the correction scheme to the data records to modify the abnormal fields, generating modified data records; inputting the modified data records into the constraint propagation mechanism for secondary verification, generating a secondary verification result; and determining that the logical anomaly in the modified data records has been corrected and passed verification based on the secondary verification result. This application, by constructing a data logic verification network based on association rule mining, combining it with a constraint propagation mechanism to verify non-zero dependency relationships and trigger automatic correction, and utilizing a historical knowledge base to generate correction schemes and perform secondary verification, achieves rapid and accurate dynamic correction of logical anomalies, overcoming the shortcomings of existing technologies that rely on manual correction and lack verification. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the data logic anomaly correction process for power projects in this application. Detailed Implementation

[0042] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that various forms of implementation of the present disclosure are intended and should not be limited to the embodiments set forth herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0043] Please refer to Figure 1 This application provides a method for correcting logical anomalies in power project data, applied in the power field, to solve logical anomaly problems in power project data. The method includes:

[0044] S101. Construct a data logic verification network based on the association rule mining algorithm. The data logic verification network is used to express the constraint relationship between fields.

[0045] By using association rule mining algorithms to analyze historical power project data, the dependencies between fields can be identified, such as rules like "when the path length field is non-zero, the loop number segment must be non-zero".

[0046] Based on these rules, each field in the data table is treated as a node, and the conditional relationships in the rules are treated as edges, constructing a multi-node data logic validation network. This network can express the constraint relationships between fields, providing a foundation for subsequent validation.

[0047] Among them, the association rule mining algorithm uses common data mining techniques, such as the Apriori algorithm or the FP-growth algorithm, to extract frequent itemsets and association rules from historical data.

[0048] Dependency rules are stored in "if-then" form, for example, if the path length is greater than 0, then the number of loops must be greater than 0.

[0049] When constructing the network, the fields include various parameters from power project data, such as path length, number of loops, material type, core count, and cross-sectional area. Historical data comes from structured information in a power project corpus, including parameters such as cable price, material, core count, and cross-sectional area. The selection of feature combinations is based on power industry standards and statistical analysis of historical project data, including material type and calculated features. For example, in the cable price detection case, the dataset contains multiple materials (such as copper core and aluminum core) and different "core count x cross-sectional area" values ​​(e.g., ...). Based on this, they are grouped as "copper core and..." Subgroup, aluminum core and Subgroups, etc., are processed independently.

[0050] The data logic verification network is built based on these grouped data to ensure that the network can cover a variety of business scenarios.

[0051] S102. Based on the data logic verification network, the non-zero dependency relationship of data records is verified through the constraint propagation mechanism, and a logic anomaly detection result is generated when a violation of the constraint is found.

[0052] Causal deduction is performed in a logic verification network using a forward inference mechanism.

[0053] Starting from the known data state, and according to the rules in the logical verification network (e.g., if the path length field satisfies a non-zero condition, then the loop number segment must also satisfy a non-zero condition), the expected values ​​or states of other fields are derived. For example, the expected value of the loop number segment is derived from the value of the path length field.

[0054] Then, the global consistency of non-zero dependencies is verified based on the constraint propagation mechanism, that is, checking whether all related fields simultaneously satisfy the constraint conditions. Global consistency means that the constraints of the path length field, the loop number field, and related fields are all met simultaneously. For example, the loop number must be non-zero when the path length is non-zero, and other dependent fields must also conform to the rules. During the verification process, if a constraint violation is found (such as a non-zero path length but a zero loop number), the data record is immediately marked as having a logical anomaly, and a logical anomaly detection result is generated.

[0055] Anomaly detection results include the anomaly location (e.g., specific row number, column name), anomaly type (e.g., "Logical Anomaly - Violation of Non-Zero Dependency"), and the specific rule identifier violated. This tagging information is appended to the original record or its metadata in a structured format (e.g., JSON fields) and stored in the anomaly results database for subsequent processing.

[0056] Verification is performed based on the expected value derived from the forward inference mechanism to ensure the accuracy of the detection. For example, in cable data, if the path length has a value but the number of loops is empty, the forward inference predicts that the number of loops should be non-zero. If the actual value does not match the expected value, an anomaly is marked.

[0057] S103. Trigger the automatic correction mechanism based on the logic anomaly detection result:

[0058] S1031. Obtain normal cases similar to the combination of features of the data records from the historical knowledge base, and generate a correction scheme based on the normal cases.

[0059] When the logic anomaly detection result indicates that the anomaly type is close to the allowable range or that there is a slight deviation in the format, the automatic correction mechanism is triggered.

[0060] The value range is close to the allowable range, which means that the data value is close to but does not exceed the preset limit. For example, the terrain proportion value is equal to 1.02 while the threshold is 1.

[0061] Minor format deviations refer to differences in data format that can be automatically corrected compared to the standard format, such as omitting a space in cross-sectional data. The historical knowledge base stores normal data examples from past projects, including cable prices and equipment parameters. Feature combinations include material type, core number, and cross-sectional area fields; for example, similar case searches can be performed based on combinations of material type, core number, and cross-sectional area.

[0062] During retrieval, a similarity calculation algorithm (such as cosine similarity or Euclidean distance) is used to find the most matching normal cases that best match the data record. Based on the retrieved normal cases, correction schemes are generated, such as replacing outliers with typical values ​​from normal cases, or adjusting the format to conform to standards.

[0063] After the correction plan is generated, the correction logic and basis are recorded to ensure traceability. For example, for cable cross-section data, if the current value is "400" instead of "3", the correction will be corrected. If "400" is found, then normal cases with the same material and core number are searched from the historical knowledge base to obtain typical cross-sectional formats (such as "3"). "400" was proposed as a revised solution.

[0064] S1032. Apply the correction scheme to the data record to modify the abnormal fields and generate the modified data record.

[0065] The correction scheme is applied to the data records, directly modifying the values ​​or formats of abnormal fields. For example, if the value range is close to the threshold, it is adjusted to a reasonable value within the threshold (e.g., correcting the terrain proportion value from 1.02 to 1.0); if the format has a slight deviation, the format is corrected (e.g., supplementing the cross-sectional data "400" with "3"). 400”).

[0066] During the modification process, ensure that the correction operation does not affect the integrity of other fields, and that the modified data record maintains the same structure and metadata as the original record. Correction operations are based on normal cases in the historical knowledge base to ensure that the modified data conforms to business logic and industry standards. For example, in cable price data, if a price value is abnormal due to a format error, it is corrected to a normal price range value with the same feature combination. After modification, a modified data record is generated, and a modification log is recorded, including the modification time, modification content, and reason for modification.

[0067] S104. Input the modified data record into the constraint propagation mechanism for secondary verification and generate secondary verification results.

[0068] The modified data records are then input into the constraint propagation mechanism again to re-verify the global consistency of non-zero dependencies. The verification process is the same as the initial verification, using forward reasoning and constraint propagation to check whether all fields satisfy the constraints.

[0069] A secondary verification result is generated, indicating whether the anomaly has been corrected. If the secondary verification result indicates that the anomaly has not been corrected, the data record is transferred to manual processing, i.e., the anomaly is marked and sent to the manual review process for further inspection and processing by technical personnel. If the secondary verification result indicates that the anomaly has been corrected, the data record is confirmed to have passed verification.

[0070] Secondary verification ensures the validity of the correction operation. For example, if the modified loop number segment is also non-zero when the path length is non-zero, then the verification passes. The verification results are stored in a structured format, including the verification time, verification result, and abnormal status.

[0071] S105. Based on the secondary verification result, it is determined that the logical anomaly in the modified data record has been corrected and passed verification.

[0072] If the secondary verification confirms that the anomaly has been corrected, the data record is marked as normal and included in the qualified dataset for subsequent use. If the anomaly has not been corrected, the data record is marked as requiring manual processing, triggering an alarm mechanism, such as sending a notification to the technical maintenance team. The alarm information includes anomaly details and processing time limits to ensure timely response. Verified data records are stored in the project database, and an anomaly handling report is generated to support data governance in the power project. The entire process forms a complete "anomaly detection-correction-verification-closed loop" mechanism, ensuring that data logic anomalies are dynamically corrected.

[0073] Data format compliance verification based on a preset set of syntax rules:

[0074] A data format validation framework is constructed based on a pre-defined set of syntax rules to perform compliance verification on unstructured data. The syntax rule set is based on the "Power Engineering Data Specification" standard, which defines the standard format requirements for specific fields or columns in power project text corpora. The rules are stored in the rule base in configurable patterns (such as regular expressions) and logical conditions (such as numerical ranges and format templates). For example, for the "section" column data, the rule explicitly stipulates that its value must conform to the pattern structure of "numerical-numerical".

[0075] A constructed validation framework is used to perform line-by-line parsing and compliance verification of unstructured data in the text corpus of power projects. The validation process includes pattern matching, structural parsing, and logical verification. Pattern matching first checks whether the fields conform to the basic format using preset regular expressions. If they fail, they are directly marked as "format error - does not conform to the pattern" and do not proceed to the next step; if they pass, they proceed to structural parsing.

[0076] The structure parser splits the key parts (such as A and B) of the data that conforms to the pattern. If the split fails (such as the split result is not a numeric type), it is marked as "format error - parsing failure". If the split is successful, it will enter the logical verification.

[0077] The logical validation first performs a format integrity check (checking for missing multiplier terms; if only "400" is entered, it is marked as "format error - missing multiplier term"), and then performs a numerical range check (verifying whether num1 and num2 are within the preset range). The two checks are marked independently, and if both are abnormal, both abnormal types are marked simultaneously.

[0078] Format integrity check refers to identifying a column as having a "format error - missing multiplier" if the value is only a single number (e.g., 400) instead of a numeric format (e.g., 3 * 400). Numerical range check verifies whether each successfully split value (a portion of a column) falls within a preset valid range. For example, if the first value A (representing the number of roots) is less than the minimum allowed value min_A (e.g., preset to 1), it is marked as "numerical anomaly - A less than minimum value." If the second value B (representing the cross-sectional area of ​​a single root) is less than the minimum allowed value min_B (e.g., 10) or exceeds the maximum allowed value max_B (e.g., 1000), it is marked as "numerical anomaly - B out of range." Any anomaly on either side will trigger the flag. The preset thresholds can be dynamically adjusted through a configuration file.

[0079] Based on the compliance verification results, the system locates abnormal data that violates the preset syntax rules. For the detected abnormalities, the system accurately marks the abnormal location (such as the specific row number and column name), abnormal type, and specific rule identifier violated in the original data record. The marking information is attached to the original record or its metadata in a structured form (such as JSON fields) and stored in the abnormal result database for subsequent review or processing. The abnormal database record contains information such as the original data fragment, error description, rule ID, and timestamp.

[0080] The application frequency and error rate of all rules will be statistically analyzed. When the error rate exceeds a set threshold (e.g., 5%), an alarm for rule base maintenance or configuration parameter adjustment will be triggered. When the error rate exceeds the threshold, the system will automatically send an email and SMS alarm to the "Technical Maintenance Team" (3 or more members) and copy the business manager. The alarm information must include "Rule ID, current error rate, threshold, example of abnormal records (first 5), and statistical period". An error rate of 5‰-10‰ is a "general alarm" (to be processed within 24 hours), 10‰-20‰ is a "critical alarm" (to be processed within 8 hours), and more than 20% is an "emergency alarm" (to be responded to within 2 hours). Different priorities correspond to different processing time limits and escalation mechanisms (e.g., a general alarm that is not processed within the time limit will be automatically escalated to a critical alarm).

[0081] First, determine the cause of the anomaly. If it is due to unreasonable parameters (e.g., the num2max value is too low), adjust the parameters. If it is due to a defect in the rule itself (e.g., the regular expression omits special cases), optimize the rule. The operation must be recorded in the "maintenance log", indicating the modified content and test results. After maintenance, 5% (at least 100 records) of the records covered by the rule should be extracted for re-verification. If the error rate drops below the threshold (e.g., <5%), the maintenance is confirmed to be effective, and the new configuration is applied to all records. If the error rate still exceeds the standard, the cause needs to be re-analyzed until the problem is resolved, forming a complete process of "alarm-processing-verification-closure".

[0082] Multidimensional data are grouped based on different feature combinations, and the statistical interquartile range (IQR) algorithm is used to accurately label outliers.

[0083] Multidimensional data is divided into multiple subgroups based on different feature combinations. The multidimensional data comes from structured information in a power project corpus, including parameters such as cable price, material, core count, and cross-sectional area. The selection of feature combinations is based on power industry standards and historical project data statistics, including material type and calculation characteristics. The grouping rule is to create a separate subgroup for each material category and each combination of calculated "core count x cross-sectional area". When dividing the subgroups, it is necessary to ensure that each group has enough data points to avoid small sample bias. Subgroups with insufficient data will be ignored or merged. The definition of feature combinations comes from the technical manuals of power equipment manufacturers and project historical databases, and the combination parameters can be dynamically added or modified through configuration files. For example, in the cable price detection case, the dataset contains multiple materials (such as copper core and aluminum core) and different (core count x cross-sectional area) values ​​(such as 2x10mm2). Based on this, it is grouped into "copper core and (2x10mm2) subgroup", "aluminum core and (3x16mm2) subgroup", etc., and each subgroup is processed independently.

[0084] The IQR algorithm is used to calculate the quantile interval and median of each subgroup. For each subgroup, the lower quartile Q1, upper quartile Q3, and quantile interval IQR are calculated first. The median M is calculated using the fast selection algorithm or a built-in library function. The weight parameter k of IQR (used to define the outlier boundary) is 1.5 by default. This value is based on the statistical experience of historical anomaly data in the power industry (such as cable price fluctuation analysis) and can be set to a variable range (such as 1.0-3.0) by the configuration file.

[0085] During the calculation, linear interpolation is used to handle non-integer indices to ensure accuracy. The algorithm is implemented based on Python or R libraries and optimized locally using power project datasets. For example, in the cable price case, Q1, Q3, IQR (difference) and M (median) are calculated for "copper core and (2x10mm2) subgroup". After cleaning the price data, only numerical records are retained, and string or missing values ​​are filled by interpolation (replaced with the median of the same group).

[0086] Based on the calculated quantile interval and median to correct the accuracy of outlier labeling, and combined with the obtained subgroups Q1, Q3, IQR, and M, the specific outlier labeling rule is as follows: if the data point is less than M-kIQR, it is labeled as abnormally small; if it is greater than M+kIQR, it is labeled as abnormally large; otherwise, it is labeled as normal. The labeling results generate a new column of data (numerical type), where 0 represents normal, 1 represents abnormally small, and 2 represents abnormally large.

[0087] The outlier correction mechanism includes missing data handling (missing values ​​are filled with the median of the same group), boundary conflict adjustment (e.g., when the k value is too large and causes too many outliers, the k value is automatically reduced to a step size of 0.1), and time sequence conflict checking (e.g., when the price record timestamps are in reverse order, they are prioritized to be reordered according to time sequence). The weight calculation of outlier labels refers to historical statistics, and the probability of anomalies is dynamically evaluated based on the k value and the IQR width. For example, in the cable price case, after calculating IQR and M, when the price point is less than M-1.5IQR, it is labeled as 1 (outlier than 1), when it is greater than M+1.5IQR, it is labeled as 2 (outlier than 2), and the rest are labeled as 0. Then a new data column (e.g., an outlier flag column) is generated and conditionally formatted to mark the color in the output interface: red indicates outlier than 1 (value is 1), and yellow indicates outlier than 2 (value is 2).

[0088] Where M is the median, k is the weighting parameter (default 1.5), and IQR is the quantile interval, i.e., IQR = Q3 - Q1.

[0089] Analyze the numerical information in the data, construct a numerical rationality assessment model, and identify abnormal data that exceeds the threshold range or violates business logic through semantic similarity calculation.

[0090] Specifically, key numerical information in the text corpus of power projects is parsed, a configurable verification framework is constructed, and a structured data verification framework is built in combination with business practice requirements. Three types of rules are defined: format rules (such as regular expressions for cable cross-sections), integrity rules (such as the number of loops and cross-sections cannot be empty when the path length is non-zero), and value range rules (such as the terrain proportion value ≤ 1). These rules are managed through XML configuration files and support dynamic loading and real-time updates of the rule base.

[0091] A numerical rationality assessment model integrating semantic similarity calculation is constructed. Thresholds and business logic constraints are set. Finite state machine technology is used to accurately locate and parse key fields in the data table. The rules in the above verification framework are applied to each field for compliance verification. When non-compliance with the rules is found, anomalies are automatically marked. The marking results are clearly identified in a separately generated "anomaly flag" column with codes (such as 0=normal, 1=format anomaly, 2=completeness missing, 3=value range exceeding limit). At the same time, conditional formatting is used to color the original data (such as red for value range exceeding limit, yellow for completeness missing).

[0092] For some of the identified anomalies, the framework can dynamically perform automatic correction: automatic correction is triggered only for two types of anomalies: "value range is close to the threshold but not completely exceeded" (e.g., terrain proportion value = 1.02, threshold is 1) and "minor format deviation" (e.g., cross-sectional data is missing 1 space). The correction is based on normal data with the same feature combination in the historical knowledge base (e.g., referring to the cross-sectional format of the last 3 similar projects). After correction, the data needs to be resubmitted into the verification framework for secondary verification to ensure that the corrected data meets all rules. Anomalies that fail the secondary verification need to be handled manually.

[0093] The framework employs an evaluation model to calculate numerical semantic similarity and logical compliance, accurately identifying and labeling numerical anomalies that exceed limits or violate logic. Multiple optimizations are then performed using the evaluation model. For some identified anomalies, the framework can dynamically and automatically correct them.

[0094] To ensure verification accuracy, 10% of the labeled results are manually sampled for quality control; if the error rate exceeds 5%, rule base optimization is triggered. Furthermore, all identified anomalous data and their detailed information are associated and stored to form a historical knowledge base, providing strong support for subsequent rule refinement, case analysis, and data governance.

[0095] A theoretical value prediction model is constructed using core data from power projects. A benchmark value range is generated through Monte Carlo simulation, and hypothesis testing is performed on the actual data.

[0096] First, clarify the business logic of data association, such as the relationship between cable price and material, core count, cross-sectional area, and average market raw material price. Then, select key influencing variables. For example, for cable price, select material type, core count, cross-sectional area, and average copper / aluminum price in the past month. Use multiple linear regression algorithm to build a theoretical value prediction model. The model is trained based on historical data of power projects in the past 5 years. The model fit R2 needs to be ≥0.85. Output the theoretical data prediction value of the corresponding feature combination. The model parameters are updated in real time through configuration file.

[0097] The number of simulations is set to 1000 by default and can be adjusted. Random sampling is performed on parameters with fluctuating characteristics in the model input variables, such as the fluctuation coefficient of the average market copper / aluminum price ±5% and the fluctuation coefficient of cross-sectional area processing error ±2%. Each sampling result is substituted into the theoretical value prediction model to generate one theoretical data value. After the simulation is completed, the 1000 theoretical data values ​​are sorted and the 95% confidence interval is taken as the benchmark value interval.

[0098] A one-sample t-test was used to compare the actual data with the benchmark range. The null hypothesis H0 was set as the actual data falling within the 95% confidence interval of the benchmark, and the alternative hypothesis H1 was the actual data falling outside the 95% confidence interval of the benchmark. The significance level was α=0.05. If the p-value was <0.05, the null hypothesis was rejected and the actual data was marked as "theoretical deviation - high risk". If the p-value was ≥0.05, the null hypothesis was accepted and the data was considered normal. The anomaly marking results were generated into a structured data table containing seven columns: data ID, feature combination, actual data value, benchmark range, p-value, anomaly marking result, and marking time. The data was colored using conditional formatting, with green for normal data and red for anomaly data. The table could be exported to Excel format and displayed in real time on the system output interface.

[0099] The model output results undergo manual sampling verification. 10% of the data points are randomly selected for evaluation of the reasonableness of anomaly labeling by power industry experts. If the error rate exceeds 5%, an adjustment process is triggered, including re-screening model input variables, supplementing training data from recent projects within the past year, or modifying Monte Carlo simulation parameters. The reasonableness of the labeling is cross-validated using a historical anomaly case database, and invalid labels are automatically stored in the anomaly database. During data grouping, if a subgroup has fewer than 10 data points, the system automatically triggers a small sample alarm and skips the calculation for that subgroup. During feature combination optimization, the weight of the core parameter combination dimension in the power industry is increased to 1.2 times that of the domain-standard parameters. If data has timestamp conflicts, it is uniformly converted to UNIX time format and reordered chronologically before calculation.

[0100] All labeled results and visualization tables are stored in the project database, and a "Theoretical Anomaly Data Analysis Report" is automatically generated, which includes the percentage of abnormal data, combinations of high-frequency anomaly features, and preliminary diagnosis of anomaly causes.

[0101] In addition, alternative implementation methods are also included. For example, for format compliance verification, alternative solutions can be to use pre-trained natural language processing models and template matching. First, use pre-trained NLP models in the power field, such as models based on BERT fine-tuning, to identify and extract fields from unstructured text data, automatically distinguishing key fields in the text, such as cable models, prices, and construction parameters. Then, for different fields, multiple sets of format templates are preset, such as number + unit or number x number templates. The template matching algorithm is used to compare the degree of fit between the extracted field content and the template. Data that fails to match or whose matching degree is lower than a set threshold, such as 80%, is marked as having an abnormal format.

[0102] For outlier labeling, alternative solutions include density clustering algorithms and dynamic radius adjustment. First, the DBSCAN algorithm is used to perform density clustering on the multidimensional data to divide the data into different clusters. Then, the density radius of the data within each cluster is dynamically calculated based on the data distribution of each cluster. Data that are more than a certain multiple of the density radius from the cluster center, such as 1.5 times, are labeled as outliers. At the same time, if the amount of data in a cluster is too small, such as less than 5 data points, the clusters are automatically merged into adjacent similar clusters before outlier identification.

[0103] For the assessment of numerical reasonableness, an alternative approach is to use knowledge graph association query and numerical trend analysis. First, a knowledge graph in the field of power projects is constructed to associate the numerical information in the text with entities in the knowledge graph, such as equipment models, material specifications, and attributes such as price ranges and parameter standards. Then, the range of entity attributes corresponding to the numerical value is queried through the knowledge graph to preliminarily determine whether the numerical value exceeds the reasonable range. At the same time, the numerical change trend of similar historical data, such as the price fluctuation trend of the same type of cable in the past 3 years, is combined with linear regression to predict the reasonable range of the current value. After comparing it with the actual value, anomalies are marked.

[0104] For the construction of a logical verification network, an alternative solution can be to use Bayesian network inference and probability threshold judgment. First, a Bayesian network is constructed based on historical power project data, with each field in the data as a network node. The logical relationship between the fields is transformed into the conditional probability between the nodes, such as the probability that the number of loops is greater than 0 when the path length is greater than 0. Then, the field values ​​of the current data are input and the posterior probability of the relevant nodes is calculated through Bayesian network inference. If the posterior probability of a certain node is lower than a set threshold, such as 5%, it is determined that the field has a logical conflict with other fields and is marked as abnormal.

[0105] For the generation of the benchmark interval, an alternative solution can be to use industry standard database queries and dynamic weighted averaging. First, establish an industry standard database for power projects to store the theoretical standard values ​​and allowable deviation ranges of data under different business scenarios. Then, match the corresponding standard value interval from the database according to the business attributes of the current data, such as the project voltage level and equipment type. At the same time, combine the actual data of similar projects in the past year to calculate the reference interval of the actual value through dynamic weighted averaging and merge it with the standard value interval to form the final benchmark interval. After comparing it with the actual data, mark the anomalies.

[0106] This application also provides a dynamic correction device for data logic anomalies in power projects, including:

[0107] The network module constructs a data logic verification network based on an association rule mining algorithm. This data logic verification network is used to express the constraint relationships between fields.

[0108] The detection module, based on the data logic verification network, verifies the non-zero dependencies of data records through a constraint propagation mechanism, and generates a logic anomaly detection result when a constraint violation is detected.

[0109] The repair module triggers an automatic correction mechanism based on the logical anomaly detection result: it retrieves normal cases similar to the feature combination of the data record from the historical knowledge base, generates a correction scheme based on the normal cases, applies the correction scheme to the data record to modify the abnormal fields, and generates a modified data record.

[0110] The verification module inputs the modified data records into the constraint propagation mechanism for secondary verification and generates secondary verification results.

[0111] The determination module determines, based on the secondary verification results, that the logical anomalies in the modified data records have been corrected and passed verification.

[0112] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0113] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0114] The above description of the embodiments is provided to enable those skilled in the art to understand and apply this application. Those skilled in the art will readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without inventive effort. Therefore, this application is not limited to the above embodiments, and any improvements and modifications made to this application based on the disclosure thereof should be within the scope of protection of this application.

Claims

1. A method for correcting logical anomalies in power project data, characterized in that, include: A data logic validation network is constructed based on an association rule mining algorithm. The data logic validation network is used to express the constraint relationships between fields. Based on the data logic verification network, the non-zero dependencies of data records are verified through a constraint propagation mechanism, and a logic anomaly detection result is generated when a constraint violation is detected. An automatic correction mechanism is triggered based on the logical anomaly detection result: normal cases similar to the combination of features of the data record are obtained from the historical knowledge base, and a correction scheme is generated based on the normal cases; The correction scheme is applied to the data record to modify the abnormal fields, and the modified data record is generated. The modified data record is input into the constraint propagation mechanism for secondary verification, and a secondary verification result is generated. Based on the secondary verification results, it is determined that the logical anomalies in the modified data records have been corrected and passed verification.

2. The method according to claim 1, characterized in that, Based on the aforementioned data logic verification network, non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including: Determine non-zero dependency rules, which specify that the loop number field must be non-zero when the path length field is non-zero; The non-zero dependency relationships of the verification data records are based on the non-zero dependency relationship rules.

3. The method according to claim 1, characterized in that, An automatic correction mechanism is triggered based on the logical anomaly detection result, including: The automatic correction mechanism is triggered when the logical anomaly detection result indicates that the anomaly type is a value range close to the allowable range or a minor deviation in the format. The value range being close to the allowable range means that the data value is close to but does not exceed the preset limit. The slight deviation in format means that the data format has a difference from the standard format that can be automatically corrected.

4. The method according to claim 1, characterized in that, Obtain normal cases similar to the combination of features of the data records from the historical knowledge base, and generate a correction scheme based on the normal cases, including: The data record feature combination includes a material type field, a core number field, and a cross-sectional area field; Similar case retrieval is performed based on the combination of the material type field, core number field, and cross-sectional area field.

5. The method according to claim 1, characterized in that, Based on the aforementioned data logic verification network, non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including: Before verification, the expected value of the loop number segment is derived from the path length field value in the data record through a forward reasoning mechanism; The non-zero dependencies of the verification data records are based on the expected values.

6. The method according to claim 1, characterized in that, Based on the aforementioned data logic verification network, non-zero dependencies of data records are verified through a constraint propagation mechanism. When a constraint violation is detected, a logic anomaly detection result is generated, including: Verify the global consistency of all non-zero dependencies in the data records, where all dependencies are true simultaneously; The global consistency refers to the simultaneous satisfaction of the constraints of the path length field, the loop number segment, and related fields.

7. The method according to claim 1, characterized in that, The modified data records are input into the constraint propagation mechanism for secondary verification, generating secondary verification results, including: If the secondary verification result indicates an anomaly that is not corrected, the data record will be transferred to manual processing. If the secondary verification result indicates that the anomaly has been corrected, then the data record is confirmed to have passed verification; The phrase "transfer to manual processing" refers to marking abnormal data and sending it to a manual review process.

8. A dynamic correction device for data logic anomalies in power projects, characterized in that, include: The network module constructs a data logic verification network based on an association rule mining algorithm. This data logic verification network is used to express the constraint relationships between fields. The detection module, based on the data logic verification network, verifies the non-zero dependencies of data records through a constraint propagation mechanism, and generates a logic anomaly detection result when a constraint violation is detected. The repair module triggers an automatic correction mechanism based on the logical anomaly detection result: it retrieves normal cases similar to the data record feature combination from the historical knowledge base and generates a correction scheme based on the normal cases; The correction scheme is applied to the data record to modify the abnormal fields, and the modified data record is generated. The verification module inputs the modified data records into the constraint propagation mechanism for secondary verification and generates secondary verification results. The determination module determines, based on the secondary verification results, that the logical anomalies in the modified data records have been corrected and passed verification.

9. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed in a computer, causes the computer to perform the method described in any one of claims 1-7.

Images