Method and System for Generating Multimodal Power Data Quality Rules

Abstract

This invention belongs to the field of power data quality governance technology, and provides a method and system for generating power multimodal data quality rules. By converting multi-source, multimodal data collected from power equipment into traceable quality evidence objects, and further performing grouping and aggregation, business constraint association, recurring quality problem identification, and structured quality attribution evidence construction, the generation of data quality rules no longer relies solely on human experience, standard documents, or fixed templates, but can automatically precipitate from recurring quality problems in actual collected data. Simultaneously, common quality evidence features are determined by comparing feature contribution with a preset contribution threshold. Structured quality attribution evidence objects are generated based on these common quality evidence features and the original data mapping set. Furthermore, candidate data quality rules are validated, solving the overfitting problem and improving the verifiability and reusability of generated power data quality rules.

Description

Technical Field

[0001] This invention belongs to the fields of power data governance, data quality management and artificial intelligence technology, and specifically relates to a method and system for generating power multimodal data quality rules. Background Technology

[0002] With the continuous advancement of digitalization and intelligentization in the power industry, a large amount of heterogeneous data from multiple sources is generated during the planning and construction, operation monitoring, inspection and maintenance, defect handling, and condition assessment of power equipment. This data includes structured data such as equipment ledgers, operational measurements, and condition monitoring data; textual data such as inspection records, maintenance records, defect work orders, and test reports; and image data such as equipment nameplate photos, on-site equipment images, infrared images, and meter images. This data forms a crucial foundation for subsequent equipment condition assessment, defect analysis, data governance, fault location, and cross-departmental business collaboration; its quality directly impacts the accuracy and reliability of subsequent business applications.

[0003] However, the current generation and maintenance of data quality rules for multi-source and multi-modal data collected from power equipment lacks the ability to automatically construct attribution evidence and aggregate common quality patterns from recurring quality problems in the data itself. For example, clustered data quality problems often trigger chain reactions and accompanying anomalies in multi-dimensional features. If all the variant features contained in the object to be generated are directly used as the basis for rule construction, it will inevitably lead to the subsequent generated rules containing a large number of irrelevant and redundant logical conditions, resulting in rule overfitting, high redundancy, and poor business interpretability. Summary of the Invention

[0004] To address the technical problems mentioned above, this invention provides a method and system for generating power multimodal data quality rules. Through grouping and aggregation, business constraint association, identification of recurring quality problems, and construction of structured quality attribution evidence, the generation of data quality rules no longer relies solely on human experience, standard documents, or fixed templates, but can be automatically derived from recurring quality problems in actual collected data. Simultaneously, by determining common quality evidence features based on a comparison of feature contribution degree with a preset contribution degree threshold, and by verifying candidate data quality rules, the overfitting problem is solved, improving the verifiability and reusability of generated power data quality rules.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a method for generating power multimodal data quality rules, comprising: Acquire structured data, text data, and image data related to power equipment; Structured data, text data, and image data are transformed into a set of quality evidence objects containing device objects, data sources, data modalities, original locations, basic quality attributes, and contextual information. Based on the set of quality evidence objects, the quality evidence objects are grouped and aggregated; the grouping and aggregation results are associated with preset business constraints to form a set of candidate quality problem groups; repeatability analysis is performed on each candidate quality problem group to determine the rule objects to be generated. A structured quality attribution evidence set is constructed based on the set of rules to be generated; a quality evidence feature set is constructed by extracting fields from the quality evidence object set; the feature contribution degree of any quality evidence in the quality evidence feature set is determined; common quality evidence features are determined by comparing the feature contribution degree with a preset contribution degree threshold; and structured quality attribution evidence objects are generated based on the common quality evidence features and the original data mapping set. Candidate data quality rules are generated based on the structured quality attribution evidence object; the candidate data quality rules are then validated.

[0006] Furthermore, the structured data includes equipment ledgers, operational measurements, status monitoring, and defect work orders; the text data includes inspection records, maintenance records, defect descriptions, test reports, and work order remarks; and the image data includes equipment nameplate images, field equipment images, infrared images, and meter images.

[0007] Furthermore, the grouping and aggregation includes: after normalizing the quality evidence objects, quality evidence from the same data source, the same type of field, or the same type of text slot is grouped into the same-source quality problem group; quality evidence from different data sources but pointing to the same power equipment object is grouped into the cross-source quality problem group; and quality evidence from structured data, text data, and image data that all point to the same equipment object or the same business event is grouped into the cross-modal quality problem group.

[0008] Furthermore, the determination of the rule object to be generated includes: when one or more of the following indicators reach the corresponding threshold: the number of quality evidence objects, the number of device objects covered, the number of data batches covered, and the number of time windows, the corresponding candidate quality problem group is determined as a repetitive quality problem pattern, and the rule object to be generated is generated.

[0009] Furthermore, the quality evidence features include at least one of the following: data source features, data modality features, original location features, basic quality attribute features, business constraint features, equipment object features, time window features, and original value performance features.

[0010] Furthermore, the feature contribution for: ; in, This indicates the number of items within the statistics window. The r-th quality evidence feature corresponding to each rule object to be generated; Indicates the frequency of occurrence after normalization; Indicates the coverage range of normalized equipment and data batches; Indicates the degree of concentration of positions after normalization; This indicates the degree of consistency of business constraints after normalization. , , , For the corresponding weights.

[0011] Furthermore, the feature contribution threshold Dynamically based on the adaptive window sliding statistics method according to the distribution of historical data quality characteristics: ; in, This represents the arithmetic mean of the contribution scores of all extracted candidate quality evidence features within the currently selected historical statistical time window. This represents the standard deviation of the contribution scores of all candidate quality evidence features within the statistical time window. This is reflected in the adjustment robustness coefficient.

[0012] Furthermore, the candidate data quality rules include rule name, applicable objects, applicable data sources, applicable locations, rule conditions, rule judgment expressions, basic quality attributes, business constraints, and evidence sources.

[0013] Furthermore, the verification of the candidate data quality rules includes: constructing a multi-dimensional verification function to jointly and quantitatively review the candidate rules; if the value of the multi-dimensional verification function is greater than or equal to the target threshold, the candidate rule is activated as the target data quality rule and formally added to the database; otherwise, it is corrected; the multi-dimensional verification function... for: ; in, , , , These are the pre-set deterministic evaluation weighting coefficients; For syntax checking indicator operators; Scoring for consistency of business constraints; For sample replay hit rate; Operator for indicating conflicts in historical rules.

[0014] A second aspect of the present invention provides a power multimodal data quality rule generation system, comprising: The data acquisition module is configured to acquire structured data, text data, and image data related to power equipment. The quality evidence object set determination module is configured to convert structured data, text data, and image data into a quality evidence object set containing device object, data source, data modality, original location, basic quality attributes, and contextual information. The module for determining the rule objects to be generated is configured to: group and aggregate the quality evidence objects based on the set of quality evidence objects; associate the grouping and aggregation results with preset business constraints to form a set of candidate quality problem groups; and perform repeatability analysis on each candidate quality problem group to determine the rule objects to be generated. The structured quality attribution evidence object determination module is configured to: construct a structured quality attribution evidence set based on the set of rule objects to be generated; extract fields from the quality evidence object set to construct a quality evidence feature set; determine the feature contribution of any quality evidence in the quality evidence feature set; determine common quality evidence features based on a comparison between the feature contribution and a preset contribution threshold; and generate structured quality attribution evidence objects based on the common quality evidence features and the original data mapping set. The verification module is configured to: generate candidate data quality rules based on the structured quality attribution evidence object; and verify the candidate data quality rules.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for generating power multimodal data quality rules.

[0016] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the above-described power multimodal data quality rule generation method.

[0017] A fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the above-described method for generating power multimodal data quality rules.

[0018] This invention transforms multi-source, multi-modal data collected from power equipment into traceable quality evidence objects. Further, it performs grouping and aggregation, business constraint association, identification of recurring quality problems, and construction of structured quality attribution evidence. This allows data quality rules to be automatically derived from recurring quality issues in actual collected data, rather than relying solely on human experience, standard documents, or fixed templates. Simultaneously, it determines common quality evidence features by comparing feature contribution with preset contribution thresholds, generates structured quality attribution evidence objects based on these common quality evidence features and the original data mapping set, and validates candidate data quality rules. This solves the overfitting problem and improves the verifiability and reusability of generated power data quality rules. Attached Figure Description

[0019] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0020] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a flowchart illustrating the construction of structured quality attribution evidence in Embodiment 1 of the present invention. Figure 3 This is a flowchart illustrating the data quality rule generation, verification, correction, and data entry process of Embodiment 1 of the present invention. Figure 4 This is a schematic diagram illustrating the process of generating integrity rules for key fields in the transformer equipment ledger according to Embodiment 1 of the present invention. Detailed Implementation The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0021] It should be noted that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Equivalent substitutions or improvements made by those skilled in the art to related steps, models, rule expressions, or data processing methods without departing from the core concept of the present invention should all be included within the scope of protection of the present invention.

[0022] The multimodal power data referred to in this article includes structured data, text data, and image data generated during the acquisition of power equipment data. Structured data may include equipment ledgers, operational measurements, and condition monitoring data; text data may include inspection records, maintenance records, defect records, and test reports; and image data may include equipment nameplate images, field equipment images, infrared images, and meter images.

[0023] The data quality rules referred to in this article are executable rules used to determine whether multimodal power data meets the requirements in terms of integrity, legality, consistency, timeliness, identifiability, and satisfaction of business constraints.

[0024] Example 1: The generation and maintenance of data quality rules for multi-source, multi-modal data acquisition from power equipment still face the following problems: Quality issues stem from complex sources, making manual rule configuration insufficient to cover them all. Power equipment data originates from various business systems, acquisition terminals, and manual recording processes. These data sources differ in field naming, acquisition frequency, time granularity, equipment identification, recording format, and representation. Taking transformer data acquisition as an example, the ledger data, operational measurement data, maintenance texts, and field images for the same equipment may originate from different systems or business processes, easily leading to quality issues such as inconsistent equipment identification, missing acquisition times, inconsistent field formats, missing key fields, missing text slots, and unrecognizable images. Traditional rule-based methods relying on manual configuration are ill-equipped to preemptively address these complex quality problems.

[0025] Existing rules largely rely on standard documents, data dictionaries, or fixed templates, making it difficult to extract implicit rules from the data itself. Current data quality rules typically originate from standard specifications, data dictionaries, expert experience, or fixed threshold templates, suitable for explicit rule checks such as missing values, formatting issues, enumerated values, and value ranges. Some technologies automatically extract rules from national, industry, enterprise, or technical specifications and build structured rule bases. However, in actual data collection, many quality rules do not exist directly in standard documents but rather manifest as recurring quality problems and anomaly patterns in historical data. Examples include repeated missing fields in multiple batches of a certain type of equipment, long-term inconsistencies in equipment identification between one data source and another, and persistent missing key slots in certain text records. These types of rules need to be discovered and summarized from the data itself.

[0026] Existing anomaly detection or quality assessment results are difficult to directly translate into rules. While existing data quality detection methods can identify missing fields, formatting errors, out-of-bounds values, or sample anomalies, and output anomaly labels, scores, or quality assessment results, they typically remain at the "whether it's anomaly" level, failing to further clarify the corresponding device object, data source, field location, text slot, image region, quality attribute, and business constraints. Due to the lack of structured attribution evidence for rule generation, anomaly detection results cannot be automatically converted into executable data quality rules, still requiring manual analysis and summarization.

[0027] Large-scale model-assisted rule generation lacks controlled evidence and verification loops. With the development of large language models and multimodal models, existing technologies have attempted to utilize large models to assist in rule generation or knowledge base construction. However, directly inputting raw data, outlier samples, or natural language descriptions into large models can easily lead to problems such as a lack of supporting evidence, unclear rule conditions, non-standard execution expressions, duplication or conflict with historical rules, and inconsistencies with power business constraints. Therefore, it is necessary to construct traceable, structured quality attribution evidence before rule generation and to verify the rules after generation through methods such as sample playback, business constraint verification, and comparison with historical rules.

[0028] In summary, while existing technologies can support the construction of data quality rules based on manual configuration, standard document extraction, or fixed templates, and can also perform anomaly detection and quality evaluation on collected data, they still lack a technical solution for automatically constructing attribution evidence and aggregating common quality patterns from recurring quality problems in power equipment multi-source and multi-modal collected data, and further generating, verifying, and storing data quality rules.

[0029] To solve at least one of the above problems, such as Figure 1 As shown, this embodiment provides a method for generating power multimodal data quality rules. It targets multi-source, multimodal data collected from power equipment, and constructs quality evidence objects by parsing structured data, text data, and image data. This identifies recurring quality problems, generates structured quality attribution evidence, and automatically generates, validates, and stores data quality rules based on a large model. Specifically, the method includes the following steps: S1. Acquisition of multimodal power data and construction of quality evidence objects: S1.1 Acquiring multi-source, multi-modal data from power equipment: The power multimodal data includes structured data, text data, and image data. Structured data may include equipment ledgers, operational measurements, status monitoring data, defect work orders, etc.; text data may include inspection records, maintenance records, defect descriptions, test reports, work order notes, etc.; and image data may include equipment nameplate images, field equipment images, infrared images, meter images, etc.

[0030] S1.2. Analyze the different types of data obtained separately: For structured data, methods such as field mapping, data type identification, unit normalization, and primary key association are used to identify field names, field types, field values, acquisition time, data source, and device objects. For text data, methods such as word segmentation, named entity recognition, slot extraction, regular expression matching, or large language model information extraction are used to identify key slots such as text source, paragraph structure, device name, time, location, phenomenon description, and processing results. For image data, methods such as image metadata parsing, object detection, OCR recognition, image quality evaluation, or multimodal model recognition are used to identify image source, shooting time, associated device, image type, and target area, and to extract acquisition quality information such as sharpness, occlusion, cropping, exposure, and identifiability.

[0031] S1.3 After parsing is complete, construct a quality evidence object for generating quality rules: The quality evidence object is used to record the quality status and source location of a single field, text slot, or image region, and can be represented as: ; in, Represents an electrical equipment object; Indicate the data source; Represents data modality; Indicates the location of a field, text slot, or image region; Indicates basic quality attributes; This represents the original value or quality index value; Indicates the time of data collection or recording; This indicates contextual information related to the current quality evidence.

[0032] In one alternative implementation, the quality evidence object may be represented by a key-value pair structure, a table structure, or a JSON structure, and its fields may include at least a device object field, a data source field, a data modality field, an original location field, a basic quality attribute field, an original value or quality indicator field, a time field, and a context field.

[0033] Through the above processing, power equipment acquisition data scattered across different systems, formats, and modalities can be transformed into a set of quality evidence objects containing equipment objects, data sources, data modalities, original locations, basic quality attributes, and contextual information. This set of quality evidence objects not only preserves the traceable mapping relationship between the original data and the quality status but also provides the basic input for subsequent quality evidence grouping, business constraint association, identification of recurring quality problems, construction of structured attribution evidence, and automatic generation of data quality rules.

[0034] S2. Associating Quality Evidence Grouping and Aggregation with Business Constraints: Based on the set of quality evidence objects constructed in step S1, the quality evidence objects are standardized and grouped and aggregated, and the grouping and aggregation results are associated with preset business constraints to form candidate quality problem groups.

[0035] S2.1 Standardization Processing: Specifically, each quality evidence object in the quality evidence object set is read. These quality evidence objects include power equipment objects, data sources, data modalities, fields, text slot or image region locations, basic quality attributes, original values ​​or quality index values, acquisition time or recording time, and contextual information. The equipment objects, data sources, field names, text slot names, image region types, and basic quality attributes within these quality evidence objects are standardized to eliminate naming differences across different systems, data sources, or recording methods.

[0036] S2.2, Grouping and Aggregation: After standardization, quality evidence objects are grouped and aggregated according to dimensions such as equipment object, data source, data modality, original location, basic quality attributes, and time range. Quality evidence from the same data source, the same type of field, or the same type of text slot is grouped into the same-source quality issue group; quality evidence from different data sources but pointing to the same power equipment object is associated based on equipment identifier, equipment name, ledger code, work order number, or context information and grouped into the cross-source quality issue group; quality evidence from structured data, text data, and image data that all point to the same equipment object or the same business event is associated based on equipment object, time window, and business context and grouped into the cross-modal quality issue group.

[0037] S2.3. Associate the grouped and aggregated quality evidence objects with the preset business constraints: The preset business constraints include at least one of the following: key fields should be complete; field values ​​should meet preset formats or value ranges; the identification of the same device should be consistent across different data sources; the collection time should meet business cycle requirements; text records should contain necessary slots; and equipment nameplate images or meter images should meet identifiable requirements. These preset business constraints originate from a pre-configured business constraint library, data dictionary, historical rule library, or manually confirmed business constraints. During association, the corresponding business constraint type is matched based on the data source, data modality, original location, basic quality attributes, and contextual information in the quality evidence object.

[0038] After completing the above grouping aggregation and business constraint association, a candidate quality issue group is formed. This candidate quality issue group represents a set of quality evidence objects with the same or similar basic quality attributes, business constraints, and data source characteristics, and can be represented as: ; in, Indicates the name of the candidate quality issue group. This represents the power equipment objects or set of equipment objects corresponding to the candidate quality problem group. This represents the set of relevant data sources. Represents the set of related data modes. This represents a collection of field, text slot, or image region locations. Indicates basic quality attributes. Indicates related business constraints. This represents the set of quality evidence objects contained in the candidate quality problem group. Indicates the corresponding time range or data batch. This indicates contextual information related to the candidate quality issue group.

[0039] After the above processing, step S2 outputs a set of candidate quality problem groups. Each candidate quality problem group includes a corresponding power equipment object or set of equipment objects, a set of related data sources, a set of related data modalities, fields, a set of text slots or image region locations, basic quality attributes, associated business constraints, a set of quality evidence objects, a time range, and contextual information. The set of candidate quality problem groups serves as input for subsequent repetitive quality problem pattern recognition and the construction of structured quality attribution evidence.

[0040] S3. Repetitive quality problem pattern recognition and determination of the object to be generated as a rule: Based on the candidate quality problem group set formed in step S2, repeatability analysis is performed on each candidate quality problem group to identify the quality problem patterns that recur in different equipment objects, different data batches, different time ranges or different data sources, and to determine the rule objects to be generated.

[0041] Specifically, each candidate quality issue group in the candidate quality issue group set is read. The candidate quality issue group includes power equipment objects or a set of equipment objects, a set of data sources, a set of data modalities, fields, a set of text slots or image region locations, basic quality attributes, associated business constraints, a set of quality evidence objects, a time range, and contextual information.

[0042] S3.1 Repeatability evaluation indicators: For each candidate quality problem group, the number of quality evidence objects, the number of involved equipment objects, the number of data batches involved, the number of occurrence time windows, the number of related data sources, and the concentration of original locations are statistically analyzed to obtain the repeatability evaluation index for that candidate quality problem group. The repeatability evaluation index is used to characterize the frequency, coverage, and persistence of the quality deviation phenomenon corresponding to that candidate quality problem group in the collected data.

[0043] In one alternative implementation, the repeatability evaluation index for the candidate quality problem group can be expressed as: ; in, Indicates the number of quality evidence objects in the candidate quality problem group; Indicates the number of device objects involved; Indicates the number of data batches involved; Indicates the number of time windows that occurred; Indicates the degree of concentration of quality evidence in fields, text slots, or image regions (dimensionless). This indicates the degree of consistency between quality evidence and related business constraints.

[0044] S3.2 Calculate the repeatability score of the candidate quality problem group based on the repeatability evaluation index. The repeatability score can be expressed as: ; in, to This indicates the weight corresponding to each repeatability evaluation indicator. The weights are configured based on business importance, historical data quality governance results, or preset rules. To facilitate comparisons between different indicators, each repeatability evaluation indicator is normalized before calculating the repeatability score.

[0045] When the repeatability score of a candidate quality problem group reaches a preset scoring threshold, or when one or more of its indicators such as the number of quality evidence objects, the number of equipment objects covered, the number of data batches covered, or the number of time windows reach the corresponding threshold, the candidate quality problem group is identified as a repeatability quality problem pattern.

[0046] After completing the pattern recognition of repetitive quality issues, a rule object to be generated is produced. The rule object to be generated includes a repetitive quality issue pattern identifier, associated candidate quality issue groups, associated business constraints, a set of quality evidence objects, common original locations, repetitiveness evaluation indicators, repetitiveness scores, and rule generation status.

[0047] After the above processing, step S3 outputs a set of rule objects to be generated. This set of rule objects records quality problem patterns that satisfy repeatability conditions, their associated candidate quality problem groups, business constraints, quality evidence object sets, and repeatability evaluation results, and is input into the subsequent structured quality attribution evidence construction step.

[0048] S4. Construction of structured quality attribution evidence for rule-based generation: like Figure 2 As shown, based on the set of rules to be generated determined in step S3, a set of structured quality attribution evidence for data quality rule generation is constructed.

[0049] Specifically, the associated candidate quality problem groups, associated business constraints, quality evidence object sets, common original locations, repeatability evaluation indicators, and repeatability scores are read from the rule object to be generated. Field extraction is performed on the quality evidence object set to extract the device object, data source, data modality, original location, basic quality attributes, original values ​​or quality indicator values, collection time or recording time, and contextual information from each quality evidence object.

[0050] S4.1 Calculation of the contribution of quality evidence features: Based on the field extraction results, a quality evidence feature set is constructed. The quality evidence features include at least one of the following: data source features, data modality features, original location features, basic quality attribute features, business constraint features, equipment object features, time window features, and original value performance features.

[0051] For any quality evidence feature in the quality evidence feature set, its feature contribution is calculated. This feature contribution is used to further decouple and precisely separate fundamental factor features from co-existing redundant features within the generated rule object, building upon the repetitive quality problem pattern recognition already achieved by S3. In multi-source, multi-modal power business scenarios, clustered data quality problems often trigger chain reactions and accompanying anomalies in multi-dimensional features (e.g., the systematic absence of a key parameter is often accompanied by fixed-frequency variations in non-key data fields). If all the variability features contained in the rule object to be generated are directly used as the rule construction benchmark, the subsequently generated rules will inevitably contain a large number of irrelevant and redundant logical conditions, resulting in rule overfitting, high redundancy, and poor business interpretability. Therefore, the feature contribution of a quality evidence feature can be expressed as: ; in, This indicates the number of items within the statistics window. The r-th quality evidence feature corresponding to each rule object to be generated; Indicates the frequency of occurrence after normalization; Indicates the coverage range of normalized equipment and data batches; Indicates the degree of concentration of positions after normalization; This indicates the degree of consistency of business constraints after normalization. , , , For the corresponding weights, and satisfying α + β + γ + δ =1.

[0052] When the feature contribution of a quality evidence feature reaches a preset contribution threshold, that quality evidence feature is identified as a common quality evidence feature. The set of common quality evidence features can be represented as: ; in, Indicates the first The set of common quality evidence features corresponding to each rule object to be generated This represents the contribution threshold.

[0053] S4.2 Adaptive threshold determination: In this embodiment, the determination of the thresholds does not rely on subjective human experience, but rather employs an adaptive window sliding statistical method based on the distribution of historically collected data quality characteristics for deterministic dynamic calculation. The feature contribution threshold in S4 is used as an example. Taking the determination process as an example, its adaptive calculation formula is as follows: ; in, This represents the arithmetic mean of the contribution scores of all extracted candidate quality evidence features within the currently selected historical statistical time window. This represents the standard deviation (mean square) of the contribution scores of all candidate quality evidence features within the statistical time window. It is manifested as an adjustable robustness coefficient, used for macroscopic fine-tuning based on the tolerance of the power grid business side to noise, and its value range is defined as [0.5, 1.5], with 1 being preferred in this embodiment.

[0054] By introducing an adaptive statistical mechanism based on mean and standard deviation, the threshold can dynamically adjust to changes in data quality and background noise under different power grid business environments. When the historical background data quality is poor and the noise is extremely high, the threshold automatically moves up to strictly control the entry of false rules into the database; when the overall data quality is good, the threshold automatically moves down to keenly capture minor, potential systemic defects, thereby ensuring the system's strong generalization and objective self-evolution capabilities in multimodal scenarios.

[0055] S4.3 Generation of Structured Quality Attribution Evidence Set: The data source fields and original location fields of each quality evidence object in the quality evidence object set are summarized to form an original data mapping set. For structured data, the original data mapping set points to the corresponding data table, field name, record row, or primary key identifier; for text data, the original data mapping set points to the corresponding document, paragraph, sentence, or text slot; for image data, the original data mapping set points to the corresponding image file, target region, OCR recognition result, or image quality index.

[0056] After determining the common quality evidence features and constructing the original data mapping set, a structured quality attribution evidence object is generated. This structured quality attribution evidence object can be represented as: ; in, This represents the corresponding rule object to be generated. Indicates the attribution object, This represents the set of relevant data sources. Represents the set of related data modes. Represents the set of common original locations. Indicates basic quality attributes. Indicates related business constraints. Represents a set of quality evidence objects. This represents a set of common quality evidence features. This represents the set of original data mapping relationships.

[0057] After the above processing, step S4 outputs a structured quality attribution evidence set. This structured quality attribution evidence set provides controlled input to subsequent data quality rule generation steps. The controlled input includes the applicable objects of the rules, the applicable locations of the rules, basic quality attributes, related business constraints, a set of quality evidence objects, common quality evidence features, and a set of original data mappings.

[0058] S5. Generation, validation, and storage of data quality rules based on structured quality attribution evidence: like Figure 3 As shown, based on the structured quality attribution evidence set constructed in step S4, candidate data quality rules are generated, and the candidate data quality rules are verified and stored in the database.

[0059] Specifically, the system reads the rule object to be generated, the attribution object, the set of relevant data sources, the set of relevant data modalities, the set of common original locations, the basic quality attributes, the associated business constraints, the set of quality evidence objects, the set of common quality evidence features, and the set of original data mappings from the structured quality attribution evidence object, and constructs rule generation prompts for the large model.

[0060] S5.1 Controlled conditional probability generation modeling of candidate data quality rules: Reading structured quality attribution evidence objects Then, it is assembled into rules to generate prompt information. Candidate data quality rules The generation process is strictly constrained by maximizing the following conditional probability distribution: ; in, This is the frozen parameter matrix of the large language model. (Hint / Prompt) The internal display contains the set of common quality evidence features extracted in step S4. The set of mapping relationships with the original data Under this constraint, the technical function of the large language model is strictly limited to "semantically aligning and grammatically translating structured common features and mapping relationships into power data quality standard expressions," thereby eliminating the model's creative freedom and ensuring the controllability of the generation rules.

[0061] The candidate data quality rules shall include at least the rule name, the object to which the rule applies, the source of the data to which the rule applies, the location to which the rule applies, the rule conditions, the rule judgment expression, the basic quality attributes, the business constraints, and the source of evidence.

[0062] In one optional implementation, the candidate data quality rules are represented using a key-value pair structure, a table structure, or a JSON structure, and their fields include at least: rule identifier field, rule name field, applicable object field, data source field, applicable location field, basic quality attribute field, business constraint field, rule condition field, judgment expression field, evidence source field, rule status field, and version field.

[0063] S5.2 Construction of Rule-Based Multidimensional Deterministic Verification Function and Adaptive Correction Feedback Loop: To ensure the accuracy and verifiability of the rules for inclusion in the database, a multi-dimensional verification function driven by deterministic code and a standard rule engine is constructed. Joint quantitative review of candidate rules: ; in, , , , These are the preset deterministic evaluation weight coefficients.

[0064] This is a syntax validation indicator operator. It outputs the rule string from the large model. The string is passed as input to the built-in AST compilation and parsing operator based on SQL / Python rules. This operator attempts to translate the string into a directed tree-structured syntax tree object. If, during parsing, semantic illusions in the large model lead to missing parentheses, misspelled keywords, or misaligned logical symbols, the parsing operator will directly trigger a compilation error or return an empty tree. In this case, the system automatically determines that the syntax is non-compliant. Otherwise, it is 1.

[0065] To achieve consistency scoring for business constraints, the system pre-stores the hard physical boundaries of the power grid as key-value pairs in a local configuration table, and uses string matching or regular expression extraction techniques to determine candidate rules. Whether it crosses the boundary or violates the hard safety and physical constraints of electricity, a perfect match is 1, and if there is a conflict, it is reduced to 0 according to the weight.

[0066] This refers to the sample replay hit rate. The system will use the current candidate rules. The data is dynamically injected into the temporary execution environment library. Subsequently, the system retrieves the complete set of quality evidence objects contained in the quality issue group bound to this issue group, as described in step S2. The execution engine is located in the dataset. The rule script is forced to run, and the number of abnormal data lines that the rule successfully intercepts and captures is counted and recorded. Finally, by calculating the ratio Obtain the hit rate score. If the hit rate is 100%, it means that the candidate rule can cover the quality deviation phenomena corresponding to the current set of quality evidence objects.

[0067] This serves as an operator to indicate historical rule conflicts. The system abstracts historically published rules into a directed topological graph of cause and effect, where nodes represent power fields and actions represent logical relationships. When a new rule... Upon input, the preceding and following nodes are extracted, and the topology graph is searched for reverse edges or paradoxical loops under the same preceding path. If semantic mutual exclusion is detected in the logical closed loop through graph path traversal, a rule conflict is determined to have occurred. Otherwise, it is 0.

[0068] The system adaptively determines the target threshold using the method in step S4.2. .like If so, the candidate rule will be activated as the target data quality rule and officially added to the database. The system automatically assembles deterministic rule correction feedback information based on the scoring defect items. This function either forces the large model to be regenerated or triggers a manual review protection mechanism after reaching the maximum number of corrections. This multidimensional deterministic verification function is used to reduce the uncertainty of the generated results from the large language model and to effectively verify the quality rules of the candidate data.

[0069] Once a candidate data quality rule passes validation, it is designated as the target data quality rule and written into the data quality rule library. During the library entry process, the association between the target data quality rule and the structured quality attribution evidence object, the set of quality evidence objects, the original data mapping set, and historical version information is established. Candidate data quality rules that fail validation and reach the preset maximum number of corrections are not written into the formal rule library, and their validation failure reasons, correction process, and manual review results are recorded in the rule generation log.

[0070] After the above processing, step S5 outputs a target data quality rule set. This target data quality rule set is used for quality verification, rule reuse, and rule maintenance of subsequent multi-modal power data acquisition.

[0071] In a specific implementation case, taking the generation of key field integrity rules for transformer equipment ledgers as an example, such as... Figure 4 As shown, this embodiment uses the generation of key field integrity rules in the transformer equipment ledger table T_ACCOUNT_2026_06 as an example to illustrate the automated generation process of data quality rules in this invention. The figure only shows a portion of the ledger data for one data batch within the statistical window; the actual statistical window is from May 2026 to June 2026.

[0072] Construction of quality evidence objects: The system retrieves transformer equipment ledger data and parses fields such as equipment number, equipment name, capacity, voltage level, and commissioning time. In the example data, the capacity field for equipment T-A001 is empty, the voltage level field for equipment T-B002 is empty, and the capacity field for equipment T-C003 is empty. Due to the historical system's default mechanism that triggers when fields are missing, the above three records, while lacking key fields, have their data collection point information fields automatically rewritten by the system to the fixed string SYS_ERR_BACK. Based on this, the system generates quality evidence objects Q1, Q2, and Q3, recording the corresponding equipment, data source, field location, basic quality attributes, and original value status, respectively.

[0073] Formation of candidate quality problem groups: The system groups and aggregates Q1, Q2, and Q3 according to data source, field location, basic quality attributes, and business constraints, and associates them with the business constraint "key ledger fields should be complete" to form the "equipment ledger key field completeness issue group".

[0074] The rule objects to be generated have been determined: The system performs repeatability identification on the candidate quality problem group. In this embodiment, the number of quality evidence N=3, the number of devices involved D=3, and the number of data batches involved B=2, thus the repeatability score Score is calculated to be 0.82. Based on the adaptive threshold determination method, the current preset threshold is 0.75. Therefore, the repeatability score is greater than the threshold, and the system determines that this phenomenon belongs to a recurring, systematic quality defect in the dataset, successfully eliminating its suspicion as random, single-point, occasional noise. Therefore, it is identified as a rule object to be generated.

[0075] Construction of structured quality attribution evidence: The system constructs structured quality attribution evidence based on the rule object to be generated. The rule object is read, and multi-dimensional feature extraction is performed. The constructed quality evidence feature set includes: feature f1 (missing capacity field), feature f2 (missing voltage level field), and feature f3 (collection point information represented as SYS_ERR_BACK). The feature contribution scores are calculated for each feature. The results are: 0.91 for the missing capacity field, 0.76 for the missing voltage level field, and f3, as a redundant feature resulting from the chain reaction with the system's default mechanism, has a low contribution score of only 0.43 because it occasionally appears in broader batches of normal data and has a low coverage weight.

[0076] At this point, the system calls the adaptive threshold determination method to calculate the arithmetic mean of the contribution of all candidate features in the current statistical window, which is 0.6, the standard deviation is 0.2, and the robustness coefficient is 0.5. Therefore, the current common feature contribution threshold is 0.7.

[0077] By comparing thresholds, the contribution of missing capacity and voltage level fields both exceeded the thresholds, and the system successfully extracted them as indispensable common quality evidence features. The contribution of f3 was below the threshold, and it was only a 'co-existing redundant feature' with statistical correlation rather than causal necessity, and it was forcibly removed.

[0078] In the final generated structured quality attribution evidence object, the common quality evidence feature set only contains features f1 and f2, which, together with the constructed original data mapping set (Q1 corresponds to the capacity field in the first row, Q2 corresponds to the voltage level field in the second row, and Q3 corresponds to the capacity field in the third row), are passed backward as input.

[0079] Target data quality rule generation and validation: Based on the above-mentioned decoupled and simplified structured attribution evidence, the system constructs prompt information input to the large model. The large model is constrained by a rigid skeleton, eliminating the illusion space and not writing redundant collection point information into the conditions, generating a structured candidate rule R_ACC_001 (the judgment expression is: if the capacity is empty or the voltage level is empty, then trigger the abnormality of the integrity verification of the key field of the ledger).

[0080] Subsequently, the system calls R_ACC_001 to perform a joint review and calculation based on general software engineering principles: Syntax validation: The built-in Python ast parsing component was used to parse the symbol tree of the regular expression. No compilation errors occurred, and the syntax was fully compliant. ; Business constraint verification: Applicable fields and judgment expressions are extracted from candidate rules and compared for consistency with the business constraint "Key ledger fields should be complete." The verification results show that the capacity (kVA) and voltage level fields are both preset key ledger fields, and the rule judgment expression is used to determine whether these fields are empty, thus meeting the key field integrity constraint. ; Sample replay verification: The verification rules are dynamically compiled into an executable script, and the script is replayed and run to the original multimodal anomaly sample dataset included in step S2. In the process, all three original abnormal data rows were successfully intercepted and accurately captured, achieving a hit rate of [missing information]. ; Historical rule comparison: Rules were injected into a directed topological conflict detection matrix in the form of cause-and-effect logical chains for graph path traversal. No reverse paradoxical cycles were found, indicating no logical conflict with existing rules. .

[0081] Set the weights for each evaluation item = = = =1, and the result of the quantitative score calculation by substituting into the system's built-in multidimensional deterministic verification function is 4. The system sets a safe pass target threshold. The quantitative calculation score was 4 > 2.5. The verification results showed that Q1, Q2 and Q3 were all hit, so they were written into the data quality rule base, and the correlation between this rule and the structured quality attribution evidence and the original data mapping set was established.

[0082] Example 2: This embodiment provides a power multimodal data quality rule generation system, including: The data acquisition module is configured to acquire structured data, text data, and image data related to power equipment. The quality evidence object set determination module is configured to convert structured data, text data, and image data into a quality evidence object set containing device object, data source, data modality, original location, basic quality attributes, and contextual information. The module for determining the rule objects to be generated is configured to: group and aggregate the quality evidence objects based on the set of quality evidence objects; associate the grouping and aggregation results with preset business constraints to form a set of candidate quality problem groups; and perform repeatability analysis on each candidate quality problem group to determine the rule objects to be generated. The structured quality attribution evidence object determination module is configured to: construct a structured quality attribution evidence set based on the set of rule objects to be generated; extract fields from the quality evidence object set to construct a quality evidence feature set; determine the feature contribution of any quality evidence in the quality evidence feature set; determine common quality evidence features based on a comparison between the feature contribution and a preset contribution threshold; and generate structured quality attribution evidence objects based on the common quality evidence features and the original data mapping set. The verification module is configured to: generate candidate data quality rules based on the structured quality attribution evidence object; and verify the candidate data quality rules.

[0083] This embodiment provides a power multimodal data quality rule generation system, which is used to execute the method described in Embodiment 1.

[0084] Example 3: This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the power multimodal data quality rule generation method described in Embodiment 1.

[0085] Example 4: This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of the power multimodal data quality rule generation method described in Embodiment 1.

[0086] Example 5: This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the power multimodal data quality rule generation method described in Embodiment 1.

Claims

1. A method for power multi-modal data quality rule generation, characterized in that, include: Acquire structured data, text data, and image data related to power equipment; Structured data, text data, and image data are transformed into a set of quality evidence objects containing device objects, data sources, data modalities, original locations, basic quality attributes, and contextual information. Based on the aforementioned set of quality evidence objects, the quality evidence objects are grouped and aggregated. The grouping and aggregation results are associated with preset business constraints to form a set of candidate quality problem groups; repeatability analysis is performed on each candidate quality problem group to determine the objects to be generated as rules. A structured quality attribution evidence set is constructed based on the set of rules to be generated; a quality evidence feature set is constructed by extracting fields from the quality evidence object set; the feature contribution degree of any quality evidence in the quality evidence feature set is determined; common quality evidence features are determined by comparing the feature contribution degree with a preset contribution degree threshold; and structured quality attribution evidence objects are generated based on the common quality evidence features and the original data mapping set. Candidate data quality rules are generated based on the structured quality attribution evidence object; the candidate data quality rules are then validated.

2. The electric power multi-modal data quality rule generation method of claim 1, wherein, The structured data includes equipment ledgers, operational measurements, status monitoring, and defect work orders; the text data includes inspection records, maintenance records, defect descriptions, test reports, and work order remarks; and the image data includes equipment nameplate images, field equipment images, infrared images, and meter images.

3. The electric power multi-modal data quality rule generation method of claim 1, wherein, The grouping and aggregation includes: after normalizing the quality evidence objects, quality evidence from the same data source, the same type of field, or the same type of text slot is grouped into the same-source quality problem group; quality evidence from different data sources but pointing to the same power equipment object is grouped into the cross-source quality problem group; and quality evidence from structured data, text data, and image data that all point to the same equipment object or the same business event is grouped into the cross-modal quality problem group.

4. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The determination of the rule object to be generated includes: when one or more of the following indicators reach the corresponding threshold: the number of quality evidence objects, the number of device objects covered, the number of data batches covered, and the number of time windows, the corresponding candidate quality problem group is determined as a repetitive quality problem pattern, and the rule object to be generated is generated.

5. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The quality evidence features include at least one of the following: data source features, data modality features, original location features, basic quality attribute features, business constraint features, equipment object features, time window features, and original value performance features.

6. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The feature contribution for: ； in, This indicates the number of items within the statistics window. The r-th quality evidence feature corresponding to each rule object to be generated; Indicates the frequency of occurrence after normalization; Indicates the coverage range of normalized equipment and data batches; Indicates the degree of concentration of positions after normalization; This indicates the degree of consistency of business constraints after normalization. , , , For the corresponding weights.

7. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The feature contribution threshold Dynamically based on the adaptive window sliding statistics method according to the distribution of historical data quality characteristics: ； in, This represents the arithmetic mean of the contribution scores of all extracted candidate quality evidence features within the currently selected historical statistical time window. denoted as the standard deviation of the contribution scores of all candidate quality evidence features within the statistical time window; k represents the adjusted robustness coefficient.

8. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The candidate data quality rules include rule name, applicable objects, applicable data sources, applicable locations, rule conditions, rule judgment expressions, basic quality attributes, business constraints, and evidence sources.

9. The method for generating power multimodal data quality rules as described in claim 1, characterized in that, The verification of the candidate data quality rules includes: constructing a multi-dimensional verification function to jointly and quantitatively review the candidate rules; if the value of the multi-dimensional verification function is greater than or equal to the target threshold, the candidate rule is activated as the target data quality rule and formally entered into the database; otherwise, it is corrected; the multi-dimensional verification function... for: ； in, , , , These are the pre-set deterministic evaluation weighting coefficients; For syntax checking indicator operators; Scoring for consistency of business constraints; For sample replay hit rate; Operator for indicating conflicts in historical rules.

10. A power multimodal data quality rule generation system, characterized in that, include: The data acquisition module is configured to acquire structured data, text data, and image data related to power equipment. The quality evidence object set determination module is configured to convert structured data, text data, and image data into a quality evidence object set containing device object, data source, data modality, original location, basic quality attributes, and contextual information. The module for determining the rule objects to be generated is configured to: group and aggregate the quality evidence objects based on the set of quality evidence objects; associate the grouping and aggregation results with preset business constraints to form a set of candidate quality problem groups; and perform repeatability analysis on each candidate quality problem group to determine the rule objects to be generated. The structured quality attribution evidence object determination module is configured to: construct a structured quality attribution evidence set based on the set of rule objects to be generated; extract fields from the quality evidence object set to construct a quality evidence feature set; determine the feature contribution of any quality evidence in the quality evidence feature set; determine common quality evidence features based on a comparison between the feature contribution and a preset contribution threshold; and generate structured quality attribution evidence objects based on the common quality evidence features and the original data mapping set. The verification module is configured to: generate candidate data quality rules based on the structured quality attribution evidence object; and verify the candidate data quality rules.

Images