Data quality management method and platform based on converter station equipment

Abstract

The invention relates to the technical field of power system management, in particular to a data quality management method and platform based on converter station equipment. The method comprises the following steps: S1, collecting multiple types of data of remote measurement, remote signaling, fault recording and video monitoring generated by converter station equipment, and preprocessing the data; s2, designing a database table model and a filling template according to a data preprocessing result, constructing a data resource directory, carding a data quality checking rule, and performing data processing on the basic data by adopting a machine learning algorithm; and S3, setting monthly and annual data analysis index calculation rules and access logic in combination with a data processing result, constructing a multi-dimensional composite index, presenting a station end data quality panorama condition through a data visualization technology, and automatically uploading data. The invention aims to provide a data quality management method and platform based on converter station equipment, so as to realize efficient management of data and improve data quality and management efficiency.

Description

Technical Field

[0001] This invention relates to the field of power system management technology, and in particular to a data quality management method and platform based on converter station equipment. Background Technology

[0002] Converter stations are key facilities in high-voltage direct current (HVDC) transmission systems, enabling the conversion between AC and DC power. Their main equipment includes converter valves, converter transformers, smoothing reactors, AC filters, and reactive power compensation devices. Their core functions are achieved through the converter unit, with thyristor converter valves becoming the mainstream equipment due to their large capacity and high reliability. Data quality management based on converter station equipment refers to managing the entire lifecycle of data acquisition, integration, cleaning, verification, storage, analysis, and transmission to ensure data accuracy, integrity, consistency, and reliability, providing data support for converter station equipment operation and maintenance, decision analysis, and other related processes.

[0003] The current data management methods for converter station equipment have the following shortcomings:

[0004] 1. Converter station equipment generates massive amounts of data during operation, covering various types, including telemetry, remote signaling, fault recording, and video monitoring. Existing management methods are difficult to efficiently and accurately collect and integrate data from different data sources, resulting in prominent issues such as data format conversion, field mapping, and duplicate processing.

[0005] 2. The converter station involves numerous information systems with different styles and operating methods, which makes data sharing between systems difficult, data quality inconsistent, and data dimensions inconsistent.

[0006] Therefore, designing a data quality management method and platform based on converter station equipment that can solve the above-mentioned technical problems is a technical issue that needs to be addressed. Summary of the Invention

[0007] To address the aforementioned issues, the present invention aims to provide a data quality management method and platform based on converter station equipment, thereby achieving efficient data management and improving data quality and management efficiency.

[0008] In a first aspect, the present invention provides a data quality management method based on converter station equipment, the method comprising the following steps:

[0009] Step S1: Collect various types of data generated by the converter station equipment, including telemetry, telesignaling, fault recording, and video monitoring, and preprocess the data;

[0010] Step S2: Based on the data preprocessing results, design the database table model and data entry template, construct the data resource catalog, sort out the data quality verification rules, and use machine learning algorithms to process the basic data;

[0011] Step S3: Based on the data processing results, set the calculation rules and data retrieval logic for monthly and annual data analysis indicators, construct multi-dimensional composite indicators, and present the overall situation of station data quality through data visualization technology and automatically upload the data;

[0012] Step S4: Compress and encrypt the processed data and perform data quality verification before uploading.

[0013] Furthermore, in step S1, the data preprocessing specifically includes the following steps:

[0014] Step S11: Use an ETL conversion tool to convert the data format, and convert the unstructured video source data and heterogeneous device protocol data into a structured data format.

[0015] Step S12: Use a mapping rule engine to map fields and establish the relationship between fields from different data sources and fields in the standard data model;

[0016] Step S13: Use the IQR method to clean the data, identify and remove outliers that exceed reasonable thresholds;

[0017] Step S14: Perform duplicate data processing and delete duplicate and redundant data.

[0018] Furthermore, in step S2, the database table model includes equipment professional ledger data and measurement data.

[0019] Furthermore, in step S2, machine learning algorithms are used to process the basic data. Specific steps include:

[0020] Step S21: Use the K-means clustering algorithm to integrate the data, and synthesize the scattered multi-source data into a unified data cluster based on the device type and data acquisition time characteristics;

[0021] Step S22: Use the PCA algorithm to transform the data, reduce the dimensionality of high-latitude measurement data, and retain key features;

[0022] Step S23: Perform intelligent cleaning and transformation on key features, build an anomaly detection model by training historical data, and automatically identify and correct data deviations.

[0023] Furthermore, in step S3, a multidimensional composite index is constructed, specifically including data quality trend analysis and variable dimension analysis.

[0024] Secondly, the present invention provides a data quality management method based on converter station equipment, which includes a data acquisition module, a data source management module, a multi-dimensional data operation management module, and a data transmission module.

[0025] The data acquisition module is used to collect data from various types of equipment in the converter station and perform data preprocessing.

[0026] The data source management module is used to design database table models and data entry templates, build a data resource catalog, and organize data quality verification rules.

[0027] The multi-dimensional operation management module is used to perform data preprocessing and design data analysis indicators;

[0028] The data transmission module is used to compress, encrypt, and transmit data.

[0029] The present invention has the following beneficial effects:

[0030] 1- This invention sorts out data quality verification rules, uses machine learning algorithms to process basic data, constructs multi-dimensional composite indicators, and presents a panoramic view of station data quality through data visualization technology, ensuring the accuracy of decision-making basis and thus improving the scientific nature of converter station equipment operation and maintenance and management decisions.

[0031] 2. This invention preprocesses various types of data generated by converter station equipment and designs and sorts out data quality verification rules to minimize errors in the data acquisition and transmission process, eliminate inaccurate, inconsistent and incomplete data, reduce erroneous decisions caused by data defects, and ensure the safe and stable operation of the converter station. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of Embodiment 1 of the present invention;

[0033] Figure 2 This is a schematic diagram of Embodiment 2 of the present invention. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0035] Example 1

[0036] like Figure 1 As shown, a data quality management method based on converter station equipment is proposed, which includes the following steps:

[0037] Step S1: Collect various types of data generated by the converter station equipment, including telemetry, telesignaling, fault recording, and video monitoring, and preprocess the data;

[0038] Step S2: Based on the data preprocessing results, design the database table model and data entry template, construct the data resource catalog, sort out the data quality verification rules, and use machine learning algorithms to process the basic data;

[0039] Step S3: Based on the data processing results, set the calculation rules and data retrieval logic for monthly and annual data analysis indicators, construct multi-dimensional composite indicators, and present the overall situation of station data quality through data visualization technology and automatically upload the data;

[0040] Step S4: Compress and encrypt the processed data and perform data quality verification before uploading.

[0041] Furthermore, in step S1, the data preprocessing specifically includes the following steps:

[0042] Step S11: Use an ETL conversion tool to convert the data format, and convert the unstructured video source data and heterogeneous device protocol data into a structured data format.

[0043] Step S12: Use a mapping rule engine to map fields and establish the relationship between fields from different data sources and fields in the standard data model;

[0044] Step S13: Use the IQR method to clean the data, identify and remove outliers that exceed reasonable thresholds;

[0045] Step S14: Perform duplicate data processing and delete duplicate and redundant data.

[0046] Furthermore, in step S2, the database table model includes equipment professional ledger data and measurement data.

[0047] Furthermore, in step S2, machine learning algorithms are used to process the basic data. Specific steps include:

[0048] Step S21: Use the K-means clustering algorithm to integrate the data, and synthesize the scattered multi-source data into a unified data cluster based on the device type and data acquisition time characteristics;

[0049] Step S22: Use the PCA algorithm to transform the data, reduce the dimensionality of high-latitude measurement data, and retain key features;

[0050] Step S23: Perform intelligent cleaning and transformation on key features, build an anomaly detection model by training historical data, and automatically identify and correct data deviations.

[0051] Furthermore, in step S3, a multidimensional composite index is constructed, specifically including data quality trend analysis and variable dimension analysis.

[0052] Example 2

[0053] like Figure 2 As shown, a data quality management method based on converter station equipment is proposed. The solution includes a data acquisition module, a data source management module, a multi-dimensional data operation management module, and a data transmission module.

[0054] The data acquisition module is used to collect data from various types of equipment in the converter station and perform data preprocessing.

[0055] The data source management module is used to design database table models and data entry templates, build a data resource catalog, and organize data quality verification rules.

[0056] The multi-dimensional operation management module is used to perform data preprocessing and design data analysis indicators;

[0057] The data transmission module is used to compress, encrypt, and transmit data.

[0058] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A data quality management method based on converter station equipment, characterized in that, Includes the following steps: Step S1: Collect various types of data generated by the converter station equipment, including telemetry, telesignaling, fault recording, and video monitoring, and preprocess the data; Step S2: Based on the data preprocessing results, design the database table model and data entry template, construct the data resource catalog, sort out the data quality verification rules, and use machine learning algorithms to process the basic data; Step S3: Based on the data processing results, set the calculation rules and data retrieval logic for monthly and annual data analysis indicators, construct multi-dimensional composite indicators, and present the overall situation of station data quality through data visualization technology and automatically upload the data; Step S4: Compress and encrypt the processed data and perform data quality verification before uploading.

2. The data quality management method based on converter station equipment according to claim 1, characterized in that: In step S1, the data preprocessing specifically includes the following steps: Step S11: Use an ETL conversion tool to convert the data format, and convert the unstructured video source data and heterogeneous device protocol data into a structured data format. Step S12: Use a mapping rule engine to map fields and establish the relationship between fields from different data sources and fields in the standard data model; Step S13: Use the IQR method to clean the data, identify and remove outliers that exceed reasonable thresholds; Step S14: Perform duplicate data processing and delete duplicate and redundant data.

3. The data quality management method based on converter station equipment according to claim 1, characterized in that: In step S2, the database table model includes equipment professional ledger data and measurement data.

4. The data quality management method based on converter station equipment according to claim 1, characterized in that: In step S2, machine learning algorithms are used to process the basic data. Specific steps include: Step S21: Use the K-means clustering algorithm to integrate the data, and synthesize the scattered multi-source data into a unified data cluster based on the device type and data acquisition time characteristics; Step S22: Use the PCA algorithm to transform the data, reduce the dimensionality of high-latitude measurement data, and retain key features; Step S23: Perform intelligent cleaning and transformation on key features, build an anomaly detection model by training historical data, and automatically identify and correct data deviations.

5. The data quality management method based on converter station equipment according to claim 1, characterized in that: In step S3, a multidimensional composite index is constructed, which specifically includes data quality trend analysis and variable dimension analysis.

6. A data quality management platform based on converter station equipment, characterized in that: It includes a data acquisition module, a data source management module, a multi-dimensional data operation management module, and a data transmission module; The data acquisition module is used to collect data from various types of equipment in the converter station and perform data preprocessing. The data source management module is used to design database table models and data entry templates, build a data resource catalog, and organize data quality verification rules. The multi-dimensional operation management module is used to perform data preprocessing and design data analysis indicators; The data transmission module is used to compress, encrypt, and transmit data.

Images