Multi-source heterogeneous power measurement data real-time standardization processing method

Abstract

The invention discloses a real-time standardization processing method for multi-source heterogeneous power measurement data, and the method comprises the following steps: S1, receiving the multi-source heterogeneous power measurement data, carrying out the cleaning and conversion of the multi-source heterogeneous power measurement data, and carrying out the classification processing of the multi-source heterogeneous power measurement data; s2, presetting a data standardization processing model, and importing the classified multi-source heterogeneous power measurement data into the data standardization processing model; s3, comparing the multi-source heterogeneous power measurement data with the data in the data standardization processing model; s4, performing similar data analysis on the multi-source heterogeneous power measurement data, and performing marking processing on unsimilar data to obtain marking information; s5, extracting mark information, and performing secondary mark processing on key information in the mark information to obtain secondary mark information; and S6, exporting the secondary marking information, and carrying out exception calculation processing. Information can be mined accurately, and electric power safety accidents are avoided.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a real-time standardized processing method for multi-source heterogeneous electric power measurement data. Background Art

[0002] With the development of power network informatization, information technology has penetrated into every link of power production. The power information system will generate a large amount of log data during operation, including system operation logs such as system network connection status and database status, as well as business-related logs such as equipment operation parameters.

[0003] If the analysis and evaluation of these massive logs are not timely and the information mining is inaccurate, serious power safety accidents may occur. Therefore, a real-time standardized processing method for multi-source heterogeneous power measurement data is proposed to solve the above problems. Summary of the invention

[0004] Based on the technical problems existing in the background technology, the present invention proposes a real-time standardized processing method for multi-source heterogeneous power measurement data, which can accurately mine information and avoid the occurrence of power safety accidents.

[0005] The present invention proposes a method for real-time standardization processing of multi-source heterogeneous power measurement data, comprising the following steps:

[0006] S1: receiving multi-source heterogeneous power measurement data, cleaning and converting the multi-source heterogeneous power measurement data, and then classifying and processing the multi-source heterogeneous power measurement data;

[0007] S2: Preset a data standardization processing model and import the classified multi-source heterogeneous power measurement data into the data standardization processing model;

[0008] S3: Compare and process the multi-source heterogeneous power measurement data with the data in the data standardization processing model;

[0009] S4: Perform similar data analysis on multi-source heterogeneous power measurement data, and mark dissimilar data to obtain marking information;

[0010] S5: extracting the marked information, performing secondary marking processing on the key information in the marked information, and obtaining secondary marked information;

[0011] S6: Export the secondary marking information and perform abnormal calculation processing.

[0012] In a preferred embodiment, in step S1, the optimal features of multi-source heterogeneous power measurement data are obtained, and the optimal features of the multi-source heterogeneous power measurement data are imported into a PLS-logic model to obtain classification results of the multi-source heterogeneous power measurement data.

[0013] In a preferred embodiment, in step S2, the data standardization processing model is obtained by training based on the first sample data, the first sample training probability corresponding to the first sample data, the second sample data, and the second sample training probability corresponding to the second sample data.

[0014] In a preferred embodiment, in step S3, when comparing the multi-source heterogeneous power measurement data with the data in the data standardization processing model, common data segments in the multi-source heterogeneous power measurement data and the data standardization processing model are selected and compared.

[0015] In a preferred embodiment, in step S4, when performing similar data analysis on multi-source heterogeneous power measurement data, firstly, the main similar data is compared, then the sub-similar data is compared, and finally, the dissimilar data is extracted for marking processing.

[0016] In a preferred embodiment, in step S5, the marked information is identified to obtain key information in the marked information, and then the key information is subjected to secondary marking processing.

[0017] In a preferred embodiment, in step S6, the amount of secondary marking information data is counted, set as S1, and the calculation formula is:

[0018]

[0019] Among them, the difference ratio is marked as X i , i=1, 2, 3, 4, ..., n, n is a positive integer;

[0020] is the average of the difference proportions;

[0021] A warning threshold is preset, marked as S.

[0022] In a preferred embodiment, when S1 is less than S, normal information is output;

[0023] When S1 is greater than or equal to S, output abnormal information;

[0024] After the abnormal information is output, abnormal alarm processing will be performed.

[0025] Beneficial effects of the present invention:

[0026] (1) The present invention proposes a real-time standardization processing method for multi-source heterogeneous power measurement data. Before classifying and processing the multi-source heterogeneous power measurement data, the multi-source heterogeneous power measurement data must first be cleaned and converted, and redundant data in the multi-source heterogeneous power measurement data can be filtered, and data with non-uniform formats of the multi-source heterogeneous power measurement data can be converted into data of the same format, so as to facilitate subsequent standardization processing of the multi-source heterogeneous power measurement data;

[0027] (2) The present invention proposes a real-time standardization processing method for multi-source heterogeneous power measurement data, which can effectively improve the accuracy of subsequent analysis by marking dissimilar data, identifying the marked information, obtaining key information in the marked information, and then performing secondary marking processing on the key information;

[0028] (3) The present invention proposes a real-time standardized processing method for multi-source heterogeneous power measurement data. By statistically analyzing the difference rate of secondary marking information and then comparing it with the warning threshold, the multi-source heterogeneous power measurement data is subjected to standardized comparative processing. After outputting abnormal information, a timely alarm is issued to remind the staff to handle it in time to avoid serious power safety accidents. DETAILED DESCRIPTION

[0029] The present invention will be further explained below in conjunction with specific embodiments.

[0030] This embodiment proposes a real-time standardization processing method for multi-source heterogeneous power measurement data, including the following steps:

[0031] S1: receiving multi-source heterogeneous power measurement data, cleaning and converting the multi-source heterogeneous power measurement data, and then classifying and processing the multi-source heterogeneous power measurement data;

[0032] S2: Preset a data standardization processing model and import the classified multi-source heterogeneous power measurement data into the data standardization processing model;

[0033] S3: Compare and process the multi-source heterogeneous power measurement data with the data in the data standardization processing model;

[0034] S4: Perform similar data analysis on multi-source heterogeneous power measurement data, and mark dissimilar data to obtain marking information;

[0035] S5: extracting the marked information, performing secondary marking processing on the key information in the marked information, and obtaining secondary marked information;

[0036] S6: Export the secondary marking information and perform abnormal calculation processing.

[0037] It should be noted that there are two main types of multi-source heterogeneous data collected in power transmission lines: one is real-time data from various monitoring systems, and the other is historical data from various reports, applications, and sensor devices. These data include but are not limited to transmission line telemetry data, smart device logs, inspection reports, and user survey reports. In the data collection stage, the key technologies for processing these multi-source heterogeneous data include data preprocessing, data cleaning, data conversion, and data standardization.

[0038] Furthermore, in step S1, optimal features of the multi-source heterogeneous power measurement data are obtained, and the optimal features of the multi-source heterogeneous power measurement data are imported into the PLS-logic model to obtain classification results of the multi-source heterogeneous power measurement data.

[0039] Through the above content, before classifying and processing the multi-source heterogeneous power measurement data, the multi-source heterogeneous power measurement data must first be cleaned and converted, the redundant data in the multi-source heterogeneous power measurement data can be filtered, and the data with inconsistent formats of the multi-source heterogeneous power measurement data can be converted into data of the same format, which is convenient for the subsequent standardized processing of the multi-source heterogeneous power measurement data.

[0040] Furthermore, in step S2, the data standardization processing model is obtained by training based on the first sample data, the first sample training probability corresponding to the first sample data, the second sample data, and the second sample training probability corresponding to the second sample data.

[0041] Through the above content, the data standardization processing model is obtained.

[0042] Furthermore, in step S3, when comparing the multi-source heterogeneous power measurement data with the data in the data standardization processing model, common data segments in the multi-source heterogeneous power measurement data and the data standardization processing model are selected and compared.

[0043] Through the above content, the comparison efficiency of multi-source heterogeneous power measurement data and data in the data standardization processing model can be improved.

[0044] Furthermore, in step S4, when performing similar data analysis on multi-source heterogeneous power measurement data, firstly, the main similar data is compared, then the sub-similar data is compared, and finally, the dissimilar data is extracted for marking processing.

[0045] Through the above content, the accuracy of similar data analysis in multi-source heterogeneous power measurement data can be improved.

[0046] Furthermore, in step S5, the marked information is identified to obtain key information in the marked information, and then a secondary marking process is performed on the key information.

[0047] Through the above content, the marking effect is improved.

[0048] Furthermore, in step S6, the secondary marking information difference rate is counted and set as S1, and the calculation formula is:

[0049]

[0050] Among them, the difference ratio is marked as X i , i=1, 2, 3, 4, ..., n, n is a positive integer;

[0051] is the average of the difference proportions;

[0052] A warning threshold is preset, marked as S.

[0053] Through the above content, by statistically analyzing the difference rate of secondary marking information and then comparing it with the warning threshold, the multi-source heterogeneous power measurement data can be standardized and compared.

[0054] Furthermore, when S1 is less than S, normal information is output;

[0055] When S1 is greater than or equal to S, output abnormal information;

[0056] After the abnormal information is output, abnormal alarm processing will be performed.

[0057] Through the above content, after the abnormal information is output, a timely alarm will be issued to remind the staff to deal with it in time to avoid serious power safety accidents.

[0058] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0059] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0060] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0062] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0065] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0066] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A real-time standardization processing method for multi-source heterogeneous power measurement data, characterized in that: The following steps are involved: S1: receiving multi-source heterogeneous power measurement data, cleaning and converting the multi-source heterogeneous power measurement data, and then classifying and processing the multi-source heterogeneous power measurement data; S2: Preset a data standardization processing model and import the classified multi-source heterogeneous power measurement data into the data standardization processing model; S3: Compare and process the multi-source heterogeneous power measurement data with the data in the data standardization processing model; S4: Perform similar data analysis on multi-source heterogeneous power measurement data, and mark dissimilar data to obtain marking information; S5: extracting the marked information, performing secondary marking processing on the key information in the marked information, and obtaining secondary marked information; S6: Export the secondary marking information and perform abnormal calculation processing.

2. A real-time standardization processing method for multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S1, the optimal features of the multi-source heterogeneous power measurement data are obtained, and the optimal features of the multi-source heterogeneous power measurement data are imported into the PLS-logistic model to obtain the classification results of the multi-source heterogeneous power measurement data.

3. A real-time standardization processing method for multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S2, the data standardization processing model is trained based on the first sample data, the first sample training probability corresponding to the first sample data, the second sample data, and the second sample training probability corresponding to the second sample data.

4. A real-time standardization processing method for multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S3, when comparing the multi-source heterogeneous power measurement data with the data in the data standardization processing model, common data segments in the multi-source heterogeneous power measurement data and the data standardization processing model are selected and compared.

5. A real-time standardization processing method for multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S4, when performing similar data analysis on multi-source heterogeneous power measurement data, firstly, the main similar data is compared, then the sub-similar data is compared, and finally, the dissimilar data is extracted for marking processing.

6. A method for real-time standardization of multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S5, the marked information is identified to obtain key information in the marked information, and then a secondary marking process is performed on the key information.

7. A real-time standardization processing method for multi-source heterogeneous power measurement data according to claim 1, characterized in that: In step S6, the amount of secondary marking information data is counted and set to S1. The calculation formula is: Among them, the difference ratio is marked as X i , i=1, 2, 3, 4, ..., n, n is a positive integer; is the average of the difference proportions; A warning threshold is preset, marked as S.

8. A method for real-time standardization of multi-source heterogeneous power measurement data according to claim 7, characterized in that: When S1 is less than S, normal information is output; When S1 is greater than or equal to S, output abnormal information; After the abnormal information is output, abnormal alarm processing will be performed.

Images