A method, system and medium based on multi-protocol power terminal equipment data association

Abstract

The present application relates to a method, system and medium based on multi-protocol power terminal equipment data association, the method comprising the following steps: 1) collecting data of various types of power communication equipment and preprocessing the data; 2) using time series data to store the collected data and build a data model and structure; 3) using data fusion technology to integrate data from different devices and establish the association between devices; 4) using Siamese neural network algorithm to learn the association between data of different protocols; 5) performing similarity measurement on new data and identifying unusual patterns and behaviors. The present invention has practical application value in network security monitoring, traffic management and other aspects by helping to identify and classify communication data of different protocols.

Description

Technical Field

[0001] The present application relates to the field of power system security, and in particular to a method, system and medium based on multi-protocol power terminal equipment data association. Background Art

[0002] With the development of technology, there are multiple communication protocols in the power system, such as Modbus, DNP3, IEC 61850, etc. These protocols usually collect and transmit data in different formats and structures. Data fusion technology covers protocol parsing, data conversion and integration, so that data from different protocols can be effectively aggregated and processed. Correlating the correlation between data of different protocols is a key challenge. Use machine learning, neural networks and other technologies for data pattern recognition and feature extraction to establish association models between devices. These models can identify and predict device behavior, helping to optimize energy utilization and equipment management. At the same time, large-scale power terminal equipment data requires efficient storage and processing. It is also crucial to select a suitable database system and data structure design to support fast data access and query, while ensuring data integrity and security.

[0003] The field of multi-protocol power terminal equipment data association involves multiple technical aspects such as data fusion, association analysis, storage processing, standardization, and security assurance. Effective integration and association of these data can provide important support for optimizing the operation of power systems, realizing intelligent energy management and predictive maintenance. Therefore, it is crucial to effectively integrate various communication protocol data to realize intelligent and efficient operation of power system data. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, system and medium based on multi-protocol power terminal equipment data association, which has practical application value in network security monitoring, traffic management and other aspects by helping to identify and classify communication data of different protocols.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for associating data of a multi-protocol power terminal device, comprising the following steps:

[0007] 1) Collect data from various types of power communication equipment and pre-process the data;

[0008] 2) Use time series data to store the collected data and build data models and structures;

[0009] 3) Use data fusion technology to integrate data from different devices and establish correlation between devices;

[0010] 4) Use Siamese neural network algorithm to learn the correlation between different protocol data;

[0011] 5) Perform similarity measures on new data and identify unusual patterns and behaviors.

[0012] In step 1), data from various types of power communication equipment are collected. First, the types of power terminal equipment that need to be connected are confirmed, including sensors, metering equipment, and controllers. Modbus, DNP3, and IEC 61850 protocols are selected for connection. For each device, the corresponding interface or gateway is implemented to ensure that the device communicates with the data acquisition system, and data is extracted from each device using appropriate data acquisition equipment.

[0013] In step 1), the collected data needs to be preprocessed. Data preprocessing is mainly divided into two categories: one is data cleaning, and the other is quality inspection of the cleaned data.

[0014] Data cleaning:

[0015] 1) Outlier processing: detect and process outliers, and use statistical methods or rules to identify abnormal data;

[0016] 2) Missing value processing: fill missing values ​​by using interpolation, averaging or model-based methods to fill in data;

[0017] 3) Data format standardization: Convert data into a unified format to ensure that data collected by different devices can be integrated under a unified data structure;

[0018] Data quality inspection:

[0019] 1) Data quality assessment: Conduct data quality assessment to verify the completeness, accuracy and consistency of the data;

[0020] 2) Abnormal data processing: Data that cannot be repaired or is unreliable can be marked, eliminated, or further investigated and processed.

[0021] In step 2), use time series data to store the collected data, and build a data model and structure. Select the time series database InfluxDB to store the collected data. The data structure is designed with time as the main index and reasonably differentiated to support fast query.

[0022] In step 3), data fusion technology is used to integrate data from different devices, establish correlation between devices, and perform correlation analysis on the data stored in the time series database. The data included are: device information: including device ID, type, manufacturer, installation location; sensor data: temperature, humidity, current, voltage; metering data: power consumption, energy consumption statistics; related data mainly includes: association identifier: device ID of each data; association characteristics: common characteristics or combined characteristics of different power communication equipment; the integrated data is stored.

[0023] In step 4), the Siamese neural network algorithm is used to learn the correlation between different protocol data. The steps of model learning are as follows:

[0024] 1) The power communication equipment data (x1, x2) of different protocols obtained in the time series database in step 3) is used as input parameters, and the feature vector (f1, f2) corresponding to the data is obtained after processing.

[0025] 2) Calculate the similarity between feature vectors. Use Euclidean distance to measure the similarity between data:

[0026]

[0027] in:

[0028] D(x1,x2) represents the Euclidean distance between data x1 and x2.

[0029] f 1i and f 2i It is the i-th element of the eigenvector corresponding to the two data.

[0030] 3) The contrast loss function is used to measure the similarity between two data pairs. For each pair of data (x1, x2), and label y, the calculation formula of the contrast loss function L is:

[0031]

[0032] in:

[0033] L((x1,x2,y) represents the loss function value,

[0034] Y is the label corresponding to the data, which is 1 if the data corresponds to the same protocol and 0 if it corresponds to different protocols.

[0035] D is the distance between the eigenvectors of the data pair, which is the Euclidean distance here.

[0036] m is a boundary hyperparameter that controls the threshold at which a pair of data is considered similar.

[0037] 4) Set the neural network parameter to ω, then the gradient of the contrast loss function L with respect to the parameter ω can be expressed as:

[0038]

[0039]

[0040] in:

[0041] is the partial derivative of the loss function L with respect to the distance D,

[0042] is the partial derivative of the distance D with respect to the eigenvectors f1, f2,

[0043] is the partial derivative of the eigenvectors f1, f2 with respect to the parameter ω,

[0044] Using the calculated gradient information, the gradient descent method is used to update the parameters ω of the neural network. The updating process is usually as follows:

[0045]

[0046] Where ω is the learning rate, which controls the step size of parameter updates.

[0047] In step 5), the new data is similarly measured and unusual patterns and behaviors are identified. The main approach is to continuously collect new data, process it using the same method, store it in a time series database, and input it into the model trained in step 4) to obtain the correlation difference with the existing data. The difference will be known through the classification results obtained by the model.

[0048] In a second aspect, an embodiment of the present application provides a system based on multi-protocol power terminal equipment data association, which is used to implement the above method, including a data acquisition unit, a data processing unit, a data integration unit, a model calculation unit and a data identification unit;

[0049] The data acquisition unit collects data from various types of power communication equipment and pre-processes the data. The data processing unit uses time series data to store the collected data and builds data models and structures. The data integration unit uses data fusion technology to integrate data from different devices and establish the correlation between devices. The model calculation unit uses the Siamese neural network algorithm to learn the correlation between data from different protocols. The data recognition unit measures the similarity of new data and identifies unusual patterns and behaviors.

[0050] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code, and when the program code is executed by a processor, the steps of the method based on multi-protocol power terminal device data association as described above are implemented.

[0051] Compared with the prior art, the beneficial effect of the present application is that compared with the existing terminal device anomaly detection method, the present invention can learn the association of data of different protocols and quickly obtain similarity scores to achieve the purpose of detecting abnormal data. The method has practical application value in network security monitoring, traffic management, etc. by helping to identify and classify communication data of different protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 This is the Siamese neural network learning algorithm diagram in the present invention

[0054] Figure 2 This is a flow chart of the method for associating data based on multi-protocol power terminal equipment in the present invention. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0056] The terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0057] The terms "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and should not be understood as indicating or implying relative importance, nor should they be understood as requiring or implying any such actual relationship or order between these entities or operations.

[0058] like Figure 1 and Figure 2 As shown, a method and system based on multi-protocol power terminal equipment data association, characterized in that it includes the following steps:

[0059] 1) Collect data from various types of power communication equipment and pre-process the data;

[0060] 2) Use time series data to store the collected data and build data models and structures;

[0061] 3) Use data fusion technology to integrate data from different devices and establish correlation between devices;

[0062] 4) Use Siamese neural network algorithm to learn the correlation between different protocol data;

[0063] 5) Perform similarity measures on new data and identify unusual patterns and behaviors.

[0064] In step 1), data of various types of power communication equipment are collected. First, the type of power terminal equipment to be connected is confirmed. The present invention mainly targets sensors, metering equipment, and controllers. Modbus, DNP3, and IEC 61850 protocol types are selected for connection. For each device, a corresponding interface or gateway is implemented to ensure that the device can communicate with the data acquisition system. Use appropriate data acquisition equipment to extract data from each device, such as reading data with a sensor, obtaining data through its own communication port, etc.

[0065] In step 1), the collected data needs to be preprocessed. Data preprocessing is mainly divided into two categories: one is data cleaning, and the other is quality inspection of the cleaned data.

[0066] Data cleaning:

[0067] 1) Outlier processing: Detect and process outliers, and use statistical methods or rules to identify abnormal data.

[0068] 2) Missing value processing: fill in missing values ​​and use interpolation, averaging or model-based methods to fill in data.

[0069] 3) Data format standardization: Convert data into a unified format to ensure that data collected by different devices can be integrated under a unified data structure.

[0070] Data quality inspection:

[0071] 1) Data quality assessment: Conduct data quality assessment to verify the completeness, accuracy and consistency of the data.

[0072] 2) Abnormal data processing: Data that cannot be repaired or is unreliable can be marked, eliminated, or further investigated and processed.

[0073] In step 2), use time series data to store the collected data and build a data model and structure. The time series database InfluxDB is selected to store the collected data because it focuses on efficient storage and query of time series data. It is used here to store power communication equipment data. The data structure design uses time as the main index and is reasonably differentiated to support fast query.

[0074] In step 3), data fusion technology is used to integrate data from different devices and establish the correlation between devices. The data stored in the time series database is mainly analyzed for correlation. The device data included are: device information: including device ID, type, manufacturer, installation location, etc.; sensor data: data collected by different sensors, such as temperature, humidity, current, voltage; metering data: such as power consumption, energy consumption statistics, etc.; related data mainly include: association identification: device ID of each data; association characteristics: common characteristics or combined characteristics of different power communication equipment; the integrated data is stored.

[0075] In step 4), the Siamese neural network algorithm is used to learn the correlation between different protocol data. The steps of model learning are as follows:

[0076] 1) The power communication equipment data (x1, x2) of different protocols obtained in the time series database in step 3) is used as input parameters, and the feature vector (f1, f2) corresponding to the data is obtained after processing.

[0077] 2) Calculate the similarity between feature vectors. Use Euclidean distance to measure the similarity between data:

[0078]

[0079] in:

[0080] D(x1,x2) represents the Euclidean distance between data x1 and x2.

[0081] f 1i and f 2i It is the i-th element of the eigenvector corresponding to the two data.

[0082] 3) The contrast loss function is used to measure the similarity between two data pairs. For each pair of data (x1, x2), and label y, the calculation formula of the contrast loss function L is:

[0083]

[0084] in:

[0085] L((x1,x2,y) represents the loss function value,

[0086] Y is the label corresponding to the data, which is 1 if the data corresponds to the same protocol and 0 if it corresponds to different protocols.

[0087] D is the distance between the eigenvectors of the data pair, which is the Euclidean distance here.

[0088] m is a boundary hyperparameter that controls the threshold at which a pair of data is considered similar.

[0089] 4) Set the neural network parameter to ω, then the gradient of the contrast loss function L with respect to the parameter ω can be expressed as:

[0090]

[0091]

[0092] in:

[0093] It is the partial derivative of the loss function L with respect to the distance D.

[0094] It is the partial derivative of the distance D with respect to the eigenvectors f1, f2.

[0095] is the partial derivative of the eigenvectors f1, f2 with respect to the parameter ω.

[0096] Using the calculated gradient information, the gradient descent method is used to update the parameters ω of the neural network. The updating process is usually as follows:

[0097]

[0098] Where ω is the learning rate, which controls the step size of parameter updates.

[0099] In step 5), the new data is similarly measured and unusual patterns and behaviors are identified. The main approach is to continuously collect new data, process it using the same method, store it in a time series database, and input it into the model trained in step 4) to obtain the correlation difference with the existing data. The difference will be known through the classification result obtained by the model.

[0100] The method for associating data of multi-protocol power terminal equipment of the present invention first collects data from various types of power communication equipment and pre-processes the data; uses time series data to store the collected data and builds a data model and structure; uses data fusion technology to integrate data from different devices and establish the association between devices; then uses the Siamese neural network algorithm to learn the association between data from different protocols. The new data is measured for similarity, and unusual patterns or behaviors are identified, and abnormal behaviors are judged by the classification results given by the model.

[0101] The embodiment of the present application provides a system based on multi-protocol power terminal equipment data association, which is used to implement the above method, including a data acquisition unit, a data processing unit, a data integration unit, a model calculation unit and a data identification unit;

[0102] The data acquisition unit collects data from various types of power communication equipment and pre-processes the data. The data processing unit uses time series data to store the collected data and builds data models and structures. The data integration unit uses data fusion technology to integrate data from different devices and establish the correlation between devices. The model calculation unit uses the Siamese neural network algorithm to learn the correlation between data from different protocols. The data recognition unit measures the similarity of new data and identifies unusual patterns and behaviors.

[0103] An embodiment of the present application provides a computer-readable storage medium, which stores program code. When the program code is executed by a processor, the steps of the method based on multi-protocol power terminal device data association as described above are implemented.

[0104] An embodiment of the present application provides a computer-readable storage medium, which stores program code. When the program code is executed by a processor, the steps of the method for securely sharing power data as described above are implemented.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0109] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0110] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0111] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0112] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for associating data of multi-protocol power terminal equipment, characterized in that: The following steps are involved: 1) Collect data from various types of power communication equipment and pre-process the data; 2) Use time series data to store the collected data and build data models and structures; 3) Use data fusion technology to integrate data from different devices and establish correlation between devices; 4) Use Siamese neural network algorithm to learn the correlation between different protocol data; 5) Perform similarity measures on new data and identify unusual patterns and behaviors.

2. The method according to claim 1, characterized in that: In step 1), data from various types of power communication equipment are collected. First, the types of power terminal equipment that need to be connected are confirmed, including sensors, metering equipment, and controllers. Modbus, DNP3, and IEC 61850 protocols are selected for connection. For each device, the corresponding interface or gateway is implemented to ensure that the device communicates with the data acquisition system, and data is extracted from each device using appropriate data acquisition equipment.

3. The method according to claim 2, characterized in that: In step 1), the collected data needs to be preprocessed. Data preprocessing is mainly divided into two categories: one is data cleaning, and the other is quality inspection of the cleaned data. Data cleaning: 1) Outlier processing: detect and process outliers, and use statistical methods or rules to identify abnormal data; 2) Missing value processing: fill missing values ​​by using interpolation, averaging or model-based methods to fill in data; 3) Data format standardization: Convert data into a unified format to ensure that data collected by different devices can be integrated under a unified data structure; Data quality inspection: 1) Data quality assessment: Conduct data quality assessment to verify the completeness, accuracy and consistency of the data; 2) Abnormal data processing: Data that cannot be repaired or is unreliable can be marked, eliminated, or further investigated and processed.

4. The method according to claim 1, characterized in that: In step 2), use time series data to store the collected data, and build a data model and structure. Select the time series database InfluxDB to store the collected data. The data structure is designed with time as the main index and reasonably differentiated to support fast query.

5. The method according to claim 1, characterized in that: In step 3), data fusion technology is used to integrate data from different devices, establish the correlation between devices, and perform correlation analysis on the data stored in the time series database. The data included are: device information: including device ID, type, manufacturer, installation location; sensor data: temperature, humidity, current, voltage; metering data: power consumption, energy consumption statistics; the correlation data mainly includes: correlation identification: the device ID of each data; Related features: common features or combined features of different power communication devices; storing the integrated data.

6. The method according to claim 1, characterized in that: In step 4), the Siamese neural network algorithm is used to learn the correlation between different protocol data. The steps of model learning are as follows: 1) The power communication equipment data (x1, x2) of different protocols obtained in the time series database in step 3) is used as input parameters, and the feature vector (f1, f2) corresponding to the data is obtained after processing. 2) Calculate the similarity between feature vectors. Use Euclidean distance to measure the similarity between data: in: D(x1,x2) represents the Euclidean distance between data x1 and x2. f 1i and f 2i is the i-th element of the eigenvector corresponding to the two data, 3) The contrast loss function is used to measure the similarity between two data pairs. For each pair of data (x1, x2) and label y, the calculation formula of the contrast loss function L is: in: L((x1,x2,y) represents the loss function value, Y is the label corresponding to the data. If the data corresponds to the same protocol, it is 1, and if it corresponds to different protocols, it is 0. D is the distance between the eigenvectors of the data pair, here it is the Euclidean distance, m is a boundary hyperparameter that controls the threshold at which a pair of data is considered similar. 4) Set the neural network parameter to ω, then the gradient of the contrast loss function L with respect to the parameter ω can be expressed as: in: is the partial derivative of the loss function L with respect to the distance D, is the partial derivative of the distance D with respect to the eigenvectors f1, f2, is the partial derivative of the eigenvectors f1, f2 with respect to the parameter ω, Using the calculated gradient information, the gradient descent method is used to update the parameters ω of the neural network. The updating process is usually as follows: Where ω is the learning rate, which controls the step size of parameter updates.

7. The method according to claim 1, characterized in that: In step 5), the new data is similarly measured and unusual patterns and behaviors are identified. The main approach is to continuously collect new data, process it using the same method, store it in a time series database, and input it into the model trained in step 4) to obtain the correlation difference with the existing data. The difference will be known through the classification results obtained by the model.

8. A system based on multi-protocol power terminal equipment data association, used to implement the method according to any one of claims 1 to 7, characterized in that: It includes a data acquisition unit, a data processing unit, a data integration unit, a model calculation unit and a data recognition unit; The data acquisition unit collects data from various types of power communication equipment and pre-processes the data. The data processing unit uses time series data to store the collected data and builds data models and structures. The data integration unit uses data fusion technology to integrate data from different devices and establish the correlation between devices. The model calculation unit uses the Siamese neural network algorithm to learn the correlation between data from different protocols. The data recognition unit measures the similarity of new data and identifies unusual patterns and behaviors.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and when the program code is executed by a processor, the steps of the method based on multi-protocol power terminal device data association as described in any one of claims 1 to 7 are implemented.

Images