Anomaly detection method, device, electronic device and storage medium

Abstract

The present invention discloses an anomaly detection method, device, electronic device and storage medium, which are used to solve the technical problem that the existing anomaly detection method has a high possibility of missed detection or false alarm. The present invention includes: obtaining device communication flow data from a master station server of a preset power system; generating a fingerprint feature vector of the device communication flow data; performing data preprocessing on the fingerprint feature vector to obtain preprocessed data; inputting the preprocessed data into a preset anomaly detection model, and outputting the flow classification of the preprocessed data; the anomaly detection model is obtained by training with a training fingerprint feature vector as a training sample and a flow classification of the training fingerprint feature vector as a sample label; when the flow classification is abnormal flow data, obtaining the physical side data corresponding to the device communication flow data; using the physical side data to determine the abnormal type of the abnormal flow data.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection technology, and in particular to an anomaly detection method, device, electronic equipment and storage medium. Background Art

[0002] With the in-depth development of smart grids, the issue of grid information security has received increasing attention from industry and academia. At present, the network security protection of power cyber physical systems (CPS) relies heavily on security boundaries based on isolation measures such as firewalls and dedicated networks. Real cases this year have proven that security boundaries are no longer completely reliable, and attackers who break through security boundaries will pose a more serious threat. In order to make up for the shortcomings of traditional defense measures, intrusion detection systems have attracted widespread attention as an important means of in-depth defense of power CPS.

[0003] In order to ensure the authenticity of the terminal device identity, many researchers have applied some new network security technologies to power grid information security defense, such as identity authentication, data encryption, access control, intrusion detection, etc. As a typical industrial control system, the communication of power CPS is generally based on small data volume and high-frequency data exchange, which has self-similarity and correlation that are significantly different from traditional network traffic. Therefore, simply using the anomaly detection method in the traditional IT system may cause a large degree of missed detection or false alarm. Summary of the invention

[0004] The present invention provides an anomaly detection method, device, electronic device and storage medium, which are used to solve the technical problem that the existing anomaly detection methods have a high possibility of missed detection or false alarm.

[0005] The present invention provides an anomaly detection method, characterized by comprising:

[0006] Obtain equipment communication flow data from the master server of the preset power system;

[0007] Generate a fingerprint feature vector of the device communication traffic data;

[0008] Performing data preprocessing on the fingerprint feature vector to obtain preprocessed data;

[0009] The preprocessed data is input into a preset anomaly detection model, and the flow classification of the preprocessed data is obtained as output; the anomaly detection model is trained by using the training fingerprint feature vector as a training sample and the flow classification of the training fingerprint feature vector as a sample label;

[0010] When the traffic is classified as abnormal traffic data, obtaining physical side data corresponding to the device communication traffic data;

[0011] The physical side data is used to determine the abnormal type of the abnormal traffic data.

[0012] Optionally, the step of generating a fingerprint feature vector of the device communication traffic data includes:

[0013] Parsing the packet header of the communication flow data of the device to obtain address information and characteristic quantities;

[0014] A fingerprint feature vector of the device communication traffic data is generated according to the address information and the feature quantity.

[0015] Optionally, the step of performing data preprocessing on the fingerprint feature vector to obtain preprocessed data includes:

[0016] Performing standardization processing on the fingerprint feature vector to obtain standardized data;

[0017] The standardized data is normalized to obtain normalized data.

[0018] Optionally, the step of using the physical side data to determine the abnormal type of the abnormal traffic data includes:

[0019] Determining whether the physical side data is abnormal;

[0020] If not, determining that the abnormal type of the abnormal traffic data is a network attack abnormality;

[0021] If so, it is determined that the abnormal type of the abnormal traffic data is a physical fault abnormality.

[0022] The present invention also provides an abnormality detection device, comprising:

[0023] The equipment communication flow data acquisition module is used to acquire the equipment communication flow data from the master station server of the preset power system;

[0024] A fingerprint feature vector generation module, used to generate a fingerprint feature vector of the device communication flow data;

[0025] A data preprocessing module, used for performing data preprocessing on the fingerprint feature vector to obtain preprocessed data;

[0026] A traffic classification output module is used to input the pre-processed data into a preset anomaly detection model, and output the traffic classification of the pre-processed data; the anomaly detection model is trained using the training fingerprint feature vector as a training sample and the traffic classification of the training fingerprint feature vector as a sample label;

[0027] A physical side data acquisition module, used for acquiring physical side data corresponding to the device communication flow data when the flow is classified as abnormal flow data;

[0028] The abnormality type judgment module is used to use the physical side data to judge the abnormality type of the abnormal flow data.

[0029] Optionally, the fingerprint feature vector generation module includes:

[0030] The packet header parsing submodule is used to parse the packet header of the device communication flow data to obtain address information and feature quantities;

[0031] The fingerprint feature vector generation submodule is used to generate a fingerprint feature vector of the device communication flow data according to the address information and the feature quantity.

[0032] Optionally, the data preprocessing module includes:

[0033] A standardization processing submodule is used to perform standardization processing on the fingerprint feature vector to obtain standardized data;

[0034] The normalization processing submodule is used to perform normalization processing on the standardized data to obtain normalized data.

[0035] Optionally, the abnormality type judgment module includes:

[0036] An abnormality judgment submodule, used to judge whether the physical side data is abnormal;

[0037] A network attack anomaly determination submodule, for determining, if not, that the anomaly type of the abnormal traffic data is a network attack anomaly;

[0038] The physical fault anomaly determination submodule is used to determine that if so, the anomaly type of the abnormal traffic data is a physical fault anomaly.

[0039] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0040] The memory is used to store program code and transmit the program code to the processor;

[0041] The processor is used to execute any of the above anomaly detection methods according to the instructions in the program code.

[0042] The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute any of the above anomaly detection methods.

[0043] It can be seen from the above technical scheme that the present invention has the following advantages: the present invention discloses an abnormality detection method, including: obtaining device communication flow data from a master station server of a preset power system; generating a fingerprint feature vector of the device communication flow data; performing data preprocessing on the fingerprint feature vector to obtain preprocessed data; inputting the preprocessed data into a preset abnormality detection model, and outputting the flow classification of the preprocessed data; when the flow classification is abnormal flow data, obtaining the physical side data corresponding to the device communication flow data; and using the physical side data to determine the abnormal type of the abnormal flow data. The technical problem that the existing abnormality detection method has a high possibility of missed detection or false alarm is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 A flowchart of a method for detecting anomalies according to an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a 7-node power network and communication network provided by an embodiment of the present invention;

[0047] Figure 3 A structural block diagram of an anomaly detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention provide an anomaly detection method, an apparatus, an electronic device and a storage medium, which are used to solve the technical problem that the existing anomaly detection methods have a high possibility of missed detection or false alarm.

[0049] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1 , Figure 1 A flowchart of the steps of an anomaly detection method provided by an embodiment of the present invention.

[0051] The present invention provides an anomaly detection method, which may specifically include the following steps:

[0052] Step 101, obtaining device communication flow data from a master station server of a preset power system;

[0053] In the embodiment of the present invention, the equipment communication flow data refers to the flow data sent from the lower-level FTU (Feeder Terminal Unit) to the collector obtained from the master station server of the power system.

[0054] like Figure 2 As shown, a 7-node power network and its corresponding communication network are taken as an example, where S1 in the physical side of power distribution is the power source point, nodes 1-7 are segmented switch nodes, and are marked as FTU1-FTU7 in sequence; the access layer switch in the communication information side is used to simulate the distribution substation, connected to the lower layer communication equipment, and assembled into a router and switch for collecting substation information and communicating with the control side, and the collector is used to capture the data packets of the device communication traffic in the system for extracting traffic characteristics. The embodiment of the present invention only considers the impact of the device status on the communication traffic status once, so only the status information upload traffic is considered, and the control information sending traffic is ignored.

[0055] Step 102, generating a fingerprint feature vector of device communication flow data;

[0056] In the embodiment of the present invention, after the device communication flow data is acquired, a corresponding fingerprint feature vector may be generated.

[0057] In one example, the fingerprint feature vector can be generated by the following steps:

[0058] S21, parsing the packet header of the device communication flow data to obtain address information and feature quantities;

[0059] S22, generating a fingerprint feature vector of the device communication flow data according to the address information and the feature quantity.

[0060] In a specific implementation, the acquired device communication flow data may be subjected to packet header analysis to obtain address information such as source address and destination address of the device communication flow data, as well as feature quantities, to form a fingerprint feature vector for each piece of device communication flow data.

[0061] The characteristic quantities may include statistics such as data packet transmission delay, delay jitter, data packet size, and the number of data packets with the same source address within the previous 2 seconds.

[0062] Step 103, performing data preprocessing on the fingerprint feature vector to obtain preprocessed data;

[0063] In the embodiment of the present invention, after the fingerprint feature vector is obtained, data preprocessing may be performed on the fingerprint feature vector to obtain preprocessed data.

[0064] The step of performing data preprocessing on the fingerprint features may include the following sub-steps:

[0065] S31, performing standardization processing on the fingerprint feature vector to obtain standardized data;

[0066] S32, performing normalization processing on the standardized data to obtain normalized data.

[0067] In a specific implementation, the standardization of the fingerprint feature vector can be achieved by the following formula:

[0068]

[0069] Among them, X ij is the jth original feature value of the i-th fingerprint feature vector, X ij ′ is the corresponding feature quantity after standardization, that is, the standardized data after standardization; AVG j is the average value of the jth feature value, STAD j is the mean absolute deviation of the jth feature.

[0070] in:

[0071]

[0072]

[0073] Among them, if AVG j =0 or STAD j = 0, let X ij ′=0.

[0074] The normalization processing of standardized data is shown in the following formula:

[0075]

[0076] Among them, X min is the minimum value of the jth feature, X max is the maximum value of the jth feature, X ij ″ is the normalized data, i.e., preprocessed data.

[0077] Step 104, input the preprocessed data into a preset anomaly detection model, and output the traffic classification of the preprocessed data; the anomaly detection model is trained using the training fingerprint feature vector as a training sample and the traffic classification of the training fingerprint feature vector as a sample label;

[0078] In the embodiment of the present invention, after the pre-processed data is acquired, the pre-processed data may be input into a preset anomaly detection model, and the flow classification of the pre-processed data may be output.

[0079] Specifically, M groups of pre-processed fingerprint feature data with sample labels can be selected as training fingerprint feature vectors, and the training samples are learned using the ELM (Extreme Learning Machine) algorithm to obtain an anomaly detection model. In an embodiment of the present invention, the sample labels can include both abnormal traffic data and normal traffic data.

[0080] Among them, the ELM algorithm is a type of machine learning system or method built based on a feedforward neural network (FNN), which is suitable for supervised learning and unsupervised learning problems.

[0081] Step 105, when the traffic is classified as abnormal traffic data, obtaining physical side data corresponding to the device communication traffic data;

[0082] In the embodiment of the present invention, when the traffic is classified as abnormal traffic data, physical side data can be obtained, such as Figure 2 The node voltage, line current and other power flow data of the FTU on the physical side of the power distribution.

[0083] Step 106: Use the physical side data to determine the abnormal type of the abnormal traffic data.

[0084] In one example, step 106 may include:

[0085] S61, determining whether the physical side data is abnormal;

[0086] S62, if not, determining that the abnormal type of the abnormal traffic data is a network attack abnormality;

[0087] S63: If yes, determine that the abnormal type of the abnormal traffic data is a physical fault abnormality.

[0088] When the physical side data of the physical side node corresponding to the source address from which the abnormal traffic data is sent is normal, it can be considered that the abnormal traffic data is caused by a network attack; otherwise, it can be considered that it is caused by a physical side failure.

[0089] The present invention discloses an abnormality detection method, including: obtaining equipment communication flow data from a master station server of a preset power system; generating a fingerprint feature vector of the equipment communication flow data; performing data preprocessing on the fingerprint feature vector to obtain preprocessed data; inputting the preprocessed data into a preset abnormality detection model, and outputting a flow classification of the preprocessed data; when the flow classification is abnormal flow data, obtaining physical side data corresponding to the equipment communication flow data; and using the physical side data to determine the abnormal type of the abnormal flow data. The method solves the technical problem that the existing abnormality detection method has a high possibility of missed detection or false alarm.

[0090] See also Figure 3 , Figure 3 A structural block diagram of an anomaly detection device provided in an embodiment of the present invention.

[0091] An embodiment of the present invention provides an abnormality detection device, including:

[0092] The device communication flow data acquisition module 301 is used to acquire the device communication flow data from the master station server of the preset power system;

[0093] A fingerprint feature vector generating module 302, used to generate a fingerprint feature vector of device communication flow data;

[0094] The data preprocessing module 303 is used to perform data preprocessing on the fingerprint feature vector to obtain preprocessed data;

[0095] The traffic classification output module 304 is used to input the pre-processed data into a preset anomaly detection model and output the traffic classification of the pre-processed data; the anomaly detection model is trained using the training fingerprint feature vector as a training sample and the traffic classification of the training fingerprint feature vector as a sample label;

[0096] The physical side data acquisition module 305 is used to acquire the physical side data corresponding to the device communication flow data when the flow is classified as abnormal flow data;

[0097] The abnormal type determination module 306 is used to determine the abnormal type of the abnormal traffic data using the physical side data.

[0098] In the embodiment of the present invention, the fingerprint feature vector generation module 302 includes:

[0099] The packet header parsing submodule is used to parse the packet header of the device communication flow data to obtain address information and feature quantities;

[0100] The fingerprint feature vector generation submodule is used to generate a fingerprint feature vector of the device communication flow data according to the address information and the feature quantity.

[0101] In the embodiment of the present invention, the data preprocessing module 303 includes:

[0102] The standardization processing submodule is used to perform standardization processing on the fingerprint feature vector to obtain standardized data;

[0103] The normalization processing submodule is used to perform normalization processing on the standardized data to obtain normalized data.

[0104] In the embodiment of the present invention, the abnormality type determination module 306 includes:

[0105] The abnormality judgment submodule is used to judge whether the physical side data is abnormal;

[0106] A network attack anomaly determination submodule is used to determine if the anomaly type of the abnormal traffic data is a network attack anomaly;

[0107] The physical fault anomaly determination submodule is used to determine that if so, the anomaly type of the abnormal traffic data is a physical fault anomaly.

[0108] An embodiment of the present invention further provides an electronic device, the device comprising a processor and a memory:

[0109] The memory is used to store the program code and transmit the program code to the processor;

[0110] The processor is used to execute the abnormality detection method of the embodiment of the present invention according to the instructions in the program code.

[0111] The embodiment of the present invention further provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the anomaly detection method of the embodiment of the present invention.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0114] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0115] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0118] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0119] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0120] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An anomaly detection method, characterized in that: include: Obtain equipment communication flow data from the master server of the preset power system; Generate a fingerprint feature vector of the device communication traffic data; Performing data preprocessing on the fingerprint feature vector to obtain preprocessed data; The preprocessed data is input into a preset anomaly detection model, and the flow classification of the preprocessed data is obtained as output; the anomaly detection model is trained by using the training fingerprint feature vector as a training sample and the flow classification of the training fingerprint feature vector as a sample label; When the traffic is classified as abnormal traffic data, obtaining physical side data corresponding to the device communication traffic data; Using the physical side data to determine the abnormal type of the abnormal traffic data; The step of generating a fingerprint feature vector of the device communication flow data includes: Parsing the packet header of the communication flow data of the device to obtain address information and characteristic quantities; Generate a fingerprint feature vector of the device communication traffic data based on the address information and feature quantity; wherein the feature quantity includes data packet transmission delay, delay jitter, data packet size, and the number of data packets with the same source address within the previous 2 seconds; The step of using the physical side data to determine the abnormal type of the abnormal traffic data includes: Determining whether the physical side data is abnormal; If not, determining that the abnormal type of the abnormal traffic data is a network attack abnormality; If yes, determining that the abnormal type of the abnormal traffic data is a physical fault abnormality; The step of performing data preprocessing on the fingerprint feature vector to obtain preprocessed data includes: Normalize the fingerprint feature vector: ； in, is the jth original feature value of the i-th fingerprint feature vector, is the corresponding feature quantity after standardization, that is, the standardized data after standardization; is the average value of the jth feature value, is the mean absolute deviation of the jth feature; in: ； ； Among them, if or ,make ; Normalize the standardized data: ； in, is the minimum value of the jth feature, is the maximum value of the jth feature, It is the normalized data, that is, the preprocessed data.

2. An abnormality detection device, characterized in that: include: The equipment communication flow data acquisition module is used to acquire the equipment communication flow data from the master station server of the preset power system; A fingerprint feature vector generation module, used to generate a fingerprint feature vector of the device communication flow data; A data preprocessing module, used for performing data preprocessing on the fingerprint feature vector to obtain preprocessed data; A traffic classification output module is used to input the pre-processed data into a preset anomaly detection model, and output the traffic classification of the pre-processed data; the anomaly detection model is trained using the training fingerprint feature vector as a training sample and the traffic classification of the training fingerprint feature vector as a sample label; A physical side data acquisition module, used for acquiring physical side data corresponding to the device communication flow data when the flow is classified as abnormal flow data; An abnormality type judgment module, used to judge the abnormality type of the abnormal flow data by using the physical side data; Wherein, the fingerprint feature vector generation module includes: The packet header parsing submodule is used to parse the packet header of the device communication flow data to obtain address information and feature quantities; A fingerprint feature vector generation submodule, used to generate a fingerprint feature vector of the device communication traffic data according to the address information and feature quantity; wherein the feature quantity includes data packet transmission delay, delay jitter, data packet size, and the number of data packets with the same source address within the previous 2 seconds; Wherein, the abnormal type judgment module includes: An abnormality judgment submodule, used to judge whether the physical side data is abnormal; A network attack anomaly determination submodule, for determining, if not, that the anomaly type of the abnormal traffic data is a network attack anomaly; A physical fault anomaly determination submodule, for determining, if yes, that the anomaly type of the abnormal traffic data is a physical fault anomaly; Wherein, the data preprocessing module includes: Normalize the fingerprint feature vector: ； in, is the jth original feature value of the i-th fingerprint feature vector, is the corresponding feature quantity after standardization, that is, the standardized data after standardization; is the average value of the jth feature value, is the mean absolute deviation of the jth feature; in: ； ； Among them, if or ,make ; Normalize the standardized data: ； in, is the minimum value of the jth feature, is the maximum value of the jth feature, It is the normalized data, that is, the preprocessed data.

3. An electronic device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the anomaly detection method according to claim 1 according to the instructions in the program code.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the abnormality detection method according to claim 1.

Images