Method and device for processing abnormal traffic in power internet of things, and storage medium

By performing multi-dimensional segmentation and pattern recognition of the flow data of power equipment in the Internet of Things, identifying key conduction points and performing flow control, the problems of slow loading and visual chaos caused by large amounts of data in the Internet of Things are solved, and the problem of slow loading and visual chaos caused by large amounts of data in the Internet of Things is improved. User experience and reduce abnormal traffic shocks.

CN118984290APending Publication Date: 2024-11-19GUANGZHOU KETENG INFORMATION TECH
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411091586.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The page loading and interaction lag caused by massive heterogeneous data in the Internet of Things in the Internet of Things affects the user experience, and the visualization of large-scale data causes visual confusion, and the user positioning of information of interest is slower.

Method used

By establishing segmentation rules for flow data in multiple dimensions for power equipment of various types in the Internet of Things, dividing real-time flow data into multiple traffic segments, identifying traffic patterns, building visual statistics, generating distribution law information, and mining Periodic regular information, identify key conduction points that cause traffic mutations, detect abnormal devices and implement flow control strategies.

Benefits of technology

Effectively reduces the amount of data, improves user experience, reduces slow page loading and interactive lag, ensures the visual neatness of data, and reduces the impact of abnormal traffic on the core business of the power Internet of Things through traffic control strategies, ensuring that Working normally.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984290A_ABST
    Figure CN118984290A_ABST
Patent Text Reader

Abstract

The invention discloses a method and device for processing abnormal traffic in an electric power internet of things, and a storage medium. The method comprises the following steps: establishing a segmentation rule of traffic data for electric power equipment in the electric power internet of things in multiple dimensions; dividing the traffic data into a plurality of traffic fragments according to a segmentation rule; identifying a traffic mode of the traffic fragment according to the multi-dimensional features of the traffic fragment; constructing visual statistical data for the traffic fragments according to the traffic mode; counting distribution rule information of partial data indicated by the interactive operation; mining periodic rule information for part of traffic fragments indicated by the interactive operation in time; according to the distribution rule information and the periodical rule information, key conduction points causing flow sudden change are identified from the electric power Internet of Things; detecting the abnormal power equipment according to the real-time flow segment of the key conduction point, and taking the abnormal power equipment as abnormal equipment; and executing the flow control strategy on the abnormal equipment. The phenomena of slow page loading, interaction lagging and the like are reduced, and the user experience is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electric power Internet, and in particular to a method, device and storage medium for processing abnormal traffic in electric power Internet of Things. Background Art

[0002] The interactive visual analysis system of power Internet of Things traffic is faced with massive heterogeneous data. These data come from different types of equipment, such as smart meters, transformers, switches, etc. The data formats are diverse and the time span is large.

[0003] Traditional visualization methods usually load and render all data at once. When the amount of data exceeds a certain scale, it will cause slow page loading and interaction jams, affecting the user experience.

[0004] At the same time, the visualization of large-scale data will also cause visual confusion. There will be more interference when users locate the information they are interested in, resulting in slower positioning. Summary of the invention

[0005] In view of this, the present invention provides a method, device and storage medium for processing abnormal traffic in an electric power Internet of Things, so as to improve the interaction efficiency of data in the traffic of the electric power Internet of Things.

[0006] A first aspect of the present invention provides a method for processing abnormal traffic in an electric power Internet of Things, comprising:

[0007] Establish segmentation rules for flow data in multiple dimensions for various types of power equipment in the power Internet of Things;

[0008] When real-time flow data is collected from the electric power equipment, the real-time flow data is divided into a plurality of flow segments according to the segmentation rule;

[0009] Identifying a traffic pattern of the traffic segment according to the multi-dimensional features of the traffic segment;

[0010] Constructing visualized statistical data for the traffic segments according to the traffic patterns;

[0011] When receiving an interactive operation for the statistical data, generating distribution law information for a portion of the statistical data indicated by the interactive operation;

[0012] Mining periodic regularity information for part of the traffic segments indicated by the interactive operation in time;

[0013] Identifying a key conduction point causing a sudden change in flow rate in the electric power Internet of Things based on the distribution law information and the periodic law information;

[0014] Detecting the abnormal power equipment according to the real-time flow segment of the key conduction point as an abnormal equipment;

[0015] A flow control strategy is executed on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device.

[0016] A second aspect of the present invention provides a device for processing abnormal traffic in an electric power Internet of Things, comprising:

[0017] A segmentation rule establishment module is used to establish segmentation rules for flow data in multiple dimensions for various types of power equipment in the power Internet of Things;

[0018] A flow segment division module, used for dividing the real-time flow data into a plurality of flow segments according to the segmentation rule when real-time flow data is collected from the electric power equipment;

[0019] A traffic pattern recognition module, used to recognize the traffic pattern of the traffic segment according to the multi-dimensional features of the traffic segment;

[0020] A statistical data building module, used for building visualized statistical data for the traffic segment according to the traffic pattern;

[0021] A statistical data generating module, configured to generate distribution regularity information for a portion of the statistical data indicated by the interactive operation when an interactive operation for the statistical data is received;

[0022] A periodic regularity information mining module, used for mining periodic regularity information on part of the traffic segments indicated by the interactive operation in time;

[0023] A key transmission point identification module, used for identifying key transmission points causing traffic mutations in the power Internet of Things based on the distribution law information and the periodic law information;

[0024] An abnormal device detection module, used for detecting the abnormal power device according to the real-time flow segment of the key conduction point as an abnormal device;

[0025] The flow control strategy execution module is used to execute the flow control strategy on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device.

[0026] A third aspect of the present invention provides an electronic device, the electronic device comprising:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for processing abnormal traffic in the electric power Internet of Things as described in the first aspect above.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for processing abnormal traffic in the electric power Internet of Things as described in the first aspect above.

[0031] A fifth aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for processing abnormal traffic in the power Internet of Things as described in the first aspect above.

[0032] In this embodiment, flow data segmentation rules are established for various types of power equipment in the power Internet of Things in multiple dimensions; when real-time flow data is collected from the power equipment, the real-time flow data is divided into multiple flow segments according to the segmentation rules; the flow pattern of the flow segment is identified according to the characteristics of the flow segment in multiple dimensions; visual statistical data is constructed for the flow segment according to the flow pattern; when receiving an interactive operation for statistical data, the distribution law information of part of the statistical data indicated by the interactive operation is obtained; periodic regularity information is mined for part of the flow segment indicated by the interactive operation in time; the key conduction points causing the sudden change of flow are identified in the power Internet of Things according to the distribution law information and the periodic law information; the power equipment that generates abnormality according to the real-time flow segment of the key conduction point is detected as an abnormal device; the flow control strategy is executed on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device. This embodiment reasonably segments the flow data, greatly reduces the amount of data loaded at one time, reduces the phenomenon of slow page loading and interaction jamming, effectively improves the user experience, and the small-scale data is visually neater, and the user can quickly locate the information of interest.

[0033] In addition, this embodiment mines the key conduction points that cause traffic mutations based on the distribution and periodicity patterns of traffic data, thereby formulating corresponding traffic control strategies, restricting and guiding abnormal traffic, and effectively reducing the impact of abnormal traffic on the core business of the power Internet of Things, thereby ensuring the normal operation of the power Internet of Things.

[0034] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 It is a flowchart of a method for processing abnormal traffic in an electric power Internet of Things provided in Embodiment 1 of the present invention.

[0037] Figure 2 It is a structural schematic diagram of a device for processing abnormal traffic in an electric power Internet of Things provided in Embodiment 2 of the present invention.

[0038] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Embodiment 1

[0042] See also Figure 1, shows a flow chart of a method for processing abnormal traffic in the electric power Internet of Things provided by the first embodiment of the present invention. The method can be executed by a device for processing abnormal traffic in the electric power Internet of Things. The device for processing abnormal traffic in the electric power Internet of Things can be implemented in the form of hardware and / or software. The device for processing abnormal traffic in the electric power Internet of Things can be configured in an electronic device. Figure 1 As shown, the method includes:

[0043] Step 101: Establish flow data segmentation rules in multiple dimensions for various types of power equipment in the power Internet of Things.

[0044] The flow data of the power Internet of Things can be segmented for fast loading and rendering, which also reduces visualization clutter and allows users to quickly locate information of interest.

[0045] When segmenting traffic data, the granularity of the segment should be considered. If the granularity is too large, the amount of data in a single segment will still be too large. If the granularity is too small, the number of segments will increase, causing users to frequently switch segments.

[0046] In this embodiment, multi-dimensional segmentation rules for flow data can be established based on the differentiated characteristics of different device types in the power Internet of Things. The segmentation rules can flexibly select the segmentation granularity and divide the flow in different scenarios to form a mapping relationship between device types and segmentation rules.

[0047] In a specific implementation, flow data as samples can be collected from each node in the power Internet of Things at a certain frequency for power equipment of various types in the power Internet of Things.

[0048] The frequencies of power equipment of different types are different. For example, for smart meters, flow data can be collected every 5 minutes, including upstream flow, downstream flow and protocol type.

[0049] The flow data used as samples are preprocessed according to the device type to obtain standardized structured data for subsequent analysis. The preprocessing includes using algorithms such as sliding windows (the window size is set to 1 hour, and if the flow value at a certain moment exceeds 3 times the window average, it is considered an outlier) to clean outliers and remove noise data, and normalize the remaining flow data using algorithms such as maximum and minimum values, mapping the flow data to the interval [0, 1], etc.

[0050] According to the equipment type and functional characteristics of the power equipment, a hierarchical clustering algorithm is used to perform hierarchical clustering on the structured data, which is convenient for multi-dimensional clustering analysis and obtains the similarity matrix between the power equipment. The similarity matrix can be used to calculate the similarity between the power equipment to determine whether the power equipment is similar.

[0051] In the similarity matrix, similar power devices are merged into a specified number of device clusters layer by layer starting from the bottom layer. Taking smart meters and distribution terminals as an example, the average hourly flow data of smart meters and distribution terminals within 24 hours is calculated to obtain two 24-dimensional vectors, and the cosine value of the vector angle is calculated as the similarity. Using the hierarchical clustering algorithm, the clustering threshold is set to 0.85, and two power devices or device clusters are merged when the similarity is greater than 0.85.

[0052] For the power equipment in the device cluster, multiple rule thresholds are set in the dimensions of time period, traffic size and protocol type as multi-dimensional candidate rules; wherein the rule threshold refers to the threshold used for segmentation, and the candidate rules include combinations of different rule thresholds.

[0053] For example, in terms of time, a day is divided into three time periods: 7:00-11:00, 11:00-19:00, and 19:00-7:00 the next day. In terms of traffic size, traffic size is divided into low, medium, and high levels according to device type characteristics. For example, the low traffic threshold of a smart meter is 10KB / h, and the high traffic threshold is 100KB / h. In terms of protocol type, MQTT, CoAP, HTTP, and other protocol types can be divided.

[0054] In this example, a candidate rule for the smart meter may be: if the smart meter uses the MQTT protocol during 7:00-11:00 and the traffic size is between 10-100 KB / h, it is determined to be in normal working state.

[0055] At this point, the candidate rules can be verified and optimized, and the accuracy of the candidate rules can be cross-validated by k-fold to obtain the verification results (i.e., the average accuracy). The k-fold cross-validation divides the candidate rules into k parts, then uses k-1 parts as the training set and the remaining 1 part as the verification set, which is repeated k times.

[0056] For example, let k = 5, divide the data set (candidate rules) into 5 parts, use 4 parts for training and 1 part for verification each time, and repeat 5 times. Assuming that the accuracy of a candidate rule in 5 verifications is 92%, 94%, 91%, 93%, and 95% respectively, the average accuracy is 93%.

[0057] The rule thresholds in the candidate rules and the combination mode between the rule thresholds are adjusted according to the verification result to obtain the segmentation rule.

[0058] Exemplarily, the high traffic threshold of the smart meter is increased from 100 KB / h to 120 KB / h to improve the overall accuracy.

[0059] Step 102: When real-time flow data is collected from the power equipment, the real-time flow data is divided into a plurality of flow segments according to a segmentation rule.

[0060] In this embodiment, the flow data of multi-source heterogeneous power Internet of Things can be aggregated in real time, and the flow data can be classified and split according to the mapping relationship between device type and segmentation rule to obtain flow segments under different device types.

[0061] In the specific implementation, data acquisition equipment can be deployed at the nodes (especially the key nodes) in the power Internet of Things. Under the distributed data acquisition framework, the data acquisition equipment collects real-time flow data from the power equipment by reading the preset collection frequency; the real-time flow data is marked with the timestamp and device identification marked by the data acquisition device.

[0062] The parser matching the protocol type of the real-time traffic data is called to pre-process the real-time traffic data to obtain the target data in a standard format (such as JSON). For example, the traffic data of the TCP / IP protocol is parsed using the Socket programming interface, the traffic data of the MQTT protocol is parsed using the Paho client library, the traffic data of the CoAP protocol is parsed using the CoAP library, and so on.

[0063] Set up a data transmission channel, use the device type of the power equipment as an independent topic (such as "smart_meter_data", "transformer_data" and "substation_data"), and push the target data to the middleware of the preset message queue (such as ApacheKafka, etc.) in real time, so as to push the target data to the distributed publishing message system (such as Kafka) in real time.

[0064] Based on factors such as load balancing and fault tolerance, the target data under the specified topic is subscribed from the message system (such as Kafka), and the device type of the subscribed target data is identified and the segmentation rules are matched according to the pre-established mapping relationship between the device type and the segmentation rules.

[0065] In a preset rule engine (such as Drools, etc.), the device type identification rules and flow data segmentation rules are defined through rule definitions. For example, "if the device ID prefix is ​​'SM' and the average flow rate per hour is less than 10KB, it is determined to be a smart meter", "if the device flow rate exceeds 100KB / h during 8:00-22:00, it is classified as peak period data", etc. The rule engine (such as Drools, etc.) is used to read the segmentation rules configured for the device type to which the power equipment belongs, and the target data is classified and split into multiple flow segments according to the segmentation rules to achieve classification and splitting of flow data.

[0066] The traffic segments are stored in a pre-set distributed database (such as Apache Cassandra, etc.). The composite primary key of the distributed database (such as Apache Cassandra, etc.) includes the device type, the time period of the traffic segment, and the device identifier (such as the device ID). For example, the primary key is (device_type, time_bucket, device_id), where the device type is used to optimize query performance, time_bucket is the time period, which can be divided by hours, and device_id is the device identifier. At this time, secondary indexes on device_type and time_bucket can be created to optimize query performance by device type and time range. The traffic segment is written in batch insertion mode, and batch writing is triggered every 1,000 data or every 30 seconds to balance writing efficiency and real-time performance.

[0067] Step 103: Identify the traffic pattern of the traffic segment based on the multi-dimensional features of the traffic segment.

[0068] In this embodiment, for each traffic segment, multi-dimensional features of the traffic segment can be extracted, such as data volume, packet size, packet transmission frequency, protocol type, etc., and the traffic pattern of each subset of the traffic segment can be determined by clustering analysis of the features.

[0069] In a specific implementation, multiple frames of raw data are read from the traffic segment.

[0070] A time window is added to multiple frames of raw data. The width of the time window is fixed (such as 5 minutes). Multi-dimensional features are extracted from the raw data in the time window and recorded as raw traffic features. The raw traffic features include the total data volume, average packet size, maximum packet size, minimum packet size, packet sending frequency, and protocol type ratio, etc.

[0071] For the original traffic characteristics, the box plot method can be used to identify and eliminate outliers. For example, data exceeding Q3+1.5IQR or below Q1-1.5IQR can be marked as abnormal, where Q1 is the lower quartile, Q3 is the upper quartile, and IQR is the interquartile range.

[0072] The original traffic characteristics are normalized by Z-score to obtain the target traffic characteristics, so that the target traffic characteristics are in a standard normal distribution. The Z-score normalization is Z = (X-μ) / σ, where X is the original value, μ is the mean, σ is the standard deviation, and Z is the standardized value.

[0073] K-means clustering (K-means) is performed on the traffic segments according to the target traffic characteristics to obtain multiple traffic clusters; in K-means clustering, the elbow rule can be used to determine the optimal K value, the sum of squared errors (SSE) under different K values ​​can be calculated, the SSE-K curve can be drawn, and the K value at the inflection point of the curve is selected as the optimal number of clusters; by calculating the Euclidean distance as the similarity measure, the center point of the traffic cluster is iteratively optimized until the center point of the traffic cluster no longer changes or the maximum number of iterations is reached; in each iteration, the Euclidean distance between each traffic segment and the center point of each traffic cluster is calculated, and the traffic cluster with the smallest Euclidean distance is selected as the traffic cluster to which the traffic segment belongs.

[0074] The anti-normalization operation is performed on the center point of each traffic cluster (such as X=Z*σ+μ) to obtain the typical traffic pattern in the original feature space. That is, the traffic segments in the same traffic cluster use the center point in the traffic cluster as the traffic pattern, forming a mapping relationship between the traffic segments and the typical traffic pattern.

[0075] For example, in the environment of the power Internet of Things, the original data is read from the traffic fragment of the smart meter, and a fixed time window of 5 minutes is set to extract the original features. For the data in a certain 5-minute window, the total data volume is calculated to be 500KB, the average packet size is 128 bytes, the maximum packet size is 1024 bytes, the minimum packet size is 64 bytes, the packet sending frequency is 10 packets / second, the MQTT protocol accounts for 80%, and the HTTP protocol accounts for 20%. The box plot method is used to identify outliers, and Q1 is calculated to be 100 bytes, Q3 is 200 bytes, and IQR is 100 bytes. The packet size less than -50 bytes (Q1-1.5IQR) or greater than 350 bytes (Q3+1.5IQR) is marked as an outlier and removed. The extracted feature values ​​are normalized by Z-score to obtain the target traffic features; for example, the average packet size of 128 bytes is normalized to 0.5. The K-means clustering algorithm is used to perform cluster analysis on the target traffic features, and the optimal K value is determined to be 3 by the elbow rule. Iteratively calculate the Euclidean distance and optimize the class center. After 50 iterations, the center point no longer changes, and three traffic clusters are obtained. Calculate the center point of the traffic cluster and denormalize it to obtain three typical traffic patterns: low traffic pattern, with a data volume of 100KB / 5min, an average packet size of 80 bytes, and a packet sending frequency of 5 packets / second; medium traffic pattern, with a data volume of 500KB / 5min, an average packet size of 128 bytes, and a packet sending frequency of 10 packets / second; high traffic pattern, with a data volume of 2MB / 5min, an average packet size of 256 bytes, and a packet sending frequency of 20 packets / second. Calculate the Euclidean distance between each traffic segment and the center points of the three traffic clusters. If a traffic segment has the smallest distance to the medium traffic pattern, which is 0.2, then it is determined that the traffic segment belongs to the medium traffic pattern, and the mapping between the traffic segment and the typical traffic pattern is completed.

[0076] Step 104: Construct visualized statistical data for the traffic segments based on the traffic patterns.

[0077] In this embodiment, each traffic segment can be classified according to its traffic pattern, and a matching visualization encoding method can be dynamically selected based on the classification result to construct visualized statistical data. For example, for traffic segments that meet the characteristics of large traffic, traffic maps and heat maps are used as macro views to display their traffic distribution; for traffic segments that meet the characteristics of small traffic, pie charts and bar charts are used as micro views to display their traffic composition ratio, and so on.

[0078] The visualization component of the D3.js library is used to dynamically select corresponding visualization statistics according to the classification results of the traffic pattern of the traffic segment, and the selected statistics are passed to the D3.js component for SVG or Canvas rendering.

[0079] In addition, a regular update mechanism can be set up to obtain the latest traffic data from the data source at regular intervals (such as 5 minutes), triggering the re-rendering of the visualization components to achieve a dynamic update effect.

[0080] In a specific implementation, a specified quantile (such as 75%) may be calculated for the historical flow data of the electric power equipment as the flow threshold.

[0081] The traffic pattern of the traffic segment is compared with the traffic threshold, thereby classifying the traffic pattern of the traffic segment.

[0082] If the traffic pattern of the traffic segment is greater than the traffic threshold, the traffic segment is determined to meet the large traffic characteristics.

[0083] If the traffic pattern of the traffic segment is less than or equal to the traffic threshold, the traffic segment is determined to meet the low traffic characteristics.

[0084] For traffic segments that meet the characteristics of large traffic, the mapping relationship between the device identification and the geographic location (coordinates) is queried from the preset relational database, and the geographic location of the current power equipment is queried based on the mapping relationship. The traffic map data is generated by combining the geographic location of the power equipment with the traffic segment, where the traffic map data includes fields such as longitude and latitude, traffic size, and timestamp.

[0085] For traffic segments that meet the characteristics of small traffic, the proportion of traffic segments under various protocol types is counted, and pie chart data is generated using the proportion of traffic segments under various protocol types, where the pie chart data includes fields such as protocol type and proportion.

[0086] For traffic segments that meet the characteristics of small traffic, the sliding window method can be used to calculate the change information of the traffic segments in each time period, and the change information of the traffic segments in each time period is used to generate bar chart data, where the bar chart data structure includes fields such as time period and traffic size.

[0087] In one example, in the interactive visual analysis system of power Internet of Things traffic, historical traffic data can be statistically analyzed, and the 75% quantile is calculated to be 10MB / hour, which is used as the traffic threshold to classify the traffic fragments of a smart substation. Its traffic pattern is 15MB / hour, which is greater than the traffic threshold and is determined to be a large traffic feature. The geographical location (latitude and longitude) of the smart substation is queried from the PostgreSQL database. A traffic map data structure is generated in combination with the traffic fragment, which contains the coordinates, the traffic size of 15MB / hour, and the current timestamp. Using a 1-hour sliding window, the cumulative traffic in the area within 24 hours is calculated to be 360MB, and the traffic map data of the heat map data structure is generated.

[0088] In addition, for the traffic segments of smart meters that are determined to have small traffic characteristics, statistics show that TCP protocol accounts for 60% and UDP protocol accounts for 40%, and pie chart data is generated. A 30-minute sliding window is used to calculate the traffic changes in 48 time periods and generate bar chart data. For example, the traffic in the 8:00-8:30 period is 1.5MB, and the traffic in the 8:30-9:00 period is 1.8MB.

[0089] The D3.js library is used to implement visualization components. For the flow segments of smart substations with large flow characteristics, the geoJSON method is called to draw the flow map, that is, the heatmap method is used to draw the heat map; for the flow segments of smart meters with small flow characteristics, the pie method is called to draw the pie chart data, and the bar method is used to draw the bar chart data.

[0090] The interactive visual analysis system of the power Internet of Things traffic obtains the latest traffic data from the Kafka message queue every 5 minutes, triggers the update method of D3.js to re-render the SVG graphics, and realizes dynamic updates. The traffic data of the power Internet of Things is intuitively presented on the visual interface for easy monitoring and analysis.

[0091] Step 105: When an interactive operation for statistical data is received, generate distribution law information for part of the statistical data indicated by the interactive operation.

[0092] In this embodiment, on the basis of visual coding, differentiated interaction methods are designed for traffic segments of different device types. Through interactive operations such as zooming in, zooming out, panning, and drilling down, focused analysis is achieved on the traffic segments of interest, and distribution pattern information of the traffic segments within the segment is analyzed through interactive operations such as traffic focusing and filtering.

[0093] In a specific implementation, interactive operations that are adapted to the device type and traffic pattern of the power equipment and are performed on statistical data are received. For example, for traffic segments that meet the characteristics of large traffic, interactive operations include two-finger zooming, single-finger dragging, etc. For traffic segments that meet the characteristics of small traffic, interactive operations include selecting a chart type from a drop-down menu, adjusting the data range with a slider, etc.

[0094] At least one of a scaling factor, a translation vector, and a drill-down level is calculated for the statistical data according to the interaction operation.

[0095] When the interactive operation instructs to scale or translate the statistical data, the scaling ratio and translation vector are calculated for the statistical data according to the interactive operation, wherein the scaling ratio can be calculated by linear interpolation, and the translation vector includes the translation distance and the translation direction.

[0096] When the interactive operation indicates drilling down, more fine-grained traffic data is obtained from the database, such as drilling down from the hourly level to the minute level, or drilling down from the regional level to the individual device level, the source traffic data of the visualization component is updated, and the statistics of the selected area are redrawn.

[0097] The statistical data is updated according to at least one of a zoom ratio, a translation vector, and a drill-down level, that is, a visual range of a visualization component is updated and the statistical data is re-rendered to focus on a traffic segment of interest.

[0098] The distribution threshold is generated according to the statistical characteristics of the historical traffic data; wherein the statistical characteristics include the total traffic volume, peak value and average value, etc.

[0099] The original data that meets the distribution threshold is filtered out from the partial traffic segments indicated by the interactive operation as the data of interest.

[0100] Perform K-means clustering on the interest data to obtain distribution law information.

[0101] In one example, for power equipment with large flow, such as smart substations, a two-finger zoom function is implemented, with the zoom ratio range set to 0.5 to 2 times, and the maximum translation distance of a single finger drag is 50% of the screen width. For power equipment with small flow, such as smart meters, a drop-down menu containing pie charts, bar charts, and line charts is provided, and a 0-24 hour slider is set to adjust the data time range.

[0102] When a zoom operation is detected, a linear interpolation algorithm is used to calculate the new zoom ratio. For example, when zooming in from 1x to 1.5x, the traffic data at the midpoint will increase linearly by 50%.

[0103] When a translation operation is detected, the translation direction and distance are obtained by calculating the vector difference between the starting point and the end point. For example, if the translation is 100 pixels to the right, the longitude on the map will increase by 0.01 degrees.

[0104] When a drill-down operation is detected, when the user clicks on a certain area, the total traffic volume at the hourly level of 10MB / h is drilled down to the minute level, displaying 60 data points, each of which represents the traffic volume at that minute, such as 0.15MB at 8:01 and 0.18MB at 8:02.

[0105] Calculate the average and standard deviation of traffic data in the same period of the past seven days, and set the average plus twice the standard deviation as the automatic threshold, such as 12MB / h.

[0106] Users can also set the time range of 9:00-17:00 and the traffic threshold of 15MB / h through the graphical interface. Spectral analysis is performed using fast Fourier transform to identify periodic patterns such as 24 hours and 7 days.

[0107] At the same time, the K-means clustering algorithm is used, and the number of clusters k is set to 3 to classify the characteristics of the data of interest, and three traffic modes of low, medium, and high are obtained. For example, the center of the low-traffic cluster is 5MB / h, the center of the medium-traffic cluster is 15MB / h, and the center of the high-traffic cluster is 30MB / h, so as to deeply analyze the traffic composition and distribution rules.

[0108] Step 106: Mining periodic regularity information for a portion of the traffic segments indicated by the interactive operation in time.

[0109] In this embodiment, the traffic segments are dynamically switched on the time axis to form a time-series visualization view of the evolution of the traffic segments. The differences in the distribution of traffic segments in different time periods are analyzed through changes in the time dimension, the peak and trough periods of the traffic segments are determined, and the periodic regularity information of the traffic segments is mined.

[0110] In the specific implementation, the time range is determined on the timeline based on the earliest timestamp and the latest timestamp, and a timeline (control) containing multiple levels of time intensity such as year, month, day, and hour is constructed, and some traffic segments indicated by interactive operations are mapped to the timeline.

[0111] When a positioning operation (interaction operation) is received, a binary search method is used to query the time point corresponding to the positioning operation on the time axis, and a scaling operation and / or a translation operation are performed on the traffic segment based on the time point to achieve dynamic switching of the traffic segment.

[0112] If the zoom operation and / or translation operation is completed, the time interval is calculated according to the length of the time window, and the time axis is divided into multiple time periods according to the specified time interval using a logarithmic scale or other method, thereby obtaining time spans of different scales.

[0113] Use line graphs, area graphs and other visualization graphics to generate visualized time series data for traffic segments in multiple time periods to characterize the changing trend of traffic segments over time. In the visualization graphics, the depth of color can be used to represent the traffic size, standard key time points, and linear interpolation can be used to calculate intermediate frames to achieve dynamic update and smooth transition of visualization graphics, etc.

[0114] Generate fragment statistical features for traffic fragments in multiple time periods; the fragment statistical features include average value, maximum value and minimum value.

[0115] The traffic segments of multiple time periods are divided into peak period and trough period according to the segment statistical characteristics; the value of the traffic segment in the peak period is greater than the specified multiple of the segment statistical characteristics (such as 1.5), and the value of the traffic segment in the trough period is less than or equal to the specified ratio of the segment statistical characteristics (such as 0.5).

[0116] The fast Fourier transform and other algorithms are used to perform spectrum analysis on the traffic segments during peak and trough periods, and the autocorrelation and other functions are used to calculate the correlation under different time delays to obtain the periodic regularity information of the traffic segments.

[0117] In one example, a multi-level timeline is constructed to support four granularities: year, month, day, and hour. The earliest timestamp of the traffic data queried from the database is 2023-01-01 00:00:00, and the latest timestamp is 2023-12-31 23:59:59, and the range of the timeline is set accordingly.

[0118] When the user selects the date 2023-06-15, the binary search algorithm is used to perform log2365≈9 comparisons to quickly locate the date.

[0119] For the selected 24-hour time window, the appropriate time interval is calculated to be 30 minutes. Use a logarithmic scale to handle time spans, such as 5 minutes for 0-1 hour, 15 minutes for 1-6 hours, and 30 minutes for 6-24 hours.

[0120] Get the traffic data for the whole day of 2023-06-15 from the database, segment it into 30-minute intervals, and generate 48 data points. Draw an area chart to represent the traffic trend, and use the shades of blue to represent the traffic size, 0-50MB / s is light blue, 50-100MB / s is medium blue, and above 100MB / s is dark blue. Mark the traffic data at the two peak points of 8:00 and 20:00. Use the linear interpolation algorithm to generate 29 intermediate frames between two adjacent 30-minute data points to achieve a smooth transition animation. A 60-minute sliding window is used to calculate the traffic characteristics, and the average daily traffic is 75MB / s, the maximum value is 150MB / s (20:00), and the minimum value is 20MB / s (3:00 am).

[0121] Using the 4-hour moving average, we can determine that 8:00-11:00 and 19:00-22:00 are peak periods, and 0:00-6:00 is the trough period. Through fast Fourier transform and autocorrelation function analysis, we found that the traffic data of that day showed an obvious 12-hour periodicity, which is speculated to be related to the work and rest schedule of the working day.

[0122] Step 107: Identify the key conduction points that cause traffic mutations in the power Internet of Things based on the distribution law information and the periodic law information.

[0123] In this embodiment, the relationship between traffic segments is analyzed by using distribution law information and periodic law information, and the correlation between traffic fragments of different device types, different time periods, and different traffic levels is explored. The matrix and Sankey diagram are used as a visualization method to show the influence transmission relationship between traffic segments, and the key transmission points that cause traffic mutations in the power Internet of Things are analyzed.

[0124] In a specific implementation, the device type, time period and traffic level are used as combination conditions, and traffic segments that meet the same combination conditions are filtered as segment subsets.

[0125] The segment subsets may be pre-processed so that they are encoded and labeled in the same format using a combination of device type, time period, and traffic level as a unique identifier.

[0126] The distribution law information and the periodic law information are used as elements to construct the association matrix between the segment subsets.

[0127] The correlation coefficient and mutual information value of the association matrix were calculated as indicators of association strength.

[0128] The association rules between segment subsets are mined based on the association strength index using algorithms such as Apriori to generate frequent item sets, where the minimum support of frequent item sets is 5% and the confidence threshold is 80%.

[0129] Extract some association rules with strong correlation from frequent item sets to form a set of association rules for segmented subsets.

[0130] The influence transmission network between segment subsets is constructed using the association matrix and the association rule set.

[0131] Use PageRank and other algorithms to calculate the importance of nodes according to the in-degree (the number of connections pointing to the node) and out-degree (the number of connections starting from the node) of each node in the influence conduction network, and select the nodes with greater influence in turn, so as to draw a flow transmission relationship diagram according to importance.

[0132] At this point, the influence conduction network can be converted into a visual data structure, generating a correlation matrix heat map, using a red-yellow-blue spectrum to represent the strength of the association, highlighting the highly correlated segmented subsets, and building a Sankey diagram data model. The node size is set to be proportional to the importance, and the connection width represents the flow conduction strength. A flow conduction relationship diagram between segmented subsets is drawn, and key nodes are marked.

[0133] In the traffic transmission relationship diagram, if the traffic change rate of a node in adjacent time periods is greater than or equal to a preset mutation threshold (such as 50%), the node is determined to be a key transmission point causing traffic mutation.

[0134] In one example, the flow data of a smart substation is processed, and the equipment type (such as "transformer"), time period (such as "2023-07-01_08:00-09:00"), and flow level (such as "high flow") are combined into a unique identifier "TR_20230701_0800_0900_H", and a 100x100 association matrix is ​​constructed, and the correlation coefficient is 0.85 and the mutual information value is 0.72.

[0135] Using the Apriori algorithm, setting the support to 5% and the confidence to 80%, 50 frequent item sets were mined from 1000 records and 20 strong association rules were extracted.

[0136] In the influence transmission network, a key node is identified with an in-degree of 15, an out-degree of 8, and a PageRank score of 0.086.

[0137] When generating the heat map, the correlation strength of 0-0.3 is mapped to blue, 0.3-0.7 is mapped to yellow, and 0.7-1 is mapped to red.

[0138] In the Sankey diagram, the node size range is set to 10-50 pixels, corresponding linearly to the importance 0-0.1.

[0139] The connection width is set to 1-10 pixels, indicating a traffic intensity of 100KB / s-1GB / s. It is detected that during the period of "2023-07-01_09:00-10:00", the traffic volume suddenly increased from 500MB / s to 800MB / s, with a change rate of 60%, exceeding the threshold of 50%. This period is marked as a key conduction point for traffic mutation, and is highlighted with a flashing red border in the Sankey diagram; this can successfully identify the key power equipment and time periods that cause traffic mutations in the substation, providing data support for further fault diagnosis and optimization.

[0140] Step 108: Detect abnormal power equipment based on the real-time flow segment of the key transmission point as abnormal equipment.

[0141] In this embodiment, a flow anomaly discrimination model can be established based on the real-time flow segments of key conduction points, and the flow patterns of each flow segment can be monitored in real time. By judging multi-dimensional attributes, flow anomalies within a single flow segment and across flow segments can be identified, and the source of the abnormal flow (power equipment) can be tracked and located and recorded as abnormal equipment.

[0142] In the specific implementation, multi-dimensional attribute data of historical traffic segments are collected from key transmission points, and the attribute data is represented in the form of vectors; among them, the attribute data includes traffic size, number of packets, protocol type distribution and time interval.

[0143] The historical attribute data are reduced in dimension using algorithms such as principal component analysis. The steps of principal component analysis include feature standardization, calculation of covariance matrix, eigenvalue decomposition, and selection of principal components with a cumulative contribution rate of 95% to obtain attribute data after dimension reduction.

[0144] If the dimensionality reduction process is completed, the isolation forest algorithm is used to train the anomaly detection model on the historical attribute data to determine the boundary of anomaly discrimination; in the isolation forest algorithm, the proportion of abnormal traffic segments to the total traffic segments is set to 1%, and the training data set (historical attribute data) is randomly sampled and the tree structure is constructed.

[0145] The attribute data of real-time traffic segments are collected from key transmission points. The attribute data is represented in the form of vectors. The attribute data includes traffic size, number of packets, protocol type distribution and time interval.

[0146] Use algorithms such as principal component analysis to reduce the dimension of real-time attribute data.

[0147] If the dimensionality reduction process is completed, the real-time attribute data is input into the anomaly detection model to calculate the anomaly value.

[0148] If the abnormal value of a single key conduction point exceeds a preset abnormal threshold, or the flow change rate between adjacent key conduction points is greater than or equal to a preset fluctuation threshold (such as 50%), an abnormal event is generated for the key conduction point.

[0149] The abnormal threshold may be calculated using an adaptive algorithm. Exemplarily, the abnormal threshold is N times (eg, 3 times) the standard deviation of all abnormal values.

[0150] In abnormal events, different weights can be assigned to abnormal indicators of different dimensions such as traffic size, number of packets, and protocol distribution, and the comprehensive abnormal score can be calculated by weighted summation.

[0151] For detected abnormal events, a graph traversal-based backtracking method can be used to construct a network topology map, perform a depth-first search in the network topology map starting from the abnormal event, and trace back the traffic path from the key transmission point.

[0152] According to the timing changes and topological relationships of the traffic path, the power equipment (source equipment or network node) that generates the abnormality is located as the abnormal device.

[0153] Generate an exception report for abnormal devices using the exception type, occurrence time, involved devices, and abnormality severity.

[0154] In one example, in the power Internet of Things flow anomaly detection, a 10-dimensional flow feature vector is constructed, including attributes such as flow per second, number of packets, TCP proportion, UDP proportion, HTTP proportion, and average packet size. After standardizing 10 million historical flow data, principal component analysis is performed to calculate the characteristic values ​​[5.2, 2.8, 1.5, 0.3, 0.1, 0.1]. The first three principal components are selected, and the cumulative contribution rate reaches 95%, reducing the 10-dimensional features to 3 dimensions. The anomaly detection model is trained using the isolation forest algorithm, 100 decision trees are set, the sampling size is 256, and the anomaly ratio is 1%. The real-time flow data is processed. For example, the characteristic vector of a data point after dimensionality reduction is [0.8, -0.5, 0.3]. The input anomaly detection model calculates the anomaly value of 0.76, which exceeds the anomaly threshold of 0.72 (3 times the standard deviation) and is determined to be a single flow segment anomaly. At the same time, the flow change rate of adjacent 5-minute segments is calculated to be 62%, which exceeds the 50% threshold and is determined to be a cross-flow segment anomaly. The traffic size weight is set to 0.4, the packet quantity weight is set to 0.3, and the protocol distribution weight is set to 0.3. The calculated comprehensive anomaly score is 0.85. After the anomaly is detected, a network topology diagram containing 100 nodes is constructed. Using depth-first search, backtracking from the anomaly point, after 7 hops, the source device IP is located at 192.168.1.57. An anomaly report is generated: the type is "traffic surge", the occurrence time is "2023-07-1514:30:25", involving the power equipment "edge router R3", and the anomaly level is "serious".

[0155] Step 109: Execute the flow control strategy on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device.

[0156] In this embodiment, a flow control strategy can be dynamically generated for the identified abnormal devices. For example, threshold rules for abnormal flow can be set to achieve precise flow limiting, or abnormal flow can be guided through path scheduling, etc., to reduce the impact of abnormal flow on the power Internet of Things.

[0157] In a specific implementation, abnormal features are extracted from the traffic segments corresponding to the abnormal devices; the abnormal features include traffic size, duration, protocol type, and source and destination addresses.

[0158] The exponentially weighted moving average (EWMA) is performed on the abnormal features to obtain a dynamic threshold for limiting the flow data of abnormal devices.

[0159] Build traffic control rules based on dynamic thresholds.

[0160] Load the network topology diagram of the power Internet of Things. In the network topology diagram, the SNMP protocol can be used to regularly count traffic information based on the interfaces of power equipment to determine bandwidth information (such as remaining bandwidth, etc.) and load information.

[0161] For abnormal traffic, algorithms such as the shortest path are used in the network topology diagram to generate alternative paths to guide the traffic data of abnormal devices.

[0162] The alternative paths are sorted using algorithms such as weighted scoring according to their bandwidth information and load information, and the optimal alternative path is selected from the sorted alternative paths to obtain a traffic scheduling solution.

[0163] Combine traffic control rules and traffic scheduling schemes into a complete traffic control strategy.

[0164] A configuration management system based on the NETCONF protocol is used to convert traffic control policies into device configuration commands, which are then distributed to power devices for execution.

[0165] Continuously monitor the change information of abnormal traffic data and the execution effect of the traffic control strategy. If the change information of abnormal traffic data is greater than or equal to the change threshold, the strategy optimization process is triggered. At this time, the reinforcement learning algorithm can be used to dynamically adjust the traffic control parameters in the traffic control strategy to optimize the control effect. Implement traffic diversion and caching to redirect some non-critical business traffic to alternative paths or temporary storage to reduce the impact of abnormal traffic on the core business of the power Internet of Things.

[0166] In one example, when abnormal traffic of a substation gateway device is detected, the traffic size is 50Mbps, the duration is 30 minutes, and it is mainly TCP protocol. The dynamic threshold is calculated using the EWMA method, and the smoothing coefficient α is set to 0.3, and the current dynamic threshold is 30Mbps. Generate a traffic control rule: limit the TCP traffic of the gateway to no more than 35Mbps. Subsequently, the network topology information is collected through the SNMP protocol, and 3 available paths are found, with the remaining bandwidths of 100Mbps, 80Mbps and 60Mbps, and the node CPU loads of 30%, 40% and 20%. The Dijkstra algorithm is used to calculate the shortest alternative path, and the first alternative path is selected as the optimal scheduling solution by combining the bandwidth information and load information. The control strategy is converted into a configuration command in XML format using the NETCONF protocol and sent to the gateway device and related routers. After 30 minutes of executing the strategy, it is detected that the traffic has dropped to 40Mbps, but it still exceeds the dynamic threshold. Trigger the reinforcement learning algorithm, use the Q-learning method, set the learning rate to 0.1, and the discount factor to 0.9. After 100 iterations, the new dynamic threshold of the traffic limit is optimized to 38Mbps. At the same time, 10Mbps of non-critical data traffic is redirected to the backup link and cached at the edge node for 5 minutes, effectively reducing the impact on the core network.

[0167] In this embodiment, flow data segmentation rules are established for various types of power equipment in the power Internet of Things in multiple dimensions; when real-time flow data is collected from the power equipment, the real-time flow data is divided into multiple flow segments according to the segmentation rules; the flow pattern of the flow segment is identified according to the characteristics of the flow segment in multiple dimensions; visual statistical data is constructed for the flow segment according to the flow pattern; when receiving an interactive operation for statistical data, the distribution law information of part of the statistical data indicated by the interactive operation is obtained; periodic regularity information is mined for part of the flow segment indicated by the interactive operation in time; the key conduction points causing the sudden change of flow are identified in the power Internet of Things according to the distribution law information and the periodic law information; the power equipment that generates abnormality according to the real-time flow segment of the key conduction point is detected as an abnormal device; the flow control strategy is executed on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device. This embodiment reasonably segments the flow data, greatly reduces the amount of data loaded at one time, reduces the phenomenon of slow page loading and interaction jamming, effectively improves the user experience, and the small-scale data is visually neater, and the user can quickly locate the information of interest.

[0168] In addition, this embodiment mines the key conduction points that cause traffic mutations based on the distribution and periodicity patterns of traffic data, thereby formulating corresponding traffic control strategies, restricting and guiding abnormal traffic, and effectively reducing the impact of abnormal traffic on the core business of the power Internet of Things, thereby ensuring the normal operation of the power Internet of Things.

[0169] Embodiment 2

[0170] See also Figure 2 , shows a schematic diagram of the structure of a device for processing abnormal traffic in the power Internet of Things provided by the second embodiment of the present invention. Figure 2 As shown, the device comprises:

[0171] The segmentation rule establishment module 201 is used to establish segmentation rules for flow data in multiple dimensions for various types of power equipment in the power Internet of Things;

[0172] A flow segment division module 202, for dividing the real-time flow data into a plurality of flow segments according to the segmentation rule when real-time flow data is collected from the electric power equipment;

[0173] A traffic pattern recognition module 203, configured to recognize the traffic pattern of the traffic segment according to the multi-dimensional features of the traffic segment;

[0174] A statistical data building module 204, configured to build visualized statistical data for the traffic segment according to the traffic pattern;

[0175] A statistical data generating module 205, configured to generate distribution regularity information for a portion of the statistical data indicated by the interactive operation when an interactive operation for the statistical data is received;

[0176] A periodic regularity information mining module 206, configured to mine periodic regularity information for a portion of the traffic segments indicated by the interactive operation in time;

[0177] A key transmission point identification module 207, configured to identify key transmission points causing flow mutations in the power Internet of Things based on the distribution law information and the periodic law information;

[0178] An abnormal device detection module 208 is used to detect the power device that generates abnormality according to the real-time flow segment of the key transmission point as an abnormal device;

[0179] The flow control strategy execution module 209 is used to execute the flow control strategy on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device.

[0180] In one embodiment of the present invention, the segmentation rule establishing module 201 is further used for:

[0181] Collect flow data of various types of power equipment in the power Internet of Things as samples;

[0182] Preprocessing the flow data as a sample according to the device type to obtain structured data;

[0183] Performing hierarchical clustering on the structured data to obtain a similarity matrix between the power devices;

[0184] In the similarity matrix, similar power devices are merged layer by layer starting from the bottom layer into a specified number of device clusters;

[0185] For the power devices in the device cluster, multiple rule thresholds are set in the dimensions of time period, traffic size and protocol type as candidate rules;

[0186] Performing k-fold cross validation on the candidate rules to obtain a validation result;

[0187] The combination mode between the rule threshold in the candidate rule and the rule threshold is adjusted according to the verification result to obtain a segmentation rule.

[0188] In one embodiment of the present invention, the traffic segment division module 202 is further configured to:

[0189] In a distributed framework, real-time flow data is collected from the power equipment at a preset collection frequency; the real-time flow data carries a timestamp and a device identifier;

[0190] Calling a parser that matches the protocol type of the real-time traffic data to pre-process the real-time traffic data to obtain target data;

[0191] Taking the equipment type to which the electric power equipment belongs as an independent topic, the target data is pushed to a preset message system in real time;

[0192] Subscribe to target data under a specified topic from the message system;

[0193] Using a rule engine to read a segmentation rule configured for a device type to which the electric power device belongs, and splitting the target data into a plurality of flow segments according to the segmentation rule;

[0194] The traffic segment is stored in a preset distributed database, wherein a composite primary key of the distributed database includes the device type, the time period of the traffic segment, and the device identifier.

[0195] In one embodiment of the present invention, the traffic pattern identification module 203 is also used to:

[0196] Reading multiple frames of raw data from the traffic segment;

[0197] Add a time window to the multiple frames of raw data, and extract multi-dimensional raw traffic features from the raw data in the time window; the raw traffic features include the total amount of data, average packet size, maximum packet size, minimum packet size, packet sending frequency, and protocol type ratio;

[0198] Performing Z-score normalization processing on the original flow characteristics to obtain target flow characteristics;

[0199] Performing K-means clustering on the traffic segments according to the target traffic characteristics to obtain multiple traffic clusters;

[0200] A denormalization operation is performed on the center point of each of the traffic clusters to obtain a traffic pattern.

[0201] In one embodiment of the present invention, the statistical data generating module 205 is further used for:

[0202] Calculating a specified quantile for the historical flow data of the electric power equipment as a flow threshold;

[0203] If the flow pattern of the flow segment is greater than the flow threshold, it is determined that the flow segment meets the large flow feature;

[0204] If the flow pattern of the flow segment is less than or equal to the flow threshold, it is determined that the flow segment meets the low flow feature;

[0205] For the flow segment that meets the large flow characteristics, generate flow map data by combining the geographical location of the power equipment with the flow segment;

[0206] For the traffic segments meeting the small traffic characteristics, pie chart data is generated using the proportions of the traffic segments under various protocol types;

[0207] For the traffic segment that meets the low traffic characteristics, bar chart data is generated using change information of the traffic segment in each time period.

[0208] In one embodiment of the present invention, the periodic regularity information mining module 206 is further used for:

[0209] receiving an interactive operation performed on the statistical data that is adapted to the device type of the electric device and the traffic pattern;

[0210] calculating at least one of a scaling factor, a translation vector, and a drill-down level for the statistical data according to the interaction operation;

[0211] updating the statistical data according to at least one of the scaling factor, the translation vector, and the drill-down level;

[0212] Generate a distribution threshold value based on the statistical characteristics of the historical traffic data; the statistical characteristics include the total traffic volume, peak value and average value;

[0213] Filtering out original data that meets the distribution threshold from the portion of the traffic segments indicated by the interactive operation as the data of interest;

[0214] K-means clustering is performed on the interest data to obtain distribution law information.

[0215] In one embodiment of the present invention, the periodic regularity information mining module 206 is further used for:

[0216] Mapping the portion of the traffic segments indicated by the interactive operation onto a time axis;

[0217] When a positioning operation is received, querying a time point corresponding to the positioning operation on the time axis, and performing a scaling operation and / or a translation operation on the traffic segment based on the time point;

[0218] If the zooming operation and / or the panning operation is completed, the time axis is divided into a plurality of time periods according to the specified time intervals;

[0219] Generate visualized time series data for the traffic segments in the multiple time periods to characterize the changing trend of the traffic segments over time;

[0220] Generating segment statistical features for the traffic segments of the plurality of time periods; the segment statistical features comprising an average value, a maximum value and a minimum value;

[0221] The traffic segments of the multiple time periods are divided into peak periods and valley periods according to the segment statistical features; the value of the traffic segments in the peak period is greater than a specified multiple of the segment statistical features, and the value of the traffic segments in the valley period is less than or equal to a specified ratio of the segment statistical features;

[0222] The traffic segments in the peak period and the valley period are subjected to spectrum analysis, and correlations under different time delays are calculated to obtain periodic regularity information of the traffic segments.

[0223] In one embodiment of the present invention, the key conduction point identification module 207 is further used to:

[0224] Taking device type, time period and traffic level as combination conditions, filtering the traffic segments meeting the same combination conditions as segment subsets;

[0225] Using the distribution law information and the periodic law information to construct a correlation matrix between the segment subsets;

[0226] Calculating the correlation coefficient and mutual information value of the association matrix as an indicator of association strength;

[0227] mining association rules between the segment subsets based on the association strength index to generate frequent item sets;

[0228] Extracting part of the association rules from the frequent item set to form an association rule set of the segment subset;

[0229] Using the association matrix and the association rule set to construct an influence transmission network between the segment subsets;

[0230] Calculate the importance of each node in the influence transmission network according to the in-degree and out-degree of each node;

[0231] Draw a traffic transmission relationship diagram according to the importance;

[0232] In the traffic transmission relationship diagram, if the traffic change rate of the node in adjacent time periods is greater than or equal to a preset mutation threshold, the node is determined to be a key transmission point causing the traffic mutation.

[0233] In one embodiment of the present invention, the abnormal device detection module 208 is further used to:

[0234] Collecting attribute data of the historical traffic segments from the key transmission points; the attribute data includes traffic size, number of packets, protocol type distribution and time interval;

[0235] Performing dimensionality reduction processing on the historical attribute data;

[0236] If the dimensionality reduction process is completed, the isolation forest algorithm is used to train the anomaly detection model on the historical attribute data;

[0237] Collecting real-time attribute data of the traffic segment from the key conduction point;

[0238] Performing dimensionality reduction processing on the real-time attribute data;

[0239] If the dimension reduction process is completed, the real-time attribute data is input into the anomaly detection model to calculate the anomaly value;

[0240] If the abnormal value exceeds a preset abnormal threshold, an abnormal event is generated for the key conduction point;

[0241] Perform a depth-first search starting from the abnormal event and trace back the traffic path starting from the key transmission point;

[0242] According to the time sequence change and topological relationship of the flow path, the power equipment that has abnormality is located as an abnormal equipment;

[0243] An exception report is generated for the abnormal device using the exception type, occurrence time, involved devices and abnormality severity.

[0244] In one embodiment of the present invention, the flow control strategy execution module 209 is further used to:

[0245] Extracting abnormal features from the traffic segment corresponding to the abnormal device; the abnormal features include traffic size, duration, protocol type, and source and destination addresses;

[0246] Performing an exponentially weighted moving average on the abnormal features to obtain a dynamic threshold for limiting the flow data of the abnormal device;

[0247] Building a flow control rule based on the dynamic threshold;

[0248] Loading the network topology diagram of the electric power Internet of Things;

[0249] generating, in the network topology map, an alternative path for directing the flow data of the abnormal device;

[0250] Selecting the best candidate path from the candidate paths according to the bandwidth information and load information of the candidate paths to obtain a traffic scheduling solution;

[0251] Combining the flow control rule and the flow scheduling scheme into a flow control strategy;

[0252] The flow control strategy is converted into a device configuration command, and the device configuration command is distributed to the power device for execution.

[0253] The device for processing abnormal traffic in the electric power Internet of Things provided in an embodiment of the present invention can execute the method for processing abnormal traffic in the electric power Internet of Things provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to executing the method for processing abnormal traffic in the electric power Internet of Things.

[0254] Embodiment 3

[0255] See also Figure 3 , shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0256] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0257] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0258] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for processing abnormal traffic in the power Internet of Things.

[0259] In some embodiments, the method for processing abnormal traffic in the electric power Internet of Things may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for processing abnormal traffic in the electric power Internet of Things described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for processing abnormal traffic in the electric power Internet of Things in any other appropriate manner (e.g., by means of firmware).

[0260] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0261] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0262] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0263] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0264] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0265] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0266] Embodiment 4

[0267] An embodiment of the present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for processing abnormal traffic in the power Internet of Things provided by any embodiment of the present invention.

[0268] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0269] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0270] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for processing abnormal traffic in an electric power Internet of Things, characterized in that: include: Establish segmentation rules for flow data in multiple dimensions for various types of power equipment in the power Internet of Things; When real-time flow data is collected from the electric power equipment, the real-time flow data is divided into a plurality of flow segments according to the segmentation rule; Identifying a traffic pattern of the traffic segment based on the multi-dimensional features of the traffic segment; Constructing visualized statistical data for the traffic segments according to the traffic patterns; When receiving an interactive operation for the statistical data, generating distribution law information for a portion of the statistical data indicated by the interactive operation; Mining periodic regularity information for part of the traffic segments indicated by the interactive operation in time; Identifying a key conduction point causing a sudden change in flow rate in the electric power Internet of Things based on the distribution law information and the periodic law information; Detecting the abnormal power equipment according to the real-time flow segment of the key conduction point as an abnormal equipment; A flow control strategy is executed on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device.

2. The method according to claim 1, characterized in that The segmentation rules for establishing flow data for various types of power equipment in the power Internet of Things in multiple dimensions include: Collect flow data of various types of power equipment in the power Internet of Things as samples; Preprocessing the flow data as a sample according to the device type to obtain structured data; Performing hierarchical clustering on the structured data to obtain a similarity matrix between the power devices; In the similarity matrix, similar power devices are merged layer by layer starting from the bottom layer into a specified number of device clusters; For the power devices in the device cluster, multiple rule thresholds are set in the dimensions of time period, traffic size and protocol type as candidate rules; Performing k-fold cross validation on the candidate rules to obtain a validation result; The combination mode between the rule threshold in the candidate rule and the rule threshold is adjusted according to the verification result to obtain a segmentation rule.

3. The method according to claim 1, characterized in that When the real-time flow data is collected from the electric power equipment, the real-time flow data is divided into a plurality of flow segments according to the segmentation rule, including: In a distributed framework, real-time flow data is collected from the power equipment at a preset collection frequency; the real-time flow data carries a timestamp and a device identifier; Calling a parser that matches the protocol type of the real-time traffic data to pre-process the real-time traffic data to obtain target data; Taking the equipment type to which the electric power equipment belongs as an independent topic, the target data is pushed to a preset message system in real time; Subscribe to target data under a specified topic from the message system; Using a rule engine to read a segmentation rule configured for a device type to which the electric power device belongs, and splitting the target data into a plurality of flow segments according to the segmentation rule; Storing the traffic segment in a preset distributed database, wherein a composite primary key of the distributed database includes the device type, the time period of the traffic segment, and the device identifier; The identifying the traffic pattern of the traffic segment according to the multi-dimensional features of the traffic segment includes: Reading multiple frames of raw data from the traffic segment; Add a time window to the original data of multiple frames, and extract multi-dimensional original traffic features from the original data in the time window; the original traffic features include the total amount of data, average packet size, maximum packet size, minimum packet size, packet sending frequency and protocol type ratio; Performing Z-score normalization processing on the original flow characteristics to obtain target flow characteristics; Performing K-means clustering on the traffic segments according to the target traffic characteristics to obtain multiple traffic clusters; A denormalization operation is performed on the center point of each of the traffic clusters to obtain a traffic pattern.

4. The method according to claim 1, characterized in that: The constructing of visualized statistical data for the traffic segment according to the traffic pattern includes: Calculating a specified quantile for the historical flow data of the electric power equipment as a flow threshold; If the flow pattern of the flow segment is greater than the flow threshold, it is determined that the flow segment meets the large flow feature; If the flow pattern of the flow segment is less than or equal to the flow threshold, it is determined that the flow segment meets the low flow feature; For the flow segment that meets the large flow characteristics, generate flow map data by combining the geographical location of the power equipment with the flow segment; For the traffic segments meeting the small traffic characteristics, pie chart data is generated using the proportions of the traffic segments under various protocol types; For the traffic segments meeting the low traffic characteristics, bar chart data is generated using change information of the traffic segments in each time period.

5. The method according to claim 1, characterized in that The generating distribution regularity information for a portion of the statistical data indicated by the interactive operation when the interactive operation for the statistical data is received includes: receiving an interactive operation performed on the statistical data that is adapted to the device type of the electric device and the traffic pattern; calculating at least one of a scaling factor, a translation vector, and a drill-down level for the statistical data according to the interaction operation; updating the statistical data according to at least one of the scaling factor, the translation vector, and the drill-down level; Generate a distribution threshold value based on the statistical characteristics of the historical traffic data; the statistical characteristics include the total traffic volume, peak value and average value; Filtering out original data that meets the distribution threshold from the portion of the traffic segments indicated by the interactive operation as the data of interest; Performing K-means clustering on the interest data to obtain distribution law information; The mining of periodic regularity information for the portion of the traffic segments indicated by the interactive operation in time includes: Mapping the portion of the traffic segments indicated by the interactive operation onto a time axis; When receiving a positioning operation, querying a time point corresponding to the positioning operation on the time axis, and performing a scaling operation and / or a translation operation on the traffic segment based on the time point; If the zooming operation and / or the panning operation is completed, the time axis is divided into a plurality of time periods according to the specified time intervals; Generate visualized time series data for the traffic segments in the multiple time periods to characterize the changing trend of the traffic segments over time; Generating segment statistical features for the traffic segments of the plurality of time periods; the segment statistical features comprising an average value, a maximum value and a minimum value; The traffic segments of the multiple time periods are divided into peak periods and valley periods according to the segment statistical features; the value of the traffic segments in the peak period is greater than a specified multiple of the segment statistical features, and the value of the traffic segments in the valley period is less than or equal to a specified ratio of the segment statistical features; The traffic segments in the peak period and the valley period are subjected to spectrum analysis, and correlations under different time delays are calculated to obtain periodic regularity information of the traffic segments.

6. The method according to any one of claims 1 to 5, characterized in that The step of identifying a key conduction point causing a sudden change in flow rate in the electric power Internet of Things based on the distribution law information and the periodic law information includes: Taking device type, time period and traffic level as combination conditions, filtering the traffic segments meeting the same combination conditions as segment subsets; Using the distribution law information and the periodic law information to construct a correlation matrix between the segment subsets; Calculating the correlation coefficient and mutual information value of the association matrix as an indicator of association strength; mining association rules between the segment subsets based on the association strength index to generate frequent item sets; Extracting part of the association rules from the frequent item set to form an association rule set of the segment subset; Using the association matrix and the association rule set to construct an influence transmission network between the segment subsets; Calculate the importance of each node in the influence transmission network according to the in-degree and out-degree of each node; Draw a traffic transmission relationship diagram according to the importance; In the traffic transmission relationship diagram, if the traffic change rate of the node in adjacent time periods is greater than or equal to a preset mutation threshold, the node is determined to be a key transmission point causing the traffic mutation.

7. The method according to any one of claims 1 to 5, characterized in that The power equipment that generates abnormality according to the real-time flow segment detection of the key conduction point, as the abnormal equipment, includes: Collecting attribute data of the historical traffic segments from the key transmission points; the attribute data includes traffic size, number of packets, protocol type distribution and time interval; Performing dimensionality reduction processing on the historical attribute data; If the dimensionality reduction process is completed, the isolation forest algorithm is used to train the anomaly detection model on the historical attribute data; Collecting real-time attribute data of the traffic segment from the key conduction point; Performing dimensionality reduction processing on the real-time attribute data; If the dimension reduction process is completed, the real-time attribute data is input into the anomaly detection model to calculate the anomaly value; If the abnormal value exceeds a preset abnormal threshold, an abnormal event is generated for the key conduction point; Perform a depth-first search starting from the abnormal event and trace back the traffic path starting from the key transmission point; According to the time sequence change and topological relationship of the flow path, the power equipment that has abnormality is located as an abnormal equipment; An exception report is generated for the abnormal device using the exception type, occurrence time, involved devices and abnormality severity.

8. The method according to any one of claims 1 to 5, characterized in that The executing a flow control strategy on the abnormal device to limit and / or guide the flow data corresponding to the abnormal device includes: Extracting abnormal features from the traffic segment corresponding to the abnormal device; the abnormal features include traffic size, duration, protocol type, and source and destination addresses; Performing an exponentially weighted moving average on the abnormal features to obtain a dynamic threshold for limiting the flow data of the abnormal device; Building a flow control rule based on the dynamic threshold; Loading the network topology diagram of the electric power Internet of Things; generating, in the network topology map, an alternative path for directing the flow data of the abnormal device; Selecting the best candidate path from the candidate paths according to the bandwidth information and load information of the candidate paths to obtain a traffic scheduling solution; Combining the flow control rule and the flow scheduling scheme into a flow control strategy; The flow control strategy is converted into a device configuration command, and the device configuration command is distributed to the power device for execution.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for processing abnormal traffic in the electric power Internet of Things as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for processing abnormal traffic in the electric power Internet of Things according to any one of claims 1 to 8 is implemented.

Citation Information

Cited By

  • Real-time isolation method and device based on AI traffic anomaly recognition

    CN120528703A

  • Real-time isolation method and device based on AI traffic anomaly identification

    CN120528703B

  • Power network abnormal flow detection method and system based on edge calculation

    CN120567483A

  • Power network abnormal traffic detection method and system based on edge computing

    CN120567483B

  • Ai-based security policy learning and anomaly detection system for household networks

    KR103009930B1