Power grid flow data acquisition and analysis method and device, power system and related equipment

Abstract

The invention provides a power grid flow data acquisition and analysis method, and the method comprises the steps: determining an acquisition channel matched with to-be-acquired flow in response to a flow acquisition starting command; collecting flow to be collected through the collection channel, and preprocessing the collected flow data; analyzing the preprocessed flow data to obtain service data; calculating related business indexes according to the business data, and performing early warning on related businesses according to a calculation result; performing correlation analysis and traceability analysis on each business index and the data factor thereof, and visually outputting a correlation analysis result and a traceability analysis result; according to the invention, the traffic data can be received through multiple channels, the comprehensive coverage and real-time monitoring of the network traffic are realized, the data of different channels are captured at the same time, and the integrity and diversity of the data are ensured; the system is suitable for power systems.

Description

Technical Field

[0001] The present application relates to the technical field of power systems, and in particular to a method and device for collecting and analyzing power grid flow data, a power system and related equipment. Background Art

[0002] In today's highly information-based and intelligent era, network equipment in power systems is becoming increasingly complex and large-scale. This complex network system not only carries a large amount of power transmission and distribution tasks, but also involves frequent interactions between many applications. Therefore, building an efficient and accurate network acquisition and analysis system is of great significance to ensuring the safe and stable operation of the power system and improving energy efficiency.

[0003] First of all, the network system in the power system is composed of numerous power plants, substations, transmission lines, distribution networks and other infrastructure. These facilities are interconnected through complex communication protocols and technical means, forming a huge power Internet of Things. In this network, data is frequently exchanged between various devices, systems, and applications. The real-time, accuracy, and integrity of these data are directly related to the safe, stable, and efficient operation of the power system.

[0004] Secondly, in the network system, flow data is an important indicator reflecting the network operation status and business activities. For the power system, flow data can not only help operation and maintenance personnel to grasp the network operation status in real time, and promptly discover and deal with potential safety hazards and fault problems; it can also provide important data support for power system planning, scheduling, and optimization. Through in-depth analysis of flow data, it can reveal the relationship between different businesses in the power system, data flow direction, and flow distribution law, and provide strong support for the intelligent management and decision-making of the power system.

[0005] In addition, due to the expansion of the scale of the power system and the increase in the access to new energy sources, the frequency and magnitude of data interaction are also increasing, which puts higher demands on data collection, storage, processing and analysis capabilities.

[0006] In short, building an efficient, accurate and intelligent network data collection and analysis system is of great significance to the power system. This system can not only improve the operating efficiency and safety of the network system in the power system, but also provide strong support for the intelligent transformation and high-quality development of the power system. Summary of the invention

[0007] In order to solve one of the above-mentioned technical defects, the present application provides a power grid flow data collection and analysis method, device, power system and related equipment.

[0008] According to a first aspect of the present application, a method for collecting and analyzing power grid flow data is provided, the method comprising: In response to a flow collection start command, determining a collection channel that matches the flow to be collected; The flow to be collected is collected through the collection channel, and the collected flow data is pre-processed; Analyze the pre-processed traffic data to obtain business data; Calculate relevant business indicators based on business data, and issue early warnings for related businesses based on the calculation results; Conduct correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

[0009] Preferably, the determining of a collection channel matching the flow to be collected specifically includes: Determine whether the startup command contains the specified command of the acquisition channel; If a specified command is included, first search in the channel automatic matching table whether there is an automatic matching record between the flow to be collected and the specified collection channel; if the automatic matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the automatic matching record; if the automatic matching record does not exist, search in the channel specified matching table whether there is a specified matching record between the flow to be collected and the specified collection channel; if the specified matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the specified matching record; if the specified matching record does not exist, store the specified matching relationship between the flow to be collected and the specified collection channel in the channel specified matching table to form a new specified matching record, and determine the specified collection channel as the collection channel that matches the flow to be collected according to the new specified matching record; If the specified command is not included, the automatic matching records of the flow to be collected and all collection channels are searched in the channel automatic matching table according to the identification characteristics of the flow to be collected, and then all the matched collection channels are determined as the collection channels that match the flow to be collected according to the found automatic matching records.

[0010] Preferably, the step of determining a collection channel that matches the flow to be collected further includes: Count the number of matches of each specified matching record in the channel specified matching table; When the matching times of a certain designated matching record reaches the designated matching threshold value N, the designated matching record is transferred to the channel automatic matching table and stored as a new automatic matching record.

[0011] Preferably, the preprocessing of the collected flow data specifically includes: In the process of receiving traffic data, a large time window is specified in time units according to the order in which the traffic data passes, and the large time window is divided into multiple small time windows in seconds. Each small time window dynamically stores all traffic data packets that pass within the respective time range. After each flow data is received, when the flow data passes through each time window in turn, the flow data content is hashed and compared with the content of all flow data packets dynamically stored in each time window. If the hash results are consistent, other auxiliary parameters are compared. If all parameters are consistent, the flow data is determined to be duplicate flow data, and finally the duplicate flow data is removed according to the determination result; Use the BPF interface to receive the mirrored traffic data collected by different collection channels and perform preliminary filtering on the received traffic data; Perform preliminary analysis on the traffic data after preliminary filtering, identify the key information in the data packet header, decode the data packet body based on the key information, restore the original data, and then extract useful information from the data packet; The traffic data after the initial analysis is filtered for the second time, and the traffic data after the second filtering is stored.

[0012] Preferably, the parsing of the pre-processed traffic data to obtain the service data specifically includes: Deserialize the preprocessed traffic data and restore the collected data packets; Intercept the first five bytes of the restored data header and parse out the protocol version number and data length of the current data packet; Serialize the data with the protocol version number and data length, and compress the serialized data; Perform protocol identification and analysis on the compressed data, parse the data packets, and parse the original traffic data messages into application data that can be identified by the business.

[0013] Preferably, the correlation analysis and traceability analysis of each business indicator and its data factors specifically include: Using a timing calculation mechanism, the relevant statistics of each business indicator are calculated according to a set periodic timing, wherein the relevant statistics are one or more of an average value, a maximum value and a minimum value; By using the event trigger mechanism, after receiving the event notification sent by the set event bus, the trend data of each business indicator is calculated, where the trend data is one or more of the year-on-year, month-on-month and change rate; Based on the calculation results of relevant statistics and trend data, trend analysis is performed on each business indicator, the data fluctuation pattern within a period of time is calculated, and the change trend of key indicators at different time points is analyzed; According to the calculation formula of each business indicator, obtain the data factors that affect the calculation results of the indicator; Trace back the alarm status of a certain business indicator at a certain moment, analyze the correlation between different alarm information, and obtain the internal connection and mutual influence between various business indicators and data factors.

[0014] According to a second aspect of the present application, a power grid flow data collection and analysis device is provided, the device comprising a module for implementing the power grid flow data collection and analysis method as described above.

[0015] According to a third aspect of the present application, there is provided a power system, the power system comprising: Traffic collection end: used to respond to the traffic collection start command, determine the collection channel that matches the traffic to be collected; collect the traffic to be collected through the collection channel, pre-process the collected traffic data, and serialize the pre-processed traffic data and report it to the data analysis end; Data analysis end: used to parse the pre-processed traffic data reported by the traffic collection end to obtain business data; calculate relevant business indicators based on the business data, and issue early warnings for related businesses based on the calculation results; perform correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

[0016] According to a fourth aspect of the present application, a power grid flow data collection and analysis device is provided, the device comprising: Memory; Processor; and Computer programs; Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the power grid flow data collection and analysis method as described above.

[0017] According to a fifth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the power grid flow data collection and analysis method as described above.

[0018] The power grid flow data collection and analysis method provided in the present application can receive flow data through multiple channels, thereby achieving comprehensive coverage and real-time monitoring of network flow; data from different channels are captured simultaneously, ensuring the integrity and diversity of the data; the real-time monitoring capability enables network management personnel to quickly grasp the dynamic changes of network flow, and timely discover potential network congestion, abnormal access and other problems, providing strong support for network optimization and fault prevention; the intelligent processing and analysis system in the present application can deeply explore the value behind the data and accurately calculate various business indicators. These indicators not only reflect the real-time status of business operation, but also reveal the inherent laws and potential problems of business operation through trend analysis, correlation analysis and other methods, providing a scientific basis for business optimization, cost control and market expansion; in addition, through real-time analysis of flow data in the present application, the system can quickly identify abnormal flow patterns, determine whether they meet the alarm conditions, and immediately trigger an alarm notification. This rapid response mechanism greatly shortens the time interval from fault discovery to processing, reduces the impact of faults on business operations, and at the same time, the accuracy of the alarm information also ensures the pertinence and effectiveness of fault processing; by tracing back historical data, the present application can reproduce a certain state of business operation, helping managers to deeply understand the inherent mechanism of business operation.

[0019] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the contents indicated in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flow chart of the power grid flow data collection and analysis method provided in this application; Figure 2 A schematic diagram of a module of a power grid flow data acquisition and analysis device provided in this application; In the figure: 101 is a channel determination module, 102 is a collection module, 103 is a pre-processing module, 104 is a business data acquisition module, 105 is a business indicator calculation module, 106 is an early warning module, 107 is an analysis module, and 108 is a visualization module. DETAILED DESCRIPTION

[0021] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0022] In view of some problems existing in the prior art: In the first aspect, the present application provides a method for collecting and analyzing power flow data, which can be performed by a power flow data collection and analysis device, or by a component configured inside the power flow data collection and analysis device, such as a chip, a chip system, etc., or by a logic module or software having some or all of the functions of a power flow data collection and analysis device. This application does not limit this.

[0023] For example, Figure 1 As shown, the power grid flow data collection and analysis method includes: In response to a flow collection start command, determining a collection channel that matches the flow to be collected; The flow to be collected is collected through the collection channel, and the collected flow data is pre-processed; Analyze the pre-processed traffic data to obtain business data; Calculate relevant business indicators based on business data, and issue early warnings for related businesses based on the calculation results; Conduct correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

[0024] Based on the above scheme, the embodiment of the present application can receive traffic data through multiple channels, thereby achieving comprehensive coverage and real-time monitoring of network traffic; data from different channels are captured simultaneously, ensuring the integrity and diversity of the data; the real-time monitoring capability enables network management personnel to quickly grasp the dynamic changes of network traffic, and timely discover potential network congestion, abnormal access and other problems, providing strong support for network optimization and fault prevention; the intelligent processing and analysis system in the embodiment of the present application can deeply explore the value behind the data and accurately calculate various business indicators. These indicators not only reflect the real-time status of business operation, but also reveal the inherent laws and potential problems of business operation through trend analysis, correlation analysis and other methods, providing a scientific basis for business optimization, cost control and market expansion; in addition, through real-time analysis of traffic data in the embodiment of the present application, the system can quickly identify abnormal traffic patterns, determine whether they meet the alarm conditions, and immediately trigger an alarm notification. This rapid response mechanism greatly shortens the time interval from fault discovery to processing, reduces the impact of faults on business operations, and at the same time, the accuracy of the alarm information also ensures the pertinence and effectiveness of fault processing; the embodiment of the present application can reproduce a certain state of business operation by tracing back historical data, helping managers to deeply understand the inherent mechanism of business operation.

[0025] In some possible implementations of the first aspect, determining a collection channel that matches the flow to be collected specifically includes: Determine whether the startup command contains the specified command of the acquisition channel; If a specified command is included, first search in the channel automatic matching table whether there is an automatic matching record between the flow to be collected and the specified collection channel; if the automatic matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the automatic matching record; if the automatic matching record does not exist, search in the channel specified matching table whether there is a specified matching record between the flow to be collected and the specified collection channel; if the specified matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the specified matching record; if the specified matching record does not exist, store the specified matching relationship between the flow to be collected and the specified collection channel in the channel specified matching table to form a new specified matching record, and determine the specified collection channel as the collection channel that matches the flow to be collected according to the new specified matching record; If the specified command is not included, the automatic matching records of the flow to be collected and all collection channels are searched in the channel automatic matching table according to the identification characteristics of the flow to be collected (the identification characteristics may be type characteristics, regional characteristics, time domain characteristics, etc.), and then all the matched collection channels are determined as the collection channels that match the flow to be collected according to the found automatic matching records.

[0026] In specific implementation, each automatic matching record in the channel automatic matching table is actually a matching relationship between various types of flow data preset by the system and the corresponding collection channel. Each designated matching record in the channel designated matching table is a matching relationship between the flow data to be collected and the corresponding collection channel specified by the user when starting flow collection. The collection channel can be a network IP, port, protocol type, MAC address, etc. When the system receives a flow collection start command, it first determines whether there is a user-specified command for the collection channel in the start command. If there is a specified command, it means that the flow to be collected needs to be accurately collected. At this time, the channel automatic matching table is first queried to see whether the matching relationship specified by the user exists in the channel automatic matching table. If it exists, the collection channel specified by the user is directly used as the accurate collection channel for the flow to be collected. If not, it means that the channel automatic matching There is no user-specified matching relationship in the matching table; at this time, the channel-specified matching table is checked to see whether the user-specified matching relationship exists in the channel-specified matching table. If it exists, it means that the matching relationship has been specified by the user before, and the collection channel specified by the user will be used as the precise collection channel for the flow to be collected. If it does not exist, it means that the matching relationship specified by the user appears for the first time. At this time, the matching relationship is first stored to form a recorded matching relationship, and then the user-specified collection channel is used as the precise collection channel for the flow to be collected, so as to realize the precise collection of the flow to be collected; if no command is specified in the startup command, it means that the flow to be collected needs to be widely collected. At this time, all collection channels corresponding to the flow to be collected are matched in the channel automatic matching table according to the characteristics of the flow to be collected, and the flow to be collected is widely collected through these channels.

[0027] Based on the above scheme, the embodiments of the present application can, according to actual usage needs, not only accurately collect the traffic to be collected, reduce unnecessary data processing, save computing resources, and improve collection efficiency, but also widely collect the traffic to be collected, ensure the integrity and diversity of the data, and provide an excellent data foundation for subsequent analysis and processing.

[0028] Optionally, when extensively collecting the traffic to be collected, in order to further improve the data collection efficiency on the basis of ensuring data integrity and diversity, when the number of corresponding collection channels found in the channel automatic matching table is large and exceeds the channel threshold M (for example, M can be set to 10), according to the channel priority list, channels with the top M priorities are selected as the final collection channels; wherein the channel priority list is arranged according to the importance of the traffic data collected by each channel to each business indicator.

[0029] In some possible implementations of the first aspect, determining a collection channel that matches the flow to be collected specifically further includes: Count the number of matches of each specified matching record in the channel specified matching table; When the number of matches of a certain designated matching record reaches a designated matching threshold N (for example, N may be set to 5), the designated matching record is transferred to the channel automatic matching table and stored as a new automatic matching record.

[0030] Based on the above scheme, the system will count the number of matches of each designated matching record in the channel designated matching table. When the number of matches of a designated matching record reaches the designated matching threshold N (for example, N can be set to 5), it means that the matching relationship corresponding to the designated matching record is frequently specified by the user, which also indirectly reflects that the matching relationship is more important. At this time, the system will transfer the designated matching record to the channel automatic matching table for storage as a new automatic matching record, so that the next time the traffic to be collected is accurately collected, it can match the collection channel at a faster speed, and when extensive collection is performed, more and more useful traffic data can be collected, so as to further improve the efficiency of accurate collection and ensure data integrity and effectiveness.

[0031] Optionally, collecting the flow to be collected through the collection channel specifically includes: Perform sampling rate identification on each determined acquisition channel respectively to determine whether the acquisition channel has a specified sampling ratio; If yes, random sampling is performed in all acquired data packets according to the specified sampling ratio, and the sampling result is used as the final acquisition result of the corresponding acquisition channel; If not, all acquired data packets will be used as the final acquisition results of the corresponding acquisition channel.

[0032] In some possible implementations of the first aspect, the preprocessing of the collected traffic data specifically includes: In the process of receiving traffic data, a large time window is specified in time units according to the order in which the traffic data passes, and the large time window is divided into multiple small time windows in seconds. Each small time window dynamically stores all traffic data packets that pass within the respective time range. After each flow data is received, when the flow data passes through each time window in turn, the flow data content is hashed and compared with the content of all flow data packets dynamically stored in each time window. If the hash results are consistent, other auxiliary parameters are compared (other auxiliary parameters can be IP, port, payload, etc.). If all parameters are consistent, the flow data is determined to be duplicate flow data, and finally the duplicate flow data is removed according to the determination result; The BPF interface is used to receive the mirrored traffic data collected by different collection channels, and the received traffic data is initially filtered. This step avoids copying other packets from the operating system kernel to the user state, reduces the CPU burden of packet capture and the required buffer space, and implements preliminary filtering of data at the kernel layer. Compared with other subsequent filtering methods, this step has the highest filtering performance. Perform preliminary analysis on the traffic data after preliminary filtering, identify key information in the data packet header (such as protocol type, source address, destination address, port number, etc.), decode the data packet body according to the key information (such as protocol type), restore the original data, and then extract useful information from the data packet (such as packet size, packet sequence number, data window size, link status, etc.); The traffic data after preliminary analysis is filtered for the second time, and the traffic data after the second filtering is stored; the second filtering step can filter the sending content in the application, abnormal data in the protocol, illegal operations on key ports, etc. through pre-set filtering rules, and provides a flexible interface. In actual access, it can be expanded at any time through the interface, realizing the ability to be flexibly customized according to the business, and filtering is more thorough.

[0033] Based on the above solution, the collected traffic data is first subjected to preliminary kernel filtering and then to secondary precision filtering, which improves the overall filtering efficiency and effect, ensures the quality of the collected traffic data, and provides a good data quality foundation for subsequent data analysis.

[0034] In some possible implementations of the first aspect, parsing the preprocessed traffic data to obtain the service data specifically includes: Deserialize the preprocessed traffic data and restore the collected data packets; The control field of the first five bytes of the restored data header is intercepted and parsed to obtain the protocol version number and data length of the current data packet. The protocol version number can be used to facilitate iterative upgrades of the protocol and to perform protocol compatibility processing. The specific data length can solve the problem of data unpacking and sticking during network transmission. Serialize the data with the protocol version number and data length, and compress the serialized data. Specifically, the Protostuff tool can be used to serialize the data. Protostuff itself reduces the size of the serialized binary data by using variable-length encoding and ignoring default values. Then, by performing gzip compression on the serialized data, the bandwidth occupancy rate can be greatly reduced. The protocol is identified and analyzed on the compressed data, the data packets (such as IEC104, IEC102, etc.) are parsed, and the original traffic data messages are parsed into business-identifiable application data (i.e. business data), realizing the analysis capability of data transmission and communication between the monitoring system and the remote station.

[0035] Optionally, the parsed business data and original traffic data can be stored in a database to provide complete data input for subsequent analysis and processing operations based on business needs, making it convenient to trace back the data interaction and operating status of each system at a certain moment.

[0036] Optionally, the stored data can be displayed, providing users with a visual business data display page to facilitate real-time viewing of data content and data transmission status. It can also display the data content reported in real time by devices such as AGC, AVC, and RTU, and provide query of reported data for a specified time period, so as to achieve the purpose of analyzing the communication process between the power plant and the dispatching master station at any time.

[0037] Optionally, the step of calculating relevant business indicators based on the business data and issuing early warnings for relevant businesses based on the calculation results specifically includes: Classify the business data, and match the business data with various calculation models used to calculate corresponding business indicators according to the classification results; Identify the real-time performance of each calculation model, and configure the business data to the corresponding calculation model for calculation based on the identification results; perform real-time calculations on business indicators that are simple to calculate, rely less on data, and can be queried and obtained in real time (such as: data anomaly point statistics, reporting point statistics, etc.); perform offline calculations on business indicators that are complex to calculate, rely more on data, and cannot be obtained in real time (such as: reporting rate and commissioning rate calculated on a daily basis, etc.); According to the calculation results of each calculation model, an early warning is output for the relevant business data that meets the early warning conditions.

[0038] Optionally, the system can also standardize and normalize all alarm information, feed the alarm data back to the indicator report, and effectively mark it in the business report to facilitate users to search and view all alarm status, providing certain data support for analyzing and processing alarm events.

[0039] Optionally, the business indicator data obtained after calculation is summarized to form a data cube, and pre-classified according to multiple dimensions; in response to the output display command, the corresponding data value, analysis dimension, summary method and other contents are output to the user; in addition, the system can also dynamically generate a data display model structure according to the settings, and when the relevant data is updated, the display data is updated in time according to the data display model structure. The embodiment of the present application can dynamically construct a model of a visualization chart according to actual needs, realize flexible customization of data display reports, and more flexibly and intuitively display business indicator data.

[0040] Optionally, the business indicators may be: wind resource data assessment-reporting rate (number of reporting points / total number of points × 100%), wind power theoretical availability-accuracy ((1-(number of abnormal points / total number of theoretical and available power generation power uploaded points)) × 100%), AGC commissioning rate (AGC substation commissioning time / wind farm operation time × 100%), AVC commissioning rate (AVC substation commissioning time / wind farm operation time × 100%), AVC qualification rate (number of qualified execution points / number of orders issued by the power dispatching agency × 100%), etc.

[0041] Based on the above scheme, the embodiment of the present application uses a self-developed computing engine, introduces operator orchestration capabilities, abstracts basic operators for the computing process, and writes the calculation logic of the pass rate, accuracy rate, etc. in advance. Only the input data needs to be configured to calculate the results. For business indicators that need to be expanded or modified, most of the calculation processes can be assembled through operators and supported in a timely manner, reducing development and testing time.

[0042] In some possible implementations of the first aspect, the correlation analysis and traceability analysis of each business indicator and its data factor specifically includes: Using a timing calculation mechanism, the relevant statistics of each business indicator are calculated regularly according to a set period (such as every hour, every day, or every month), where the relevant statistics are one or more of the average value, maximum value, and minimum value; By using the event trigger mechanism, after receiving the event notification sent by the set event bus, the trend data of each business indicator is calculated, where the trend data is one or more of the year-on-year, month-on-month and change rate; Based on the calculation results of relevant statistics and trend data, we conduct trend analysis on the year-on-year and month-on-month changes of each business indicator on a daily, weekly, monthly and annual basis, calculate the data fluctuation pattern within a period of time, analyze the change trend of key indicators at different time points, and mark the data fluctuation range that greatly exceeds the previous range to identify potential problems or anomalies; According to the calculation formula of each business indicator, obtain the data factors that affect the calculation results of the indicator; By looking back at the alarm status of a business indicator at a certain moment, analyzing the correlation between different alarm information, we can obtain the internal connection and mutual influence between various business indicators and data factors. For example, the AGC operation rate indicator and the AVC operation rate indicator both depend on the station operation time. If the above two indicators alarm at the same time, it may be that the station operation time is abnormal. By taking the intersection of the data factors of the two indicators, it can be marked and output to the user, so that the user can judge the operation status of the station or equipment.

[0043] Optionally, the system can also calculate the weight of each data factor in the entire business indicator system based on the calculation results of each business indicator, the alarm status, and the degree of influence of each data factor; trace the source of each data factor to determine the collection channel corresponding to each data factor; calculate the weight of each collection channel in the entire business indicator system according to the weight of each data factor corresponding to each collection channel; sort the collection channels according to their weights, set the collection channels in the first P positions as the main channel, and the rest as backup channels; when using business data to calculate business indicators, give priority to using business data collected by the main channel for calculation, and only when the business data calculation of the main channel is abnormal, use the business data of the backup channel to calculate the business indicators.

[0044] Optionally, the system can also provide detailed detailed data by backtracking the business indicators that triggered the alarm conditions, build an alarm event chain based on timestamps and logical relationships, and display the generation, propagation and processing of alarm information. Relying on the preprocessing technology of alarm data, the system preprocesses the data and classifies and aggregates the data in multiple dimensions based on time and time type. It supports users to backtrack alarm information according to timeline, event type and other conditions, and view detailed processing process and results.

[0045] Optionally, the system can also provide a raw data query function for calculating business indicators based on business indicators. Through correlation analysis, the raw data input of the business indicator calculation formula is associated with related raw data to display the calculated raw data of the alarm indicator. For example, for an alarm with a reporting rate indicator, the calculation formula is: reporting rate = number of reporting points / total number of points × 100%. The data of the reporting points and the total number of points can be associated and displayed to provide support for fault location and cause analysis. The equipment operation status and working status can be further judged through the raw data.

[0046] Optionally, the system can also associate the original data packet of the traffic based on the original data of the calculation indicator. The original data packet is the most original data in the protocol layer. The original data of the calculation indicator is obtained by parsing the original data packet, providing the most original binary data, which can be traced back to the specific process of device communication, making it convenient for professionals to judge the operating status at that time.

[0047] Optionally, the system can also analyze the network connectivity status, changes in network transmission rate, whether there is abnormal traffic, etc. by statistically analyzing the changing trends of the original traffic data corresponding to each business indicator, so as to promptly issue early warnings for network quality issues, so as to facilitate users to intervene in time and avoid the spread of faults and business losses.

[0048] In a second aspect, an embodiment of the present application provides a device, which is a power grid flow data collection and analysis device, and the power grid flow data collection and analysis device includes a module for implementing the aforementioned power grid flow data collection and analysis method.

[0049] For example, Figure 2 As shown, the power grid flow data collection and analysis device includes: The channel determination module 101 is used to determine a collection channel matching the flow to be collected in response to a flow collection start command; Collection module 102: used for collecting the flow to be collected through the collection channel; Preprocessing module 103: used to preprocess the collected flow data; The service data acquisition module 104 is used to parse the pre-processed traffic data to obtain service data; Business indicator calculation module 105: used to calculate relevant business indicators based on business data: Early warning module 106: used to issue early warnings to related businesses based on calculation results; Analysis module 107: used to perform correlation analysis and traceability analysis on each business indicator and its data factors; Visualization module 108: used for visualizing and outputting the correlation analysis results and the traceability analysis results.

[0050] In some possible implementations of the second aspect, the channel determination module 101 includes: The first judgment unit is used to judge whether the start command contains the specified command of the acquisition channel; The first search unit is used for searching whether there is an automatic matching record between the flow to be collected and the specified collection channel in the channel automatic matching table when the start command contains a specified command of the collection channel; A first determining unit is used for determining the designated collection channel as a collection channel matching the flow to be collected according to the automatic matching record when there is an automatic matching record between the flow to be collected and the designated collection channel in the channel automatic matching table; The second search unit is used to search in the channel designated matching table whether there is a designated matching record between the flow to be collected and the designated collection channel when there is no automatic matching record between the flow to be collected and the designated collection channel in the channel automatic matching table; The second determining unit is used for determining the designated collection channel as the collection channel matching the flow to be collected according to the designated matching record when there is a designated matching record between the flow to be collected and the designated collection channel in the channel designated matching table; The first storage unit is used for storing the designated matching relationship between the flow to be collected and the designated collection channel into the channel designated matching table to form a new designated matching record when there is no designated matching record between the flow to be collected and the designated collection channel in the channel designated matching table; The third determining unit is used for determining the designated collection channel as the collection channel matching the flow to be collected according to the new designated matching record after storing the designated matching relationship between the flow to be collected and the designated collection channel in the channel designated matching table to form a new designated matching record; The third search unit is used for searching the automatic matching records of the flow to be collected and all the collection channels in the channel automatic matching table according to the identification characteristics of the flow to be collected when the start command does not contain the designated command of the collection channel; The fourth determining unit is used to determine all matched collection channels as collection channels that match the flow to be collected according to the found automatic matching records.

[0051] In some possible implementations of the second aspect, the channel determination module 101 further includes: Counting unit: used to count the number of matches of each designated matching record in the channel designated matching table; Transfer unit: used to transfer a specified matching record to the channel automatic matching table for storage as a new automatic matching record when the matching times of a specified matching record reaches a specified matching threshold N.

[0052] In some possible implementations of the second aspect, the preprocessing module 103 includes: Designation unit: used to designate a large time window in units of time in the process of receiving traffic data according to the order in which the traffic data passes; Division unit: used to divide the large time window into multiple small time windows in seconds, each small time window dynamically stores all traffic data packets passing through the respective time range; Hash calculation unit: for performing hash comparison between the content of each traffic data and the content of all traffic data packets dynamically stored in each time window after receiving each traffic data. If the hash results are consistent, continue to compare other auxiliary parameters. The second judgment unit is used to judge that the flow data is repeated flow data when all parameters are consistent; Deduplication unit: used to remove duplicate traffic data according to the determination result; Preliminary filtering unit: used to receive the mirrored traffic data collected by different collection channels using the BPF interface, and perform preliminary filtering on the received traffic data; The first parsing unit is used to perform preliminary parsing on the traffic data after preliminary filtering, identify key information in the data packet header, decode the data packet body according to the key information, restore the original data, and then extract useful information from the data packet; Secondary filtering unit: used to perform secondary filtering on the flow data after the initial analysis, and store the flow data after the secondary filtering.

[0053] In some possible implementations of the second aspect, the business data acquisition module 104 includes: Deserialization unit: used to deserialize the pre-processed traffic data and restore the collected data packets; The second parsing unit is used to intercept the control field of the first five bytes of the restored data header and parse out the protocol version number and data length of the current data packet; Serialization unit: used to serialize data with protocol version number and data length, and compress the serialized data; The third parsing unit is used to perform protocol identification and analysis on the compressed data, parse the data packets, and parse the original traffic data messages into application data that can be identified by the business.

[0054] In some possible implementations of the second aspect, the analysis module 107 includes: A first calculation unit is used to calculate relevant statistics of each business indicator according to a set periodic timing by using a timing calculation mechanism, wherein the relevant statistics are one or more of an average value, a maximum value and a minimum value; The second calculation unit is used to calculate the trend data of each business indicator by using the event trigger mechanism after receiving the event notification sent by the set event bus, wherein the trend data is one or more of year-on-year, month-on-month and change rate; The third calculation unit is used to perform trend analysis on each business indicator based on the calculation results of relevant statistics and trend data, calculate the data fluctuation law within a period of time, and analyze the change trend of key indicators at different time points; Data factor acquisition unit: used to obtain data factors that affect the calculation results of indicators according to the calculation formulas of various business indicators; Retrospective analysis unit: used to trace back the alarm status of a certain business indicator at a certain moment, analyze the correlation between different alarm information, and obtain the internal connection and mutual influence between various business indicators and data factors.

[0055] In a third aspect, an embodiment of the present application provides a power system, including: Traffic collection end: used to respond to the traffic collection start command, determine the collection channel that matches the traffic to be collected; collect the traffic to be collected through the collection channel, pre-process the collected traffic data, and serialize the pre-processed traffic data and report it to the data analysis end; Data analysis end: used to parse the pre-processed traffic data reported by the traffic collection end to obtain business data; calculate relevant business indicators based on the business data, and issue early warnings for related businesses based on the calculation results; perform correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

[0056] In some possible implementations of the third aspect, the flow collection end is specifically used to determine whether the start command contains a designated command for a collection channel; if the designated command is contained, first search in the channel automatic matching table whether there is an automatic matching record between the flow to be collected and the designated collection channel; if the automatic matching record exists, then determine the designated collection channel as the collection channel that matches the flow to be collected according to the automatic matching record; if the automatic matching record does not exist, then search in the channel designated matching table whether there is a designated matching record between the flow to be collected and the designated collection channel; if the designated matching record exists, then determine the designated collection channel as the collection channel that matches the flow to be collected according to the designated matching record. The collection channel that matches the collection flow is to be collected. If the specified matching record does not exist, the specified matching relationship between the flow to be collected and the specified collection channel is stored in the channel specified matching table to form a new specified matching record, and the specified collection channel is determined as the collection channel that matches the flow to be collected according to the new specified matching record; if it does not contain the specified command, the automatic matching record between the flow to be collected and all the collection channels is searched in the channel automatic matching table according to the identification characteristics of the flow to be collected, and then all the matching collection channels are determined as the collection channels that match the flow to be collected according to the found automatic matching records; the matching times of each specified matching record in the channel specified matching table are counted. Count; when the number of matches of a specified matching record reaches the specified matching threshold N, the specified matching record is transferred to the channel automatic matching table for storage as a new automatic matching record; in the process of receiving traffic data, a large time window is specified in time units according to the order in which the traffic data passes, and the large time window is divided into multiple small time windows in seconds. Each small time window dynamically stores all traffic data packets passing within the respective time range; after each traffic data is received, when the traffic data passes through each small time window in turn, the traffic data content is compared with all traffic data packets dynamically stored in each small time window. The content is hashed and compared. If the hash results are consistent, other auxiliary parameters are compared. If all parameters are consistent, the traffic data is determined to be duplicate traffic data, and finally the duplicate traffic data is removed according to the determination result. The mirrored traffic data collected by different collection channels is received by using the BPF interface, and the received traffic data is preliminarily filtered. The traffic data after the preliminary filtering is preliminarily parsed to identify the key information in the data packet header, and the data packet body is decoded according to the key information to restore the original data, and then useful information is extracted from the data packet. The traffic data after the preliminary parsing is secondary filtered, and the secondary filtered traffic data is stored.

[0057] In some possible implementations of the third aspect, the data analysis end is specifically used to deserialize the preprocessed traffic data and restore the collected data packets; intercept the control field of the first five bytes of the restored data header, and parse the protocol version number and data length of the current data packet; serialize the data with the protocol version number and data length, and compress the serialized data; perform protocol identification and analysis on the compressed data, parse the data packet, and parse the original traffic data message into application data that can be identified by the business; use the timing calculation mechanism to calculate the relevant statistics of each business indicator according to the set periodic timing, where the relevant statistics are one of the average value, maximum value and minimum value. one or more; using the event trigger mechanism, after receiving the event notification sent by the set event bus, calculate the trend data of each business indicator, where the trend data is one or more of the year-on-year, month-on-month and change rate; according to the calculation results of relevant statistics and trend data, perform trend analysis on each business indicator, calculate the data fluctuation law within a period of time, and analyze the change trend of key indicators at different time points; according to the calculation formula of each business indicator, obtain the data factors that affect the calculation results of the indicator; trace back the alarm status of a certain business indicator at a certain moment, analyze the correlation between different alarm information, and obtain the internal connection and mutual influence between various business indicators and various data factors.

[0058] In a fourth aspect, an embodiment of the present application provides a power grid flow data collection and analysis device, which can be any device capable of collecting and analyzing power grid flow data. The device can be various terminal devices, such as desktop computers, laptops, tablet computers, handheld devices, etc., and can be implemented specifically through software and / or hardware.

[0059] Exemplarily, the device comprises: Memory; Processor; and Computer programs; Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the power grid flow data collection and analysis method as described above.

[0060] In a fifth aspect, a computer-readable storage medium is provided in an embodiment of the present application. The computer-readable storage medium may be: ROM, RAM, a disk or an optical disk, etc.

[0061] Exemplarily, a computer program is stored on the computer-readable storage medium; the computer program is executed by a processor to implement the power grid flow data collection and analysis method as described above.

[0062] Compared with the prior art, this application has the following significant improvements: 1. Comprehensive and real-time traffic data monitoring: By receiving traffic data through multiple channels, comprehensive coverage and real-time monitoring of network traffic are achieved; data from different channels are captured simultaneously to ensure data integrity and diversity; real-time monitoring capabilities enable network managers to quickly grasp the dynamic changes of network traffic, and promptly discover potential network congestion, abnormal access and other problems, providing strong support for network optimization and fault prevention; 2. Accuracy and depth of business indicator analysis: Based on the traffic data received through multiple channels, the intelligent processing and analysis system can deeply explore the value behind the data and accurately calculate various business indicators, such as voltage stability, switch status, commissioning rate, qualified rate, etc. These indicators not only reflect the real-time status of business operation, but also reveal the inherent laws and potential problems of business operation through trend analysis, correlation analysis and other methods. In-depth analysis provides a scientific basis for business optimization, cost control and market expansion; 3. Rapidity and accuracy of fault alarm processing: The intelligent processing and analysis system has a built-in efficient fault detection and alarm mechanism. Through real-time analysis of traffic data, the system can quickly identify abnormal traffic patterns, determine whether they meet the alarm conditions, and immediately trigger alarm notifications. This rapid response mechanism greatly shortens the time interval from fault discovery to processing, reducing the impact of faults on business operations. At the same time, the accuracy of alarm information also ensures the pertinence and effectiveness of fault processing. 4. Comprehensiveness and traceability of backtesting of business operation status: During the business operation process, the intelligent processing and analysis system supports backtesting analysis of specific time periods and specific indicators; by tracing back historical data, the system can reproduce a certain state of business operation, helping managers to deeply understand the internal mechanism of business operation; in addition, comprehensive data records and traceability also provide strong support for business audits, compliance inspections, etc. 5. Provide solid support for business decisions: The comprehensive application of multi-channel receiving traffic data and intelligent processing and analysis methods provides comprehensive, accurate and timely data support for business decisions; through in-depth analysis of traffic data, business indicators and fault alarm information, managers can grasp the pulse of business operations, predict market trends, and formulate scientific and reasonable business strategies; these decisions not only help to enhance business competitiveness, but also maintain a leading position in the fierce market competition.

[0063] To sum up, this application has demonstrated significant effects and advantages in traffic data monitoring, business indicator analysis, fault alarm processing, business operation status backtesting, and providing support for business decision-making. With the continuous advancement of technology and the continuous expansion of application scenarios, this method will play an important role in more fields and promote digital transformation and intelligent upgrading.

[0064] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The scheme in the embodiments of the present application may be implemented in various computer languages, for example, C language, VHDL language, Verilog language, object-oriented programming language Java, and literal scripting language JavaScript, etc.

[0065] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, systems and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0066] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0068] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0069] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0070] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for collecting and analyzing power grid flow data, characterized in that: include: In response to a flow collection start command, determining a collection channel that matches the flow to be collected; The flow to be collected is collected through the collection channel, and the collected flow data is pre-processed; Analyze the pre-processed traffic data to obtain business data; Calculate relevant business indicators based on business data, and issue early warnings for related businesses based on the calculation results; Conduct correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

2. The method for collecting and analyzing power grid flow data according to claim 1, characterized in that: The step of determining a collection channel that matches the flow to be collected specifically includes: Determine whether the startup command contains the specified command of the acquisition channel; If a specified command is included, first search in the channel automatic matching table whether there is an automatic matching record between the flow to be collected and the specified collection channel; if the automatic matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the automatic matching record; if the automatic matching record does not exist, search in the channel specified matching table whether there is a specified matching record between the flow to be collected and the specified collection channel; if the specified matching record exists, determine the specified collection channel as the collection channel that matches the flow to be collected according to the specified matching record; if the specified matching record does not exist, store the specified matching relationship between the flow to be collected and the specified collection channel in the channel specified matching table to form a new specified matching record, and determine the specified collection channel as the collection channel that matches the flow to be collected according to the new specified matching record; If the specified command is not included, the automatic matching records of the flow to be collected and all collection channels are searched in the channel automatic matching table according to the identification characteristics of the flow to be collected, and then all the matched collection channels are determined as the collection channels that match the flow to be collected according to the found automatic matching records.

3. The method for collecting and analyzing power grid flow data according to claim 2 is characterized in that: The step of determining a collection channel that matches the flow to be collected specifically includes: Count the number of matches of each specified matching record in the channel specified matching table; When the matching times of a certain designated matching record reaches the designated matching threshold value N, the designated matching record is transferred to the channel automatic matching table and stored as a new automatic matching record.

4. The method for collecting and analyzing power grid flow data according to claim 1, characterized in that: The preprocessing of the collected flow data specifically includes: In the process of receiving traffic data, a large time window is specified in time units according to the order in which the traffic data passes, and the large time window is divided into multiple small time windows in seconds. Each small time window dynamically stores all traffic data packets that pass within the respective time range. After each flow data is received, when the flow data passes through each time window in turn, the flow data content is hashed and compared with the content of all flow data packets dynamically stored in each time window. If the hash results are consistent, other auxiliary parameters are compared. If all parameters are consistent, the flow data is determined to be duplicate flow data, and finally the duplicate flow data is removed according to the determination result; Use the BPF interface to receive the mirrored traffic data collected by different collection channels and perform preliminary filtering on the received traffic data; Perform preliminary analysis on the traffic data after preliminary filtering, identify the key information in the data packet header, decode the data packet body based on the key information, restore the original data, and then extract useful information from the data packet; The traffic data after the initial analysis is filtered for the second time, and the traffic data after the second filtering is stored.

5. The method for collecting and analyzing power grid flow data according to claim 1, characterized in that: The pre-processed traffic data is parsed to obtain service data, specifically including: Deserialize the preprocessed traffic data and restore the collected data packets; Intercept the first five bytes of the restored data header and parse out the protocol version number and data length of the current data packet; Serialize the data with the protocol version number and data length, and compress the serialized data; Perform protocol identification and analysis on the compressed data, parse the data packets, and parse the original traffic data messages into application data that can be identified by the business.

6. The method for collecting and analyzing power grid flow data according to claim 1, characterized in that: The correlation analysis and traceability analysis of each business indicator and its data factors specifically include: Using a timing calculation mechanism, the relevant statistics of each business indicator are calculated according to a set periodic timing, wherein the relevant statistics are one or more of an average value, a maximum value and a minimum value; By using the event trigger mechanism, after receiving the event notification sent by the set event bus, the trend data of each business indicator is calculated, where the trend data is one or more of the year-on-year, month-on-month and change rate; Based on the calculation results of relevant statistics and trend data, trend analysis is performed on each business indicator, the data fluctuation pattern within a period of time is calculated, and the change trend of key indicators at different time points is analyzed; According to the calculation formula of each business indicator, obtain the data factors that affect the calculation results of the indicator; Trace back the alarm status of a certain business indicator at a certain moment, analyze the correlation between different alarm information, and obtain the internal connection and mutual influence between various business indicators and data factors.

7. A power grid flow data collection and analysis device, characterized in that: It comprises a module for implementing the power grid flow data collection and analysis method described in any one of claims 1 to 6.

8. A power system, characterized in that: include: Traffic collection end: used to respond to the traffic collection start command and determine the collection channel that matches the traffic to be collected; The traffic to be collected is collected through the collection channel, and the collected traffic data is preprocessed. The preprocessed traffic data is serialized and reported to the data analysis end; Data analysis end: used to analyze the pre-processed traffic data reported by the traffic collection end to obtain business data; calculate relevant business indicators based on the business data, and issue early warnings for related businesses based on the calculation results; Conduct correlation analysis and traceability analysis on each business indicator and its data factors, and visualize the correlation analysis results and traceability analysis results.

9. A power grid flow data collection and analysis device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the power grid flow data collection and analysis method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the power grid flow data collection and analysis method as described in any one of claims 1 to 6.

Images