A method, system, and medium for fusion analysis and automatic handling of multi-source power anomaly data

Abstract

This application relates to a method, system, and medium for the fusion analysis and automatic handling of multi-source anomaly data in the power industry. The method includes: real-time collection of heterogeneous logs and status data from information network devices via a log collection agent; real-time storage of the aggregated multi-source data in an Elasticsearch cluster to construct a panoramic power data lake; fusion analysis and collaborative anomaly detection of the multi-source data based on Elasticsearch's built-in machine learning engine and combined with third-party analysis algorithms to identify hidden faults and complex threats that are difficult to detect from a single data source, generating anomaly results with confidence levels; and the real-time conversion of analysis results into handling instructions through an integrated alarm and automated handling system to drive the execution of contingency plans or notify maintenance personnel via a messaging platform. This application addresses the problems of high daily maintenance costs and low efficiency in the power grid.

Description

Technical Field

[0001] This invention relates to the field of intelligent safety operation and maintenance, and more specifically, to a method, system, and medium for the fusion analysis and automatic handling of multi-source power anomaly data. Background Technology

[0002] With the deepening of smart grid construction, the power system has formed a complex heterogeneous environment integrating multi-source information network systems, and various devices and systems continuously generate massive amounts of operational logs and status data. Currently, intelligent operation and maintenance (O&M) systems generally adopt distributed acquisition frameworks to acquire data and utilize traditional rule engines or single machine learning algorithms for anomaly detection, achieving certain results at the basic monitoring level. Power companies are committed to improving system security and operational reliability through data analysis technology, and intelligent O&M has gradually become a key direction for technological evolution in the industry.

[0003] However, existing technical solutions have significant limitations: data is collected independently by various business systems, creating information silos and lacking a unified fusion and analysis mechanism for multi-source data; threshold-based detection methods have poor adaptability and struggle to identify new hidden faults and complex threats; and the abnormal alarm and handling processes are disconnected, relying on manual judgment and resulting in low response efficiency. These problems lead to high false alarm rates and delayed responses in the existing operation and maintenance system, failing to meet the real-time and precise control requirements of new power systems. Therefore, there is an urgent need to build a closed-loop solution that integrates data fusion, intelligent analysis, and automated handling. Summary of the Invention

[0004] To address the aforementioned shortcomings or improvement needs of existing technologies, this paper provides a method, system, and medium for the fusion analysis and automatic handling of multi-source abnormal power data, thereby resolving issues such as high daily operation and maintenance costs and low efficiency in the power grid.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] In a first aspect, embodiments of this application provide a method for the fusion analysis and automatic handling of multi-source abnormal power data, comprising the following steps:

[0007] Step S1: Collect heterogeneous logs and status data from information network devices in real time using the log collection agent;

[0008] Step S2: Utilize Elasticsearch to store the aggregated multi-source data in real time on an Elasticsearch cluster, constructing a panoramic power data lake;

[0009] Step S3: Based on Elasticsearch's built-in machine learning engine and combined with third-party analysis algorithms, perform fusion analysis and collaborative anomaly detection on multi-source data to identify hidden faults and complex threats that are difficult to detect from a single data source, and generate anomaly results with confidence.

[0010] Step S4: Through the integrated alarm and automated handling system, the analysis results are transformed into handling instructions in real time, driving the execution of the contingency plan or notifying the operation and maintenance personnel through the messaging platform.

[0011] In step S1, a distributed log collection agent is deployed to extract operation logs, security logs, and status data from each monitoring point of the information network system in real time. After standardizing the format and unifying the fields, the data is transmitted to the data aggregation node through an encrypted channel.

[0012] In step S2, configure the Elasticsearch index template, write standardized data to the Elasticsearch cluster in real time, create sharded indexes according to data sources, and build a panoramic power data lake containing raw data, parsed fields, and metadata to support multi-dimensional retrieval and analysis.

[0013] In step S3, a machine learning anomaly detection task is configured in Elasticsearch, and a third-party data analysis algorithm library is called synchronously. Multi-source data is fused and calculated through feature cross-correlation and time series pattern analysis. An ensemble learning weighted voting mechanism is used to identify hidden faults and compound threats that are related across systems, and the anomaly detection results are output.

[0014] In step S3, configuring the machine learning anomaly detection task in Elasticsearch specifically involves configuring an Elasticsearch single-metric anomaly detection task, creating an independent detector for each key metric of each data source, automatically building a dynamic baseline model based on historical data, identifying anomalies where metric values ​​deviate from the normal fluctuation range through statistical learning, and outputting the anomaly scores and deviation analysis results of each data source in real time.

[0015] In step S3, the invocation of the third-party data analysis algorithm library specifically involves loading the Scikit-learn isolated forest and TensorFlow LSTM models into Python, extracting multi-dimensional time-series data from Elasticsearch, first using isolated forest to detect global outliers, then using LSTM network to build a sequence prediction model, identifying local anomaly patterns by comparing the deviation between predicted and actual values, and finally fusing the detection results of the two algorithms to achieve deep sequence anomaly analysis.

[0016] In step S4, based on the anomaly detection results, the preset alarm rules and handling plan library are matched, and the corresponding control commands are automatically triggered through the API gateway to execute the preset plan; at the same time, alarm details are pushed to the operation and maintenance personnel in real time through email and large screen message notification functions, and handling work orders are generated for closed-loop management.

[0017] Secondly, embodiments of this application provide a power multi-source anomaly data fusion analysis and automatic handling system, including a memory and a processor. The memory includes a program for a power multi-source anomaly data fusion analysis and automatic handling method. When the program for the power multi-source anomaly data fusion analysis and automatic handling method is executed by the processor, it implements the steps of the power multi-source anomaly data fusion analysis and automatic handling method described above.

[0018] Thirdly, embodiments of this application provide a computer-readable storage medium storing program code, which, when executed by a processor, implements the steps of the above-described method for the fusion analysis and automatic handling of multi-source abnormal power data.

[0019] Compared with the prior art, the present invention has significant beneficial effects.

[0020] Unified data acquisition framework: Real-time collection and aggregation of data from different heterogeneous sources in the power grid information network. This breaks down data silos in traditional operation and maintenance, providing a complete and consistent data foundation for achieving cross-system and cross-level global analysis.

[0021] Automated response: Automatically matches anomaly detection results with response plans, triggers the control system to execute instructions or notify personnel, realizes an automated response loop, shortens the cycle from fault occurrence to response, and improves operation and maintenance efficiency.

[0022] Platform-based centralized operation and maintenance: This invention integrates previously scattered tools and processes into a unified interface for management, building an integrated platform from data access, storage, analysis to processing, reducing the complexity of collaboration. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 The system architecture flowchart of the power multi-source anomaly data fusion analysis and automatic handling method provided in the embodiments of the present invention is shown.

[0025] Figure 2 This is a data acquisition and storage structure diagram in the power multi-source anomaly data fusion analysis and automatic handling method provided in the embodiments of the present invention.

[0026] Figure 3 This is a data fusion analysis structure diagram in the power multi-source anomaly data fusion analysis and automatic handling method provided in the embodiments of the present invention.

[0027] Figure 4 This is a diagram illustrating the automated processing structure in the power multi-source anomaly data fusion analysis and automated processing method provided in this embodiment of the invention.

[0028] Figure 5 This is a product interface diagram of the power multi-source anomaly data fusion analysis and automatic handling method provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. It should be noted that similar reference numerals and letters in the following drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0030] The terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0031] The terms “first,” “second,” etc., are used only to distinguish one entity or operation from another, and should not be construed as indicating or implying relative importance, nor as requiring or implying any such actual relationship or order between these entities or operations.

[0032] like Figure 1The illustration shows a method for the fusion analysis and automatic handling of multi-source power anomaly data provided by this invention. First, data from various heterogeneous sources in the information network is collected, completing unified data acquisition and preliminary processing. Then, the data is stored in real-time in an Elasticsearch cluster, constructing a comprehensive power data lake supporting high-performance retrieval, providing a data foundation for upper-level analysis. The multi-source data analysis capability is achieved through two collaborative aspects: firstly, real-time baseline monitoring and single-indicator anomaly detection are performed using Elasticsearch's built-in machine learning engine; secondly, deep mining and pattern recognition are conducted through a third-party analysis library. The two are then fused to obtain the detected anomaly results. The system's handling function matches and executes pre-planned actions based on the results, automatically performing handling actions or generating alarm work orders to notify maintenance personnel. This invention achieves a complete closed loop for multi-source power data from unified acquisition and fusion analysis to automatic handling, significantly improving the accuracy of power grid anomaly detection and the efficiency of response and handling.

[0033] This invention provides a method for the fusion analysis and automatic handling of multi-source power anomaly data, comprising the following steps:

[0034] Step S1: Collect heterogeneous logs and status data from information network devices in real time through the log collection agent.

[0035] Specifically, such as Figure 2 As shown, heterogeneous data sources in information networks mainly include firewall logs, information system logs, and security device logs. After collecting this raw data, the data acquisition agent performs basic data processing operations on the heterogeneous data. Format standardization involves converting logs from different vendors and with varying formats into a pre-defined standard structure, and normalizing the formats of key fields such as timestamps, IP addresses, and event levels. Field mapping maps fields with different names but the same semantics in the source data to standard field names, and supplements necessary metadata to ensure semantic consistency, thereby providing high-quality standardized input for subsequent fusion analysis and collaborative detection.

[0036] Step S2: Use Elasticsearch to store the aggregated multi-source data in real time on an Elasticsearch cluster to build a panoramic power data lake.

[0037] Specifically, such as Figure 2As shown, standardized data after processing is fed into a power industry panoramic data lake built on Elasticsearch. This data lake employs a hierarchical and domain-based indexing strategy, categorizing and storing data according to its business attributes and uses. Specifically, the detection data index centrally stores alarm data directly related to proactive security threats, such as intrusion attempts and vulnerability scan results; the network data index indexes network traffic, session logs, and access control information; and the status data index primarily stores performance metrics, resource utilization, and operational status logs of information systems and security devices. This business attribute-based index separation design not only optimizes data retrieval efficiency and storage costs but also provides a data foundation for subsequent anomaly monitoring and fusion analysis.

[0038] Step S3: Based on Elasticsearch's built-in machine learning engine and combined with third-party analysis algorithms, perform fusion analysis and collaborative anomaly detection on multi-source data to identify hidden faults and complex threats that are difficult to detect from a single data source, and generate abnormal results.

[0039] Specifically, such as Figure 3 As shown: The data fusion and analysis layer integrates Elasticsearch's built-in machine learning engine with third-party open-source and proprietary analysis algorithm libraries. These third-party libraries include Isolation Forest from Scikit-learn and LSTM networks from TensorFlow / Keras, performing anomaly detection and analysis on standardized data across multiple dimensions. The Elasticsearch engine is primarily responsible for real-time, single-indicator baseline modeling and deviation detection on massive amounts of time-series data, while the third-party analysis algorithms perform multivariate association analysis, deep sequence pattern learning, and nonlinear relationship mining. The analysis results from both sides are processed by a collaborative analysis control algorithm. This algorithm first aligns and correlates data and features, then performs weighted voting calculations on the detection results, ultimately outputting a composite anomaly detection result with confidence. The formula for calculating the confidence of the collaborative analysis control algorithm is as follows: Where S is the confidence score of the final fusion anomaly result, ranging from [0,1]. It is an Elasticsearch weight factor. It is the anomaly score of the i-th detector in Elasticsearch. It is the normalized weight of the i-th detector in Elasticsearch. , It is the anomaly score of the j-th detector in the third-party algorithm. It is the normalized weight of the j-th detector in the third-party algorithm. .

[0040] Step S4: Through the integrated alarm and automated handling system, the analysis results are transformed into handling instructions in real time, driving the execution of the contingency plan or notifying the operation and maintenance personnel through the messaging platform.

[0041] Specifically, such as Figure 4 As shown, abnormal results are matched against a pre-defined policy library based on their attributes (such as type, severity, and business scope). This library contains pre-stored contingency plans for various power system anomalies, defining specific response logic. Upon successful matching, the system automatically generates and issues handling instructions based on the contingency plan, including IP blocking, traffic interruption, and service shutdown—a series of security control operations—to achieve rapid fault isolation and recovery. Simultaneously, the abnormal results are quickly communicated to relevant operations and maintenance personnel via information system push notifications, telephone, or email, ensuring synchronized alarm information and timely follow-up monitoring. This forms a complete closed-loop handling process from intelligent analysis to automated execution and human collaboration.

[0042] like Figure 5 As shown in the embodiments of this application, an innovative method for the fusion analysis and automatic handling of multi-source power anomaly data is proposed. This method is based on Elasticsearch machine learning and combined with third-party collaborative analysis technology, which solves the "data silo" problem and realizes a closed loop of operation and maintenance from unified collection to automatic handling, significantly improving the overall efficiency and intelligence level of power grid anomaly management.

[0043] This application provides a power multi-source anomaly data fusion analysis and automatic handling system, including a memory and a processor. The memory includes a program for a power multi-source anomaly data fusion analysis and automatic handling method. When the program for the power multi-source anomaly data fusion analysis and automatic handling method is executed by the processor, it implements the steps of the power multi-source anomaly data fusion analysis and automatic handling method.

[0044] This application provides a computer-readable storage medium storing program code. When the program code is executed by a processor, it implements the steps of the above-described method for the fusion analysis and automatic handling of multi-source abnormal power data.

[0045] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0046] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0047] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0048] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0049] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0050] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0051] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0052] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for fusion analysis and automatic processing of multi-source power anomaly data, characterized in that, Includes the following steps: Step S1: Collect heterogeneous logs and status data from information network devices in real time using the log collection agent; Step S2: Utilize Elasticsearch to store the aggregated multi-source data in real time on an Elasticsearch cluster, constructing a panoramic power data lake; Step S3: Based on Elasticsearch's built-in machine learning engine and combined with third-party analysis algorithms, perform fusion analysis and collaborative anomaly detection on multi-source data to identify hidden faults and complex threats that are difficult to detect from a single data source, and generate anomaly results with confidence. Step S4: Through the integrated alarm and automated handling system, the analysis results are transformed into handling instructions in real time, driving the execution of the contingency plan or notifying the operation and maintenance personnel through the messaging platform.

2. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 1, characterized in that, In step S1, a distributed log collection agent is deployed to extract operation logs, security logs, and status data from each monitoring point of the information network system in real time. After standardizing the format and unifying the fields, the data is transmitted to the data aggregation node through an encrypted channel.

3. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 2, characterized in that, In step S2, configure the Elasticsearch index template, write standardized data to the Elasticsearch cluster in real time, create sharded indexes according to data sources, and build a panoramic power data lake containing raw data, parsed fields, and metadata to support multi-dimensional retrieval and analysis.

4. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 1, characterized in that, In step S3, a machine learning anomaly detection task is configured in Elasticsearch, and a third-party data analysis algorithm library is called synchronously. Multi-source data is fused and calculated through feature cross-correlation and time series pattern analysis. An ensemble learning weighted voting mechanism is used to identify hidden faults and compound threats that are related across systems, and the anomaly detection results are output.

5. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 4, characterized in that, In step S3, configuring the machine learning anomaly detection task in Elasticsearch specifically involves configuring an Elasticsearch single-metric anomaly detection task, creating an independent detector for each key metric of each data source, automatically building a dynamic baseline model based on historical data, identifying anomalies where metric values ​​deviate from the normal fluctuation range through statistical learning, and outputting the anomaly scores and deviation analysis results of each data source in real time.

6. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 4, characterized in that, In step S3, the invocation of the third-party data analysis algorithm library specifically involves loading the Scikit-learn isolated forest and TensorFlow LSTM models into Python, extracting multi-dimensional time-series data from Elasticsearch, first using isolated forest to detect global outliers, then using LSTM network to build a sequence prediction model, identifying local anomaly patterns by comparing the deviation between predicted and actual values, and finally fusing the detection results of the two algorithms to achieve deep sequence anomaly analysis.

7. The method for fusion analysis and automatic processing of multi-source power anomaly data according to claim 1, characterized in that, In step S4, based on the anomaly detection results, the preset alarm rules and handling plan library are matched, and the corresponding control commands are automatically triggered through the API gateway to execute the preset plan; at the same time, alarm details are pushed to the operation and maintenance personnel in real time through email and large screen message notification functions, and handling work orders are generated for closed-loop management.

8. A system for the fusion analysis and automatic handling of multi-source power anomaly data, characterized in that, The device includes a memory and a processor. The memory contains a program for a method for fusion analysis and automatic handling of multi-source power anomaly data. When the program for fusion analysis and automatic handling of multi-source power anomaly data is executed by the processor, it implements the steps of the method for fusion analysis and automatic handling of multi-source power anomaly data as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, which, when executed by a processor, implements the steps of the method for fusion analysis and automatic handling of multi-source power anomaly data as described in any one of claims 1 to 7.

Images