Distributed photovoltaic safety monitoring method and system based on log correlation analysis

Abstract

The invention discloses a distributed photovoltaic safety monitoring method and system based on log association analysis, and the method comprises the following steps: collecting continuous data in log data of all equipment in a distributed photovoltaic system, and carrying out the segmentation processing of the collected continuous data, and obtaining the discretization standards of all continuous data; performing primary processing on the collected log data of each device in the distributed photovoltaic system; and in combination with a discretization standard, performing association analysis on the primarily processed log data based on an improved association rule mining algorithm of storage structure recombination and parallel computing to obtain a safety monitoring result. Continuous data which cannot be processed by log correlation analysis are converted into discrete data, processing is simple, and internal characteristics of the data can be reserved; the improved association rule mining algorithm based on storage structure recombination and parallel computing is adopted, data analysis efficiency is improved, abnormal behaviors of equipment can be monitored more efficiently, and potential hostile attacks or abnormal equipment can be found and isolated in time.

Description

Technical Field

[0001] The present invention relates to the technical field of security monitoring and attack isolation of distributed photovoltaic systems, and more specifically, to a distributed photovoltaic security monitoring method and system based on log correlation analysis. Background Art

[0002] With the continuous advancement of Internet technology, we are currently facing an increasingly complex network environment. The means of network attacks are becoming more diverse and complex, which has brought unprecedented challenges to network and information security. In this context, it is particularly important to accurately and timely detect abnormal events in the network and equipment, which is the key to maintaining the security of the entire system.

[0003] Log analysis, as an important anomaly detection method, has become a core component of active network defense systems. However, traditional log analysis methods mostly rely on predefined rules, which makes them incapable of dealing with unknown attacks.

[0004] To overcome this limitation, researchers began to explore the possibility of combining log analysis with machine learning technology. This combination has indeed improved the accuracy of detection and the ability to identify complex attacks to a certain extent. But at the same time, we must also face a realistic problem: many machine learning algorithms will encounter high computational complexity when processing large-scale data sets.

[0005] In addition, with the widespread popularity of distributed photovoltaic systems, their security issues have gradually surfaced. These systems are usually distributed over a wide area, and the complexity of management and susceptibility to severe weather make their security monitoring particularly important. Unfortunately, the traditional centralized data processing model can no longer meet the needs of real-time monitoring due to the problems of high latency and slow response. At the same time, existing security technologies lack effective defenses against unknown threats and new attacks.

[0006] The existing technology uses log correlation technology to process distributed photovoltaic data, usually directly discretizing it according to a fixed length, but this method is prone to losing the intrinsic characteristics of the data, resulting in inaccurate data processing results; in the existing technology, professional technicians can also manually discretize the data, but this process is costly and time-consuming, which is not conducive to real-time data processing. Summary of the invention

[0007] In order to solve the deficiencies in the prior art, the present invention provides a distributed photovoltaic safety monitoring method based on log correlation analysis, which can solve the technical problems in the prior art that data processing is not accurate and timely enough, and potential attacks cannot be discovered in time.

[0008] The present invention adopts the following technical solution.

[0009] A distributed photovoltaic safety monitoring method based on log correlation analysis includes the following steps:

[0010] Step 1, collecting continuous data from the log data of each device in the distributed photovoltaic system, and obtaining various continuous data discretization standards by segmenting the collected continuous data;

[0011] Step 2, preliminarily processing the collected log data of each device in the distributed photovoltaic system;

[0012] Step 3: Combined with the discretization standard, an improved association rule mining algorithm based on storage structure reorganization and parallel computing is used to perform association analysis on the log data after preliminary processing to obtain security monitoring results.

[0013] Preferably, the step 1 specifically includes:

[0014] Step 1-1: Collect continuous data from the device log data to obtain data sequence A, remove duplicate values ​​from each data sequence A, and sort them in ascending order to obtain the sorted sequence A';

[0015] Step 1-2: Calculate the mean of the sorted sequence A' and the sum of squared deviations;

[0016] Step 1-3: Obtain a partitioning method for dividing sequence A' into m categories. For each partitioning method w, sequence A' is divided into m subsequences. Calculate the sum of the squares of the deviations of each subsequence SDCM, and calculate the variance goodness of fit GVF under the division method based on the sum of the squares of the deviations of each subsequence;

[0017] Step 1-4: Take the division method when GVF is the largest as the optimal division method when the number of clusters is m. Under this division, each subsequence is regarded as a category, and the first element in each subsequence is the lower bound E of this category of the data. bottom , the last element is the upper bound E of this category top .

[0018] Preferably, the step 2 specifically includes:

[0019] Step 2-1: Deploy edge monitoring nodes at key locations of the distributed photovoltaic system, and use the edge monitoring nodes to collect and process the raw log data of nearby devices in real time;

[0020] Step 2-2: The collected raw log data is preliminarily processed at the edge monitoring node to obtain preliminarily processed log data;

[0021] Step 2-3: Encrypt the processed log data and transmit it to the cloud computing platform through a secure communication link.

[0022] Preferably, the step 3 specifically includes:

[0023] Step 3-1, the cloud computing platform decompresses and parses the log data received from the edge monitoring node;

[0024] Step 3-2, discretize the continuous data in the log data according to the continuous data discretization standards constructed in step 1;

[0025] Step 3-3, using an improved association rule mining algorithm based on storage structure reorganization and parallel computing to mine association rules on log data;

[0026] Step 3-4, identifying potential regularities and abnormal patterns of log data of edge monitoring nodes through correlation analysis;

[0027] Step 3-5, when an abnormal pattern is identified, the cloud computing platform classifies and rates the anomaly to determine whether it is a potential malicious attack or equipment failure.

[0028] Preferably, the step 3-1 specifically includes:

[0029] Step 3-1-1: Decompress the log data D' using a decompression algorithm;

[0030] D u =Decompress(D′)

[0031] Among them, D u It is the decompressed data, and Decompress is the decompression function;

[0032] Step 3-1-2: Parse the data format and extract the various fields in the log entry, including timestamp, device ID, and sensor data. The data parsing is as follows:

[0033] D parsed =Parse(D u )

[0034] Among them, D parsed is the parsed data, D u For the decompressed data, Parse is the parsing function.

[0035] Preferably, the step 3-2 specifically includes:

[0036] For a data value data in a continuous data collected by an edge node, if it is in the upper bound of a cluster a of the data The Nether between:

[0037]

[0038] The data is considered to belong to category a, and the value of the data is recorded as a;

[0039] If the data data is outside the upper and lower bounds of each cluster of the data, then find the cluster closest to data. The distance between cluster a and data data is defined as:

[0040]

[0041] Calculate the distance between data and each cluster. The cluster with the smallest distance is recorded as nearest_class. Then the data data can be considered to belong to nearest_class, and the value of the data is recorded as nearest_class.

[0042] Preferably, the step 3-3 specifically includes:

[0043] Step 3-3-1: Convert the transaction database from the TransactionsId-Item structure to the Item-TransactionId structure, that is, convert the database from using transaction ID as the primary key to using item as the primary key;

[0044] Step 3-3-1: Build a transaction database, including using the unique identifier of the log as the transaction ID, and the various feature items of the log, including IP port, occurrence time, and exception type;

[0045] Step 3-3-2: Use the MapReduce framework to process data in parallel. The processing steps are as follows:

[0046] Map phase: data is sharded and distributed to multiple computing nodes;

[0047] Reduce phase: Summarize the calculation results of each node and generate a global frequent item set;

[0048] Step 3-3-3: Calculate the support of each item support(c):

[0049]

[0050] Among them, num_of_transactions refers to the number of transactions in the database with the item as the primary key, and num_of_all_transactions refers to the number of all different transactions;

[0051] Set parameter k = 1, and the items satisfying support(c ≥ min_support constitute the frequent k-item set Lk , currently k = 1, which is a frequent 1-item set L 1 , where min_support is the set minimum support threshold;

[0052] Step 3-3-4: Determine the current frequent k-item set L k Is the number of items in greater than k? If not, then the current frequent item set L k That is the final frequent itemset;

[0053] If so, update the parameter k, let k = k + 1, for the frequent item set L k-1 and frequent itemsets L 1 Connect by primary key, take the intersection of transaction ID sets, and get candidate item set C;

[0054] According to step 3-3-3, calculate the support of the candidate set support (C k ), and filter out frequent k-item sets L k , repeat steps 3-3-4 until no new frequent itemsets are generated.

[0055] Preferably, the steps 3-4 specifically include:

[0056] Step 3-4-1: For each frequent item set, generate all possible association rules and calculate their confidence. If the confidence is greater than or equal to the minimum confidence threshold defined by the user, output the rule;

[0057] Step 3-4-2, use association rules to identify abnormal patterns. The anomaly detection calculation formula is as follows:

[0058]

[0059] Where r is the association rule, min_confidence is the minimum confidence threshold, and anomaly(r)=1 indicates that an anomaly is detected.

[0060] The present invention also proposes a distributed photovoltaic safety monitoring system based on log correlation analysis, which is used to implement the distributed photovoltaic safety monitoring method based on log correlation analysis, including: a collection module, a segmentation processing module, a preliminary processing module, and a correlation analysis module;

[0061] The acquisition module is used to collect continuous data from the log data of each device in the distributed photovoltaic system;

[0062] The segmentation processing module is used to segment the collected continuous data to obtain various continuous data discretization standards;

[0063] The preliminary processing module is used to perform preliminary processing on the collected log data of each device in the distributed photovoltaic system;

[0064] The association analysis module combines the discretization standard and the improved association rule mining algorithm based on storage structure reorganization and parallel computing to perform association analysis on the log data after preliminary processing to obtain security monitoring results.

[0065] The present invention also provides a terminal, comprising a processor and a storage medium;

[0066] The storage medium is used to store instructions;

[0067] The processor is used to operate according to the instructions to execute the steps of the distributed photovoltaic safety monitoring method based on log correlation analysis.

[0068] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the distributed photovoltaic safety monitoring method based on log correlation analysis are implemented.

[0069] The beneficial effect of the present invention is that, compared with the prior art, the present invention proposes an innovative security monitoring method, which deeply integrates cloud-edge collaboration and log association analysis technology, performs real-time data processing through edge computing nodes, and ensures the immediacy and accuracy of data; while the cloud computing center is responsible for in-depth analysis of these data, thereby providing more comprehensive and in-depth security insights; the present invention combines the Jenks natural breakpoint algorithm with log association analysis, and converts continuous data that cannot be processed by log association analysis into discrete data through the Jenks algorithm, which is more scientific than direct discretization by fixed length, retains the intrinsic characteristics of the data, and does not require professional knowledge compared to manual discretization; in addition, the association rule analysis algorithm adopted by the present invention is an improved Apriori (association rule mining) algorithm based on storage structure reorganization and parallel computing, which greatly improves the data analysis efficiency compared to the traditional Apriori algorithm, and by mining the association rules in the log data, it can more efficiently monitor the abnormal behavior of the equipment, and timely discover and isolate potential malicious attacks or abnormal equipment; the present invention integrates advanced cloud-edge collaboration technology and log association analysis technology to build a more intelligent and efficient security monitoring system. This not only helps to improve the accuracy and efficiency of abnormal detection of distributed photovoltaic systems, but also provides strong technical support for the security protection of the entire system. The present invention has at least the following beneficial effects:

[0070] (1) Improve monitoring efficiency and accuracy: Through cloud-edge collaborative technology, data processing is faster, reducing the delay of traditional centralized data processing, thereby improving the real-time monitoring efficiency of distributed photovoltaic system status. Enhance the accuracy of anomaly detection. By deeply mining the potential connections in log data, it can more accurately identify abnormal behaviors and potential security threats of the system;

[0071] (2) Strengthening security protection capabilities: The present invention can quickly respond to and isolate potential network attacks or equipment failures, effectively preventing the spread of security accidents and protecting the overall security of distributed photovoltaic systems. Dynamically updated security strategies enhance the system's defense capabilities against unknown threats and new attacks, ensuring the long-term security and stability of the system.

[0072] (3) Reduce operation and maintenance costs and management complexity: Through the automated monitoring and early warning system, the frequency and cost of manual inspections are reduced, and the management process is simplified, making the operation and maintenance of distributed photovoltaic systems more efficient and convenient.

[0073] (4) Improve system reliability: Timely anomaly detection and rapid response mechanism help reduce system downtime and maintenance costs, thereby improving the reliability of the entire distributed photovoltaic system.

[0074] (5) Optimizing energy management and power quality: Based on the cloud-edge collaborative control strategy, this invention can also effectively solve the energy management and power quality problems caused by the connection of large-scale distributed photovoltaics to the power grid, such as voltage increase, transformer overload, etc., thereby ensuring the stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flow chart of a distributed photovoltaic safety monitoring method based on log correlation analysis in the present invention;

[0076] Figure 2 It is a flow chart of an improved association rule mining algorithm based on storage structure reorganization and parallel computing in the design and application embodiment of the present invention;

[0077] Figure 3 It is a comparison chart of the running time of different algorithms under different support levels in the design and application embodiment of the present invention;

[0078] Figure 4 It is a structural diagram of the distributed photovoltaic safety monitoring system based on log correlation analysis in the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0080] like Figure 1As shown, the present invention proposes a distributed photovoltaic security monitoring method based on log correlation analysis, relying on a system composed of edge monitoring nodes, cloud computing platform and security management center, using cloud-edge collaboration and log correlation analysis technology to achieve comprehensive, real-time and efficient security monitoring and protection of distributed photovoltaic systems. The edge monitoring nodes are deployed at key locations in the distributed photovoltaic system, responsible for real-time collection and processing of equipment log data; the cloud computing platform is responsible for correlation analysis and intelligent judgment of log data; the security management center is used to display monitoring results, manage security policies and implement isolation measures.

[0081] The distributed photovoltaic safety monitoring method based on log correlation analysis includes the following steps:

[0082] Step 1, construction of continuous data discretization standard;

[0083] The continuous data in the log data of each device in the distributed photovoltaic system are collected in advance to obtain the data sequence A, and the data sequence is segmented by the Jenks natural breakpoint algorithm to obtain the upper and lower bounds of each discrete category of each data item, which serve as the basis for the subsequent discretization of continuous data.

[0084] Specifically, step 1 includes the following steps:

[0085] Step 1-1: Remove duplicate values ​​from each data sequence A, and sort the data in the data sequence from small to large.

[0086] A′=sort(distinct(A))

[0087] Among them, distinct is a function to remove duplicate values ​​from a sequence, and sort is a function to sort a sequence from small to large.

[0088] Step 1-2: Calculate the mean of sequence A' and the sum of squared deviations SDAM.

[0089]

[0090] Where l represents the length of sequence A', and A'(s) is the sth element in sequence A'.

[0091] Step 1-3: Find all possible ways to divide sequence A' into m categories. For each possible division method w, A' is divided into m subsequences. Calculate the sum of the squared deviations of each subsequence SDCM:

[0092]

[0093] Calculate the variance goodness of fit GVF in this case.

[0094]

[0095] The partitioning method when GVF is the largest is taken as the optimal partitioning method when the number of clusters is m. Under this partitioning, each subsequence is a category, and the first element of the subsequence is the lower bound of the category of the data, denoted by E. bottom , the last element is the upper bound, denoted by E top .

[0096] Step 2, log data collection and processing: The edge monitoring node collects the log data of each device in the distributed photovoltaic system in real time, including device operation status, network communication records, etc. After preliminary processing, the collected log data is encrypted and transmitted to the cloud computing platform for further analysis.

[0097] Specifically, step 2 includes the following steps:

[0098] Step 2-1: Deploy edge monitoring nodes at key locations of the distributed photovoltaic system, and use the edge monitoring nodes to collect and process the raw log data of nearby devices in real time;

[0099] The key locations of the distributed photovoltaic system include photovoltaic panels, inverters, energy storage systems and other important equipment. Edge monitoring nodes are deployed near important equipment.

[0100] The edge node connects to the device through a dedicated sensor or interface to collect the device's original log data in real time. This data includes but is not limited to: device operating status, voltage and current data, environmental parameters, and network communication records;

[0101] The equipment operation status includes normal, fault, shutdown and other states; voltage (V) and current (I) data are used to analyze the power quality and performance of the equipment; environmental parameters include temperature (T) and humidity (H), which affect the working efficiency and life of the equipment; network communication records include sent and received data packets, communication timestamps, etc.

[0102] Step 2-2: The collected raw log data is preliminarily processed at the edge monitoring node to obtain processed log data to improve data quality and reduce transmission costs.

[0103] The original log data is preliminarily processed through data cleaning. The data cleaning process is as follows:

[0104] Assume that the original log data D = {d 1 ,d 2 ,...,d n}, the cleaned data set is D'.

[0105] For each data point d i, if it satisfies condition P (e.g., data range, format, invalid data filtering, etc.), then add it to D':

[0106] D'={d i ∣d i ∈D and d i Satisfy the condition P}

[0107] Step 2-3: Encrypt the preliminarily processed log data and transmit it to the cloud computing platform via a secure communication link.

[0108] Since data encryption and secure network connection are involved, VPN (Virtual Private Network) or dedicated line is used in the present invention to ensure the security of data transmission.

[0109] Regarding steps 2-3 above, perform the following operations:

[0110] Step 2-3-1: Before transmission, use an encryption algorithm to encrypt the data. Even if the encrypted data is intercepted, it is difficult to decrypt, thereby protecting the confidentiality of the data. The data encryption process is as follows:

[0111] Let E(D') represent the function for encrypting the cleaned data set D'. K represents the encryption key. Then the encryption process is:

[0112] C E =E(D′,K)

[0113] Step 2-3-2: Use VPN or dedicated line to establish a secure network connection to ensure that data is not tampered with or stolen during transmission. The decryption process (on the cloud computing platform) can be expressed as:

[0114] D′=F(C E ,K)

[0115] Where F is the decryption function that uses the same key K as used for encryption to restore the original data.

[0116] Step 3: The cloud computing platform performs association analysis on the received log data based on an improved association rule mining algorithm based on storage structure reorganization and parallel computing to obtain abnormal device behavior or potential malicious attacks.

[0117] The cloud computing platform uses log correlation analysis technology to perform correlation analysis on the received log data. By mining the potential rules and abnormal patterns in the log data, it can realize intelligent identification and early warning of abnormal device behavior.

[0118] like Figure 2As shown, the present invention discretizes continuous data using the Jenks natural breakpoint algorithm, and then uses the improved association rule mining (Apriori) algorithm based on storage structure reorganization and parallel computing proposed by the present invention to mine association rules for log data. The algorithm first discretizes continuous data items, which can make more full and effective use of data; then, by reorganizing the transaction database storage structure, the number of database scans is reduced, and the efficiency of association rule mining is improved.

[0119] The step 3 specifically includes:

[0120] Step 3-1, decompress and parse the received log data. After the cloud computing platform receives the log data from the edge monitoring node, it first decompresses and parses the data. This step ensures that the data can be further processed and analyzed.

[0121] Regarding step 3-1 above, perform the following operations:

[0122] Step 3-1-1: Decompress the log data D' using a decompression algorithm.

[0123] D u =Decompress(D′)

[0124] Among them, D u It is the decompressed data, and Decompress is the decompression function.

[0125] Step 3-1-2: Parse the data format and extract the various fields in the log entry (such as timestamp, device ID, sensor data, etc.).

[0126] D parsed =Parse(D u )

[0127] Among them, D u For the decompressed data, Parse is the parsing function.

[0128] Step 3-2: discretize the continuous data in the log data according to the continuous data discretization standards constructed in step 1.

[0129] Step 3-2 converts continuous data into discrete data that can be used for the next step of association rule analysis, and also makes the differences between different categories of data more obvious.

[0130] For a data value data in a continuous data collected by an edge node, if it is between the upper and lower bounds of a cluster a of the data, that is, The data is considered to belong to category a, and the value of the data is recorded as a.

[0131] If the data data is outside the upper and lower bounds of each cluster of the data, then find the cluster closest to data. The distance between cluster a and data data is defined as:

[0132]

[0133] Calculate the distance between data and each cluster. The cluster with the smallest distance is recorded as nearest_class. Then the data data can be considered to belong to nearest_class, and the value of the data is recorded as nearest_class.

[0134] Step 3-3, using the improved association rule mining (Apriori) algorithm based on storage structure reorganization and parallel computing proposed by the present invention (the algorithm flow is as follows Figure 2 As shown in Figure 1, the algorithm mines association rules for log data. The algorithm reduces the number of database scans by restructuring the transaction database storage structure and uses the MapReduce programming paradigm for parallel computing to improve the efficiency of association rule mining.

[0135] Regarding the above step 3-3, perform the following operations:

[0136] Step 3-3-1: Generate common exception types through simulation, and build a transaction database based on the log data set, including using the unique identifier of the log as the transaction ID, and the various features of the log, including IP port, occurrence time, exception type, etc. as the various items of the transaction;

[0137] The log data set can use the labeled log data set obtained through step 3-1 and step 3-2, or use a pre-prepared log data set.

[0138] Step 3-3-2: Convert the transaction database from the TransactionsId-Item structure to the Item-TransactionsId structure, that is, convert the database from using the transaction ID as the primary key to using the item as the primary key.

[0139] Step 3-3-3: Use the MapReduce framework to process data in parallel to reduce the number of database scans. The calculation steps are as follows:

[0140] 1) Map phase: data is sharded and distributed to multiple computing nodes.

[0141] 2) Reduce phase: Summarize the calculation results of each node and generate a global frequent itemset.

[0142] Step 3-3-4: Calculate the support of each item.

[0143]

[0144] Among them, num_of_transactions refers to the number of transactions in the database with the item as the primary key, and num_of_all_transactions refers to the number of all different transactions. Set k = 1, and the items satisfying support(c) ≥ min_support constitute a frequent k-item set L k , currently k = 1, which is a frequent 1-item set L 1 , where min_support is the set minimum support threshold.

[0145] Step 3-3-5: Determine the current frequent k-item set L k Is the number of items in greater than k? If not, then the current frequent item set L k That is the final frequent item set; if yes, then k = k + 1, for the frequent item set L k-1 and frequent itemsets L 1 Connect by primary key and take the intersection of transaction ID sets to get candidate item set C k , calculate the support of the candidate set according to step 3-3-3 (C k ), and filter out frequent k-item sets L k , repeat steps 3-3-4 until no new frequent itemsets are generated.

[0146] Step 3-4. Through correlation analysis, the cloud computing platform can identify abnormal patterns in log data, such as abnormal communication between devices, abnormal time series data, etc.

[0147] Regarding steps 3-4 above, perform the following operations:

[0148] Step 3-4-1: For each frequent item set (l∈L), generate all possible association rules and calculate their confidence. If the confidence is greater than or equal to min_confidence (the minimum confidence threshold defined by the user), output the rule.

[0149]

[0150] Among them, confidence(A→B) is the confidence of rule A→B, and support(A∪B) is the support of item set A∪B. Taking the abnormal pattern as B, calculate the item set A that satisfies confidence(A→B)≥min_confidence in this case, and get the association rule A→B that may lead to the abnormal pattern.

[0151] Step 3-4-2, use association rules to identify abnormal patterns, such as frequent but unreasonable communication between devices. According to the association rule A→B obtained in step 3-4-1, the collected log data is tested. If item set A exists in the log, it can be considered that the log has abnormal pattern B.

[0152] Step 3-5: When an abnormal pattern is identified, the cloud computing platform further uses machine learning and data analysis techniques to classify and rate the anomaly to determine whether it is a potential malicious attack or equipment failure. The random forest model is used to classify the anomaly.

[0153] class(B)=RandomForestClassifier(B)

[0154] Among them, class(B) is the classification result of abnormal pattern B, and RandomForestClassifier is the random forest learning model.

[0155] Step 4, abnormal monitoring and attack isolation: When the cloud computing platform identifies abnormal device behavior or potential malicious attacks, it will immediately trigger the early warning mechanism and notify the security management center. At the same time, according to the preset security policy, the system will automatically execute corresponding isolation measures, such as disconnecting abnormal devices from the network to prevent the spread of malicious attacks.

[0156] Step 4 specifically includes:

[0157] Step 4-1: When the cloud computing platform confirms that there is abnormal device behavior or potential malicious attack, the early warning mechanism is immediately triggered.

[0158] Step 4-2: The warning information will be displayed in a visual way through the security management center. The information formatting formula is as follows:

[0159] alert_info={type,timestamp,impact_range,details}

[0160] Among them, alert_info is the formatted information, type is the exception type, timestamp is the occurrence time, impact_range is the impact range, and details are detailed information.

[0161] In step 4-3, according to the preset security policy, the system will automatically execute the corresponding isolation measures. For example, for a device that is confirmed to be attacked, the system can automatically disconnect it from the network to prevent the further spread of malicious code.

[0162] Step 4-4: The security management center records all abnormal events and handling processes.

[0163] The record format event_log is as follows:

[0164] event_log={timestamp,device_id,anomaly_type,action_taken}

[0165] Among them, timestamp records the specific timestamp of the event, device_id uniquely identifies the device ID where the anomaly occurs, anomaly_type describes the type of anomaly detected, and action_taken describes the measures taken for the detected anomaly.

[0166] Step 5: Optimize log correlation analysis algorithms and isolation measures;

[0167] The cloud computing platform regularly collects and analyzes the latest security threat intelligence, dynamically updates security policies and early warning models, and continuously optimizes log correlation analysis algorithms and isolation measures based on historical data and user feedback to improve the accuracy and efficiency of security monitoring.

[0168] Step 5-1: The cloud computing platform regularly collects the latest security threat information from external security intelligence sources, such as newly discovered vulnerabilities, malware samples, attack patterns, etc.

[0169] Step 5-2: Based on the information collected in step 5-1, the cloud computing platform will dynamically update its security policies and early warning models to improve its defense capabilities against new attacks, as follows:

[0170] 1) For newly discovered security vulnerabilities, the system will generate or update corresponding firewall rules to prevent potential attacks.

[0171] 2) The access control list will also be dynamically adjusted based on the latest threat intelligence to ensure that only authorized access requests are passed, thereby effectively reducing security risks.

[0172] Step 5-3. The cloud computing platform will also continuously optimize its log correlation analysis algorithm and isolation measures based on historical data and user feedback.

[0173] Regarding step 5-3 above, perform the following operations:

[0174] Step 5-3-1: Analyze historical data and adjust the threshold formula of the association rule as follows, where new_min_support represents the new minimum support threshold and new_min_confidence represents the new minimum confidence threshold:

[0175]

[0176] Among them, detected_anomalies represents the number of anomalies detected in the historical data, and total_transactions represents the total number of transactions in the historical log data;

[0177]

[0178] Among them, successful_detections indicates the number of true anomalies that are successfully detected, and total_detcections indicates the total number of anomalies detected.

[0179] Step 5-3-2: Integrate user feedback and historical data, and adjust algorithm parameters, including adjusting the final minimum support threshold final_min_support and the final minimum confidence threshold final_min_confidence:

[0180]

[0181] Among them, feedback represents user feedback information, feedback(user_min_support) represents the minimum support threshold recommended by the user, and feedback(user_min_confidence) represents the minimum confidence threshold recommended by the user.

[0182] Step 5-4. Improve the system's isolation measures based on the optimized association rules and anomaly detection results. According to different types of anomalies, perform corresponding isolation measures as follows:

[0183] 1) For network security anomalies, such as detecting malicious traffic or unauthorized access attempts, the system will automatically disconnect the network connection of the relevant devices to prevent potential attackers from further infiltrating the system.

[0184] 2) When anomalies occur at the system or application level, such as performance degradation or service interruption, access to the abnormal part will be restricted to reduce the scope of the failure and keep other parts running normally as much as possible.

[0185] 3) If data anomalies or equipment failures are detected, the system will automatically restart the relevant equipment according to preset rules to restore it to normal working state.

[0186] In order to verify the beneficial effects of the present invention, the distributed photovoltaic safety monitoring method based on log correlation analysis proposed in the present invention is applied in practice.

[0187] The running time of the original Apriori algorithm and the improved Apriori algorithm proposed in the present invention are statistically analyzed. Figure 3 As shown, Figure 3The comparison of algorithm running time when the minimum support is selected from 0.02 to 0.3 is shown. It can be seen that the smaller the support, the longer it takes to mine frequent itemsets. Compared with the original Apriori algorithm, the Apriori algorithm improved by the present invention has a smaller change in support over time and is more stable. In practical applications, this means that the Apriori algorithm improved by the present invention can more effectively discover frequent itemsets and association rules in data sets, providing more accurate results for data mining and data analysis.

[0188] like Figure 4 As shown, the present invention also proposes a distributed photovoltaic safety monitoring system based on log correlation analysis, which is used to implement the above-mentioned distributed photovoltaic safety monitoring method based on log correlation analysis. The system includes: a collection module, a segmentation processing module, a preliminary processing module, and a correlation analysis module;

[0189] The acquisition module is used to collect continuous data from the log data of each device in the distributed photovoltaic system;

[0190] The segmentation processing module is used to segment the collected continuous data to obtain various continuous data discretization standards;

[0191] The preliminary processing module is used to perform preliminary processing on the collected log data of each device in the distributed photovoltaic system;

[0192] The association analysis module combines the discretization standard and the improved association rule mining algorithm based on storage structure reorganization and parallel computing to perform association analysis on the log data after preliminary processing to obtain security monitoring results.

[0193] The beneficial effects of the present invention are that, compared with the prior art, continuous data that cannot be processed by log association analysis is converted into discrete data, which is simple to process and can retain the inherent characteristics of the data; an improved association rule mining algorithm based on storage structure reorganization and parallel computing is adopted to improve data analysis efficiency, and the abnormal behavior of the device can be monitored more efficiently, and potential malicious attacks or abnormal devices can be discovered and isolated in time.

[0194] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0195] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0196] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0197] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A distributed photovoltaic safety monitoring method based on log correlation analysis, characterized in that: The steps include: Step 1, collecting continuous data from the log data of each device in the distributed photovoltaic system, and obtaining various continuous data discretization standards by segmenting the collected continuous data; Step 2, preliminarily processing the collected log data of each device in the distributed photovoltaic system; Step 3: Combined with the discretization standard, an improved association rule mining algorithm based on storage structure reorganization and parallel computing is used to perform association analysis on the log data after preliminary processing to obtain security monitoring results.

2. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 1 is characterized in that: The step 1 specifically includes: Step 1-1: Collect continuous data from the device log data to obtain data sequence A, remove duplicate values ​​from each data sequence A, and sort them in ascending order to obtain the sorted sequence A'; Step 1-2: Calculate the mean of the sorted sequence A' and the sum of squared deviations; Step 1-3: Obtain a partitioning method for dividing sequence A' into m categories. For each partitioning method w, sequence A' is divided into m subsequences. Calculate the sum of the squares of the deviations of each subsequence SDCM, and calculate the variance goodness of fit GVF under the division method based on the sum of the squares of the deviations of each subsequence; Step 1-4: Take the division method when GVF is the largest as the optimal division method when the number of clusters is m. Under this division, each subsequence is regarded as a category, and the first element in each subsequence is the lower bound E of this category of the data. bottom , the last element is the upper bound E of this category top .

3. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 1 is characterized in that: The step 2 specifically includes: Step 2-1: Deploy edge monitoring nodes at key locations of the distributed photovoltaic system, and use the edge monitoring nodes to collect and process the raw log data of nearby devices in real time; Step 2-2: The collected raw log data is preliminarily processed at the edge monitoring node to obtain preliminarily processed log data; Step 2-3: Encrypt the processed log data and transmit it to the cloud computing platform through a secure communication link.

4. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 1 is characterized in that: The step 3 specifically includes: Step 3-1, the cloud computing platform decompresses and parses the log data received from the edge monitoring node; Step 3-2, discretize the continuous data in the log data according to the continuous data discretization standards constructed in step 1; Step 3-3, using an improved association rule mining algorithm based on storage structure reorganization and parallel computing to mine association rules on log data; Step 3-4, identifying potential regularities and abnormal patterns of log data of edge monitoring nodes through correlation analysis; Step 3-5, when an abnormal pattern is identified, the cloud computing platform classifies and rates the anomaly to determine whether it is a potential malicious attack or equipment failure.

5. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 4 is characterized in that: The step 3-1 specifically includes: Step 3-1-1: Decompress the log data D' using a decompression algorithm; D u =Decompress(D) Among them, D u It is the decompressed data, and Decompress is the decompression function; Step 3-1-2: Parse the data format and extract the various fields in the log entry, including timestamp, device ID, and sensor data. The data parsing is as follows: D parsed =parse(D u ) Among them, D parsed is the parsed data, D u For the decompressed data, Parse is the parsing function.

6. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 4 is characterized in that: The step 3-2 specifically includes: For a data value data in a continuous data collected by an edge node, if it is in the upper bound of a cluster a of the data The Nether between: The data is considered to belong to category a, and the value of the data is recorded as a; If the data data is outside the upper and lower bounds of each cluster of the data, then find the cluster closest to data. The distance between cluster a and data data is defined as: Calculate the distance between data and each cluster. The cluster with the smallest distance is recorded as nearest_class. Then the data data can be considered to belong to nearest_class, and the value of the data is recorded as nearest_class.

7. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 4 is characterized in that: The step 3-3 specifically includes: Step 3-3-1: Convert the transaction database from the TransactionsId-Item structure to the Item-TransactionId structure, that is, convert the database from using transaction ID as the primary key to using item as the primary key; Step 3-3-1: Build a transaction database, including using the unique identifier of the log as the transaction ID, and the various feature items of the log, including IP port, occurrence time, and exception type; Step 3-3-2: Use the MapReduce framework to process data in parallel. The processing steps are as follows: Map phase: data is sharded and distributed to multiple computing nodes; Reduce phase: Summarize the calculation results of each node to generate a global frequent item set; Step 3-3-3: Calculate the support of each item support(c): Among them, num_of_transactions refers to the number of transactions in the database with the item as the primary key, and num_of_all_transactions refers to the number of all different transactions; Set parameter k = 1, and the items satisfying support(c ≥ min_support constitute the frequent k-item set L k , currently k = 1, that is, the frequent 1-item set L1, where min_support is the set minimum support threshold; Step 3-3-4: Determine the current frequent k-item set L k Is the number of items in greater than k? If not, then the current frequent item set L k That is the final frequent itemset; If so, update the parameter k, let k = k + 1, for the frequent item set L k-1 Connect with the frequent item set L1 by primary key, take the intersection of the transaction ID set, and get the candidate item set C k ; According to step 3-3-3, calculate the support of the candidate set support (C k ), and filter out frequent k-item sets L k , repeat steps 3-3-4 until no new frequent itemsets are generated.

8. The distributed photovoltaic safety monitoring method based on log correlation analysis according to claim 4 is characterized in that: The steps 3-4 specifically include: Step 3-4-1: For each frequent item set, generate all possible association rules and calculate their confidence. If the confidence is greater than or equal to the minimum confidence threshold defined by the user, output the rule; Step 3-4-2, use association rules to identify abnormal patterns. The anomaly detection calculation formula is as follows: Where r is the association rule, min_confidence is the minimum confidence threshold, and anomaly(r)=1 indicates that an anomaly is detected.

9. A distributed photovoltaic safety monitoring system based on log correlation analysis, used to implement the distributed photovoltaic safety monitoring method based on log correlation analysis as described in any one of claims 1 to 8, characterized in that: include: Acquisition module, segmentation processing module, preliminary processing module, and correlation analysis module; The acquisition module is used to collect continuous data from the log data of each device in the distributed photovoltaic system; The segmentation processing module is used to segment the collected continuous data to obtain various continuous data discretization standards; The preliminary processing module is used to perform preliminary processing on the collected log data of each device in the distributed photovoltaic system; The association analysis module combines the discretization standard and the improved association rule mining algorithm based on storage structure reorganization and parallel computing to perform association analysis on the log data after preliminary processing to obtain security monitoring results.

10. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images