Complex system health degree evaluation and anomaly detection method and device based on auto-encoder

Through the autoencoder model, abnormal detection and health assessment of the oil pipe data in the transmission network are solved, and the problem of failure to effectively deal with unknown abnormalities and high-dimensional data in the existing technology is achieved, and efficient and accurate abnormal detection and health assessment are achieved. Improve the safety and reliability of the oil pipes in the transmission grid.

CN119089356APending Publication Date: 2024-12-06NORTHWESTERN POLYTECHNICAL UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411218362.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-11-27
Filing Date
2024-09-02
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has a large number of domain knowledge requirements in the detection of oil pipe abnormalities in transmission grids, difficult to deal with unknown anomalies, and inapplicable to nonlinear and high-dimensional data, resulting in insufficient detection efficiency and accuracy.

Method used

Using an autoencoder-based method, by obtaining real-time data from the transmission network monitoring sensor, preprocessing and smoothing, using the Mean Squared Error loss function to train the compressed encoder and decoder, perform abnormal detection and health assessment, and use reconstruction errors Calculate health scores.

Benefits of technology

It realizes efficient and accurate abnormality detection and health assessment of oil pipe data in the transmission grid, improves the real-time and accuracy of detection, avoids the occurrence of faults and accidents, and enhances the reliability and safety of oil pipes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089356A_ABST
    Figure CN119089356A_ABST
Patent Text Reader

Abstract

The invention provides a complex system health degree evaluation and anomaly detection method and device based on an auto-encoder. The method comprises the following steps: acquiring real-time power transmission network data from a power transmission network monitoring sensor and preprocessing the power transmission network data; a Mean Squared Error loss function is used for model training, and a compression encoder and a decoder are obtained; performing anomaly detection and health degree evaluation on the power transmission network data; abnormal data obtained through abnormal detection are processed; and performing data translation according to time parameters and data volume set by a user, taking new data as a training set, and repeating the steps in sequence. According to the method, the AutoEncoder model is used for carrying out anomaly detection on the oil pipe data of the power transmission network, the evaluation index of the health degree of the system is constructed according to the reconstruction error on the basis, and the method has higher precision and accuracy and better generalization ability and robustness.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of big data analysis and machine learning, and in particular to a complex system health assessment and anomaly detection method and device based on an autoencoder. Background Art

[0002] With the rapid development of the global economy, the scale and complexity of the power system are constantly increasing, making the safe operation of the power system increasingly concerned. The transmission network is an important part of the power system, and its safe operation is crucial to ensuring the normal operation of the power system. The oil pipe is an important equipment in the transmission network, and its main function is to increase the heat dissipation area and improve the cooling effect. When the oil temperature of the transformer rises, it automatically circulates and cools through the cooling pipe, while providing insulation to ensure the normal operation of the transformer. However, the oil pipe equipment is also easily interfered by various external factors, resulting in abnormalities in the oil pipe equipment, which seriously affects the safe operation of the transmission network. Therefore, how to detect abnormalities in the oil pipe of the transmission network in a timely and accurate manner has become an important issue for the safe operation of the power system.

[0003] At present, there are two main methods for detecting abnormalities in oil pipelines in power transmission networks: one is a rule-based method and the other is a machine learning-based method. The rule-based method mainly defines a series of rules to determine whether the oil pipeline equipment is abnormal. For example, according to the pressure, temperature, flow and other indicators of the oil pipeline, the corresponding threshold is set, and the difference between the actual measurement value and the threshold is compared to determine whether the oil pipeline is abnormal. This method has the advantages of simple operation, easy implementation, and easy to be widely used.

[0004] Although rule-based methods can handle known abnormal situations well, they often cannot handle unknown abnormal situations. In addition, rule-based methods usually require a lot of domain knowledge and expert experience to build an effective rule system, which is not very realistic in practical applications. Rule-based methods have the following disadvantages:

[0005] 1. It requires a lot of domain knowledge and expert experience, and it is difficult to build a rule system.

[0006] 2. Unable to handle unknown abnormal situations.

[0007] 3. Due to the limitations of rule design, it is difficult to cover all abnormal situations.

[0008] 4. Not applicable to nonlinear problems.

[0009] 5. Not applicable to high-dimensional data.

[0010] In practical applications, transmission network pipeline data usually has the characteristics of high dimension, multivariate, nonlinear and dynamic changes, and rule-based methods have many problems in processing these data, so it is necessary to find a more efficient and accurate anomaly detection method. In power grid application scenarios, data anomaly detection usually needs to be efficient, real-time and accurate, because any fault or anomaly may cause accidents or losses in the power grid, so the accuracy and timeliness of anomaly detection are very important.

[0011] In practical applications, transmission network pipeline data usually contains a large amount of sensor data, such as temperature, pressure, flow, etc. These data are usually highly correlated and nonlinear, so they need to be processed using appropriate algorithms. At the same time, because transmission network pipeline data are usually high-dimensional and complex, traditional anomaly detection methods often have difficulty in coping with these challenges, so new data-driven methods are needed to solve the anomaly detection problem. Summary of the invention

[0012] In order to solve the above-mentioned deficiencies in the prior art, the present invention provides a complex system health assessment and anomaly detection method and device based on an autoencoder.

[0013] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0014] In a first aspect, the present invention provides a complex system health assessment and anomaly detection method and device based on an autoencoder, comprising:

[0015] Acquire real-time transmission network data from transmission network monitoring sensors and pre-process the transmission network data;

[0016] The Mean Squared Error (MSE) loss function is used for model training to obtain the compression encoder and decoder.

[0017] Anomaly detection and health assessment of transmission grid data;

[0018] Process the abnormal data detected by anomaly detection;

[0019] Data is shifted according to the time parameters and data volume set by the user, and the new data is used as the training set, and the above steps are repeated in sequence.

[0020] As a further technical solution of the present invention, the real-time transmission network data is obtained from the transmission network monitoring sensor and the transmission network data is preprocessed; specifically, it includes:

[0021] Obtain real-time transmission grid data from transmission grid monitoring sensors;

[0022] Preprocessing the acquired real-time transmission network data to obtain preprocessed data;

[0023] Fill in the acquired real-time power grid data;

[0024] Smoothing operation is performed on the filled real-time power grid data.

[0025] As a further technical solution of the present invention, the real-time transmission network data includes: temperature and pressure data of a single data; three-phase pressure and temperature of three-phase data; data and pressure difference data of multiple devices at a single site.

[0026] As a further technical solution of the present invention, the filling of the acquired real-time power grid data specifically includes:

[0027] The upward filling method is adopted to retain the time node of each record, and the missing data is filled with the data of the previous time node to ensure that the data of all time nodes can be fully utilized;

[0028] The time domain matching method is used to match the data of the nearest time node, retain the data points with the least time data difference, and then discard the time data points that cannot be filled in and matched.

[0029] As a further technical solution of the present invention, the smoothing operation is performed on the filled real-time power grid data; specifically comprising:

[0030] The average smoothing method of the selected window is used for the filled real-time power grid data. The system and the user select the appropriate setting window according to the frequency of power grid data acquisition and their own needs, and average all values ​​in each data item window to obtain the smoothed value range of the data item as the new value of the data item;

[0031] For a given window N and the value Value in the window = (Value1, Value " ,...,Value / ), there is a formula:

[0032]

[0033] The real-time power grid data after filling is smoothed by sliding average. For a time series of observation value sequence x1,x " ,…,x ' , where x ( is the i-th observation value, and the decay rate is defined as 0≤α≤1; the closer the decay rate is to 1, the slower the weight of the historical observation value decays; the closer the decay rate is to 0, the faster the weight of the historical observation value decays. There is a formula:

[0034] y ( =α·x ( +@1-αB·x()1 .

[0035] As a further technical solution of the present invention, it also includes: performing preliminary detection on data errors of the preprocessed data to determine whether the preprocessed data is abnormal.

[0036] As a further technical solution of the present invention, the preliminary detection of data errors of the preprocessed data to determine whether the preprocessed data is abnormal specifically includes:

[0037] Use the pressure difference data to perform preliminary detection of data errors, and use the three-phase pressure difference to preliminarily determine whether there are any abnormalities in the power grid data. Perform model training and abnormal data monitoring as needed.

[0038] As a further technical solution of the present invention, the model training is performed using the Mean Squared Error (MSE) loss function to obtain a compression encoder and decoder; specifically comprising:

[0039] Using the mean squared error (MSE) as the loss function, the training data is trained to obtain a compression encoder and decoder, and the training data is trained to obtain a compression encoder and decoder, which are used to compress the original data into a low-dimensional code Z = (z1, z " ,...,z / ), and then decode it back to the original data; specifically, let the input data be x, input it into the encoder to get its corresponding encoded representation z:

[0040] z=f(x)

[0041] Where f is the encoder function; input the code z into the decoder to obtain its corresponding reconstructed data xG:

[0042]

[0043] Where g is the decoder function; the training goal is to make the reconstructed data xG as close as possible to the original data x, that is, to minimize the mean square error between them:

[0044]

[0045] Where n represents the data dimension, x ( and Represent the values ​​of the original data and the reconstructed data in the i-th dimension respectively;

[0046] According to the above, let f 3'4 and f 534 Represent the encoder and decoder functions respectively, and the loss function during training is:

[0047]

[0048] As a further technical solution of the present invention, abnormality detection and health assessment are performed on the transmission network data; specifically, the following steps are included:

[0049] The new real-time data is input into the trained AutoEncoder model, and the encoder compresses the data into a low-dimensional feature vector Z = (z1, z " ,...,z / ), and then decode the feature vector into reconstructed data through the decoder By comparing the error between the original data and the reconstructed data, the reconstruction error E = (∈1,∈ " ,...,∈ ' ),in Used to determine whether the current data is abnormal; let μ and σ represent the mean and standard deviation of the reconstruction error in the training data respectively. The threshold T of anomaly detection can be determined according to the distribution of the training data. For example, μ+kσ can be selected as the threshold, where k is a constant determined according to the specific application scenario;

[0050] After obtaining the reconstruction error, the system maps the reconstruction error to a health evaluation range of 0 to 100 points. For each reconstruction error ∈ ( , the following formula can be used to calculate the health score:

[0051]

[0052] Among them, the minimum error and maximum error are determined according to the reconstruction error range in the training data. The minimum error is the minimum reconstruction error value in the training data, and the maximum error is the maximum reconstruction error value in the training data;

[0053] The health score ranges from 0 to 100. The closer the value is to 100, the healthier the data is, while the closer the value is to 0, the more abnormal or unhealthy the data is. The reconstruction error is mapped to the health score to evaluate the health of the system.

[0054] As a further technical solution of the present invention, the abnormal data detected by the abnormality is processed; specifically, it includes: data repair, alarm prompt and manual inspection of the abnormal data detected by the abnormality.

[0055] In a second aspect, the present invention further provides a complex system health assessment and anomaly detection method and device based on an autoencoder, comprising:

[0056] A data acquisition and processing unit, which obtains real-time transmission network data from transmission network monitoring sensors and pre-processes the transmission network data;

[0057] The model training unit uses the Mean Squared Error loss function to train the model and obtain the compression encoder and decoder;

[0058] Data detection and evaluation unit, which performs anomaly detection and health evaluation on transmission network data;

[0059] The abnormal data processing unit processes the abnormal data detected by the abnormality.

[0060] The present invention has the following beneficial effects compared with the prior art:

[0061] The present invention uses the AutoEncoder model to perform anomaly detection on the transmission network oil pipeline data, and on this basis, constructs the evaluation index of the system health by reconstructing the error, which has higher precision and accuracy than the traditional method. By compressing the data into a low-dimensional feature vector, the key features in the data can be better extracted, and the data can be judged whether it is abnormal by calculating the restoration error. At the same time, the present invention can realize real-time detection and early warning, effectively avoid the occurrence of failures and accidents of the transmission network oil pipeline, improve the reliability and safety of the transmission network oil pipeline, and can realize the effective compression and dimensionality reduction of large-scale complex data, while having good generalization ability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0063] Figure 1 A flow chart of a complex system health assessment and anomaly detection method and device based on an autoencoder proposed by the present invention;

[0064] Figure 2 A flow chart of a specific embodiment of the present invention;

[0065] Figure 3 This is a device diagram of a complex system health assessment and anomaly detection method and device based on an autoencoder proposed in the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. It should be understood that the specific embodiments described are only used to explain the present application and are not used to limit the present application.

[0067] See also Figure 1 and Figure 2 The present invention provides a complex system health assessment and anomaly detection method and device based on an autoencoder, comprising:

[0068] Step 101, acquiring real-time power transmission network data from a power transmission network monitoring sensor and preprocessing the power transmission network data;

[0069] Step 102, using the Mean Squared Error (MSE) loss function to perform model training to obtain a compression encoder and decoder;

[0070] Step 103, performing anomaly detection and health assessment on the transmission network data;

[0071] Step 104, processing the abnormal data detected by the abnormality;

[0072] Step 105, perform data translation according to the time parameters and data volume set by the user, use the new data as a training set, and repeat the above steps 102 to 104 in sequence.

[0073] In the embodiment of the present invention, the method of acquiring real-time power transmission network data from the power transmission network monitoring sensor and preprocessing the power transmission network data specifically includes:

[0074] Obtain real-time transmission grid data from transmission grid monitoring sensors;

[0075] Preprocessing the acquired real-time transmission network data to obtain preprocessed data;

[0076] Fill in the acquired real-time power grid data;

[0077] Smoothing operation is performed on the filled real-time power grid data.

[0078] Among them, the real-time transmission network data includes: temperature and pressure data of single data; three-phase pressure and temperature of three-phase data; data and pressure difference data of multiple devices at a single site.

[0079] Obtain real-time transmission network data from the transmission network monitoring sensor, including pressure P, temperature T and other parameters, preprocess and clean these data, and extract the features. Let X = (x1, x " ,...,x ' ) is the extracted transmission network data feature vector, where x ( Represents the value of the i-th parameter, and n is the number of parameters. These parameters can cover different types of data, such as temperature T and pressure P of single-phase data, three-phase pressure and temperature of three-phase data, data of multiple devices at a single site, pressure difference data ΔP, etc., as well as other parameter data that grow dynamically. Users can select specific parameters or data types as needed to customize parameter configuration. After selecting the given parameters, the system obtains a certain amount of normal data from historical data as training data based on the given parameters to prepare for subsequent AutoEncoder model training.

[0080] The data of the power grid often has certain time instability and data asynchrony between measurements of different phases. Therefore, a method of filling the data is needed so that the data can be used normally.

[0081] In this regard, this patent provides methods of upward filling and time domain matching. The upward filling method refers to retaining the time node of each record, and filling the missing data with the data of the previous time node to ensure that the data of all time nodes can be fully utilized. The time domain matching method refers to matching the data of the nearest time node, retaining the data points with the least time data difference, and then discarding the time data points that cannot be filled and matched.

[0082] In the data preprocessing stage, the transmission network data is smoothed in view of its low acquisition frequency and large fluctuation range. Users can choose a variety of smoothing methods suitable for the data characteristics, such as average smoothing of the selected window and sliding average smoothing method.

[0083] For the average smoothing method of the selected window, the system and the user select a suitable setting window according to the frequency of power grid data acquisition and their own needs, average all values ​​in each data item window, and obtain the smoothed value range of the data item as the new value of the data item; for a given window N and the value in the window Value=(Value1,Value " ,...,Value / ), there is a formula:

[0084]

[0085] For the sliding average smoothing method, for a time series of observations x1,x " ,…,x' , where x ( is the i-th observation value, and the decay rate is defined as 0≤α≤1. The closer the decay rate is to 1, the slower the weight of the historical observation value decays; the closer the decay rate is to 0, the faster the weight of the historical observation value decays. There is a formula:

[0086] y ( =α·x ( +@1-αB·x ()1 .

[0087] In an embodiment of the present invention, before training using an autoencoder, pressure difference data is used to perform preliminary detection of data errors, and a preliminary judgment is made on whether there are abnormalities in the power grid data through the three-phase pressure difference, and model training and abnormal data monitoring are performed as needed.

[0088] For the missing data that may exist in the data, the upward filling method is adopted. If the corresponding data is not obtained at the current moment, the data of the previous moment will be filled into the current moment, ensuring the integrity and accuracy of all data.

[0089] In the embodiment of the present invention, the Mean Squared Error (MSE) loss function is used for model training to obtain a compression encoder and decoder; specifically, the following steps are included:

[0090] Using the mean squared error (MSE) as the loss function, the training data is trained to obtain a compression encoder and decoder, and the training data is trained to obtain a compression encoder and decoder, which are used to compress the original data into a low-dimensional code Z = (z1, z " ,...,z / ), and then decode it back to the original data; specifically, let the input data be x, input it into the encoder to get its corresponding encoded representation z:

[0091] z=f(x)

[0092] Where f is the encoder function; input the code z into the decoder to obtain its corresponding reconstructed data xG:

[0093]

[0094] Where g is the decoder function; the training goal is to make the reconstructed data xG as close as possible to the original data x, that is, to minimize the mean square error between them:

[0095]

[0096] Where n represents the data dimension, x ( and Represent the values ​​of the original data and the reconstructed data in the i-th dimension respectively;

[0097] According to the above, let f 3'4 and f 534 Represent the encoder and decoder functions respectively, and the loss function during training is:

[0098]

[0099] The Mean Squared Error (MSE) is used as the loss function to train the training data and obtain a compression encoder and decoder, which are used to compress the original data into a low-dimensional encoding and then decode it back to the original data. During the training process, cross-validation and regularization techniques are used to improve the robustness and generalization performance of the model.

[0100] In the embodiment of the present invention, abnormality detection and health assessment are performed on the power transmission network data; specifically, the following steps are performed:

[0101] The new real-time data is input into the trained AutoEncoder model, and the encoder compresses the data into a low-dimensional feature vector Z = (z1, z " ,...,z / ), and then decode the feature vector into reconstructed data through the decoder By comparing the error between the original data and the reconstructed data, the reconstruction error E = (∈1,∈ " ,...,∈ ' ),in Used to determine whether the current data is abnormal; let μ and σ represent the mean and standard deviation of the reconstruction error in the training data respectively. The threshold T of anomaly detection can be determined according to the distribution of the training data. For example, μ+kσ can be selected as the threshold, where k is a constant determined according to the specific application scenario;

[0102] After obtaining the reconstruction error, the system maps the reconstruction error to a health evaluation range of 0 to 100 points. For each reconstruction error ∈ ( , the following formula can be used to calculate the health score:

[0103]

[0104] Among them, the minimum error and maximum error are determined according to the reconstruction error range in the training data. The minimum error is the minimum reconstruction error value in the training data, and the maximum error is the maximum reconstruction error value in the training data;

[0105] The health score ranges from 0 to 100. The closer the value is to 100, the healthier the data is, while the closer the value is to 0, the more abnormal or unhealthy the data is. The reconstruction error is mapped to the health score to evaluate the health of the system.

[0106] The present invention inputs new real-time data into the trained AutoEncoder model, compresses the data into a low-dimensional feature vector through the encoder, and then decodes the feature vector into reconstructed data through the decoder. By comparing the error between the original data and the reconstructed data, the reconstruction error of the data is calculated, and the health of the system is evaluated.

[0107] According to the set threshold, the reconstruction error score is averaged and mapped to 0 to 100 points for quantitative health modeling, and the health of the system is used to determine whether the data is abnormal. When the health index is lower than the set threshold, the data is considered abnormal.

[0108] In the embodiment of the present invention, the abnormal data detected by the anomaly is processed; specifically, the abnormal data detected by the anomaly is repaired, an alarm is prompted, and a manual inspection is performed. When abnormal data is detected, the system will automatically trigger the abnormality handling program, and take corresponding measures to handle the abnormality according to the pre-set rules and strategies.

[0109] Shift the above model. Shift the data according to the time parameters and data volume set by the user, use the new data as the training set, repeat steps (2), (3), (4), and complete the iterative update of the model.

[0110] The present invention uses the AutoEncoder model to perform anomaly detection on the transmission network oil pipeline data, and on this basis, constructs the evaluation index of the system health by reconstructing the error, which has higher precision and accuracy than the traditional method. By compressing the data into a low-dimensional feature vector, the key features in the data can be better extracted, and the data can be judged whether it is abnormal by calculating the restoration error. At the same time, the present invention can realize real-time detection and early warning, effectively avoid the occurrence of failures and accidents of the transmission network oil pipeline, improve the reliability and safety of the transmission network oil pipeline, and can realize the effective compression and dimensionality reduction of large-scale complex data, while having good generalization ability and robustness.

[0111] See also Figure 3 Another specific embodiment of the present invention is a complex system health assessment and anomaly detection method and device based on an autoencoder, which adopts the following technical solution:

[0112] A complex system health assessment and anomaly detection method and device based on an autoencoder, comprising:

[0113] The data acquisition and processing unit 201 acquires real-time power transmission network data from power transmission network monitoring sensors and pre-processes the power transmission network data;

[0114] A model training unit 202 performs model training using a Mean Squared Error loss function to obtain a compression encoder and a decoder;

[0115] The data detection and evaluation unit 203 performs abnormality detection and health evaluation on the transmission network data;

[0116] The abnormal data processing unit 204 processes the abnormal data detected by the abnormality.

[0117] The various variations and specific examples of the methods in the aforementioned embodiments are also applicable to a complex system health assessment and anomaly detection method and device based on an autoencoder in the present embodiment. Through the aforementioned detailed description of the point debugging method for background monitoring of the substation, technicians in this field can clearly know a complex system health assessment and anomaly detection method and device based on an autoencoder in the present embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0118] The present invention is described in detail above, but the present invention is not limited to the above-mentioned embodiments. Various changes can be made within the knowledge of ordinary technicians in the field without departing from the purpose of the present invention. Many other changes and modifications can be made without departing from the concept and scope of the present invention. It should be understood that the present invention is not limited to specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. A complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: include: Acquire real-time transmission network data from transmission network monitoring sensors and pre-process the transmission network data; Use the Mean Squared Error loss function to train the model and obtain the compression encoder and decoder; Anomaly detection and health assessment of transmission grid data; Process the abnormal data detected by anomaly detection; Data is shifted according to the time parameters and data volume set by the user, and the new data is used as the training set, and the above steps are repeated in sequence.

2. According to claim 1, a complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: The method of acquiring real-time transmission network data from the transmission network monitoring sensor and preprocessing the transmission network data specifically includes: Obtain real-time transmission grid data from transmission grid monitoring sensors; Preprocessing the acquired real-time transmission network data to obtain preprocessed data; Fill in the acquired real-time power grid data; Perform smoothing operations on the filled real-time power grid data.

3. According to claim 1, a complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: The real-time transmission network data includes: temperature and pressure data of single data; three-phase pressure and temperature of three-phase data; data and pressure difference data of multiple devices at a single site.

4. According to claim 2, a complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: The filling of the acquired real-time power grid data specifically includes: The upward filling method is adopted to retain the time node of each record, and the missing data is filled with the data of the previous time node to ensure that the data of all time nodes can be fully utilized; The time domain matching method is used to match the data of the nearest time node, retain the data points with the least time data difference, and then discard the time data points that cannot be filled in and matched.

5. According to claim 2, a complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: The smoothing operation of the filled real-time power grid data specifically includes: The average smoothing method of the selected window is used for the filled real-time power grid data. The system and the user select the appropriate setting window according to the power grid data acquisition frequency and their own needs, and average all the values ​​in each data item window to obtain the smoothed value range of the data item as the new value of the data item; For a given window N and the value Value in the window = (Value1, Value2, ..., Value m ), there is a formula: The real-time power grid data after filling is smoothed by sliding average. For a time series of observation value sequence x1, x2, ..., x n , where x i is the i-th observation value, and the decay rate is defined as 0≤α≤1; the closer the decay rate is to 1, the slower the weight of the historical observation value decays; the closer the decay rate is to 0, the faster the weight of the historical observation value decays. The specific formula is: y i =α·x i +(1-a)·x i-1 。 6. The method and device for complex system health assessment and anomaly detection based on autoencoder according to claim 1, characterized in that: Also includes: Perform preliminary detection on the data error of the preprocessed data to determine whether the preprocessed data is abnormal, including: Use the pressure difference data to perform preliminary detection of data errors, and use the three-phase pressure difference to preliminarily determine whether there are any abnormalities in the power grid data. Perform model training and abnormal data monitoring as needed.

7. The method and device for complex system health assessment and anomaly detection based on autoencoder according to claim 1, characterized in that: The method uses the Mean Squared Error loss function to perform model training to obtain a compression encoder and decoder; specifically includes: Using Mean Squared Error as the loss function, the training data is trained to obtain a compression encoder and decoder, and the training data is trained to obtain a compression encoder and decoder, which are used to compress the original data into a low-dimensional code Z = (z1, z2, ..., z m ), and then decode it back to the original data; specifically, let the input data be x, input it into the encoder to get its corresponding encoded representation z: z=f(x) Where f is the encoder function; input the code z into the decoder to obtain its corresponding reconstructed data Where g is the decoder function; the training goal is to reconstruct the data As close as possible to the original data x, that is, to minimize the mean square error between them: Where n represents the data dimension, x i and Respectively represent the values ​​of the original data and the reconstructed data in the Mth dimension; According to the above, let f enc and f enc Represent the encoder and decoder functions respectively, and the loss function during training is:

8. The method and device for complex system health assessment and anomaly detection based on autoencoder according to claim 1, characterized in that: Anomaly detection and health assessment of transmission network data; specifically including: The new real-time data is input into the trained AutoEncoder model, and the encoder compresses the data into a low-dimensional feature vector Z = (z1, z2, ..., z m ), and then decode the feature vector into reconstructed data through the decoder By comparing the error between the original data and the reconstructed data, the reconstruction error E = (∈1,∈2,...,∈ n ),in Used to determine whether the current data is abnormal; let μ and σ represent the mean and standard deviation of the reconstruction error in the training data respectively, and the threshold T of anomaly detection can be determined according to the distribution of training data; After obtaining the reconstruction error, the system maps the reconstruction error to a health evaluation range of 0 to 100 points. For each reconstruction error ∈ i , the health score can be calculated using the following formula: Among them, the minimum error is the minimum reconstruction error value in the training data, and the maximum error is the maximum reconstruction error value in the training data; The health score ranges from 0 to 100. The closer the value is to 100, the healthier the data is, while the closer the value is to 0, the more abnormal or unhealthy the data is. The reconstruction error is mapped to the health score to evaluate the health of the system.

9. The method and device for complex system health assessment and anomaly detection based on autoencoder according to claim 1, characterized in that: The abnormal data detected by the abnormality is processed; specifically includes: data repair, alarm prompt and manual inspection of the abnormal data detected by the abnormality.

10. A complex system health assessment and anomaly detection method and device based on autoencoder, characterized in that: A complex system health assessment and anomaly detection method and device based on an autoencoder as described in any one of claims 1 to 9 is adopted, comprising: A data acquisition and processing unit, which obtains real-time transmission network data from transmission network monitoring sensors and pre-processes the transmission network data; The model training unit uses the Mean Squared Error loss function to train the model and obtain the compression encoder and decoder; Data detection and evaluation unit, which performs anomaly detection and health evaluation on transmission network data; The abnormal data processing unit processes the abnormal data detected by the abnormality.

Citation Information

Cited By

  • Device health degree dynamic evaluation method and system based on auto-encoder

    CN120873768A

  • Method and system for dynamic assessment of device health based on autoencoder

    CN120873768B