MBO-LSTM-AE-based large-scale carbon emission monitoring enterprise abnormal value detection method and system

Abstract

The invention relates to a large carbon emission monitoring enterprise abnormal value detection method and system based on MBO-LSTM-AE. The method comprises the following steps: step 1, collecting energy consumption data of a large carbon emission detection enterprise; step 2, performing data format conversion on the energy consumption data of the large-scale carbon emission detection enterprise acquired in the step 1; step 3, carrying out vacancy value interpolation and normalization processing on the energy consumption data of the large-scale carbon emission detection enterprise in the DataFrame format; and step 4, establishing a king butterfly optimization algorithm to optimize a long and short term memory automatic encoder model, and identifying an abnormal value in the energy consumption data of the large-scale carbon emission monitoring enterprise in the DataFrame format in the step 3. According to the invention, abnormal value detection can be efficiently and reliably realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system optimization and dispatching, and relates to a method and system for detecting abnormal values ​​of energy consumption data of a large carbon emission monitoring enterprise, in particular to a method and system for detecting abnormal values ​​of a large carbon emission monitoring enterprise based on MBO-LSTM-AE. Background Art

[0002] In today's society, global climate change and environmental issues are increasingly attracting people's attention, driving the demand for carbon emission monitoring and energy management. Large carbon emission monitoring companies play a key role in monitoring and managing energy consumption in production activities to reduce negative impacts on the environment. With the rapid development of data science and artificial intelligence, a large amount of energy consumption data is collected and recorded, providing large carbon emission monitoring companies with the opportunity to gain in-depth understanding of their energy usage.

[0003] However, large carbon emission monitoring companies often face a severe challenge when dealing with large and complex energy consumption data, namely how to effectively detect and process outliers in the data. Outliers may be caused by equipment failure, operational errors or other unknown factors. If not discovered and handled in time, it may lead to wrong energy management decisions.

[0004] The following is an analysis of the causes of abnormal energy consumption data in large carbon emission monitoring companies. First, equipment failure may be a common cause of abnormal energy consumption data. If the equipment fails during operation, it may cause abnormalities in the collection and recording of energy consumption data. Secondly, improper human operation and management are also an important cause of abnormal energy consumption data. During the production and use of the enterprise, if the operator does not operate in accordance with the prescribed procedures and requirements, it may cause abnormal energy consumption data. Finally, external environmental factors may also cause abnormal energy consumption data. For example, climate change and changes in weather conditions may have an impact on energy consumption data.

[0005] At present, some companies are still using traditional statistical analysis methods to detect outliers, such as threshold judgment based on the mean. These methods may not be able to effectively identify all outliers in the face of complex and changeable carbon emission data. With the development of technologies such as big data and artificial intelligence, although there are some outlier detection methods based on machine learning, the application of these methods in the field of carbon emission monitoring is not extensive and in-depth enough, resulting in some outliers not being discovered and processed in a timely manner.

[0006] After searching, no public documents of the prior art that are identical or similar to the present invention were found. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and propose an outlier detection method and system for large-scale carbon emission monitoring enterprises based on MBO-LSTM-AE, which can efficiently and reliably realize the detection of outliers.

[0008] The present invention solves the practical problem by adopting the following technical solutions:

[0009] A method for detecting outliers in large carbon emission monitoring enterprises based on MBO-LSTM-AE includes the following steps:

[0010] Step 1: Collect energy consumption data of large carbon emission detection enterprises;

[0011] Step 2: Convert the data format of the energy consumption data of large carbon emission detection enterprises collected in step 1;

[0012] Step 3: Perform missing value interpolation and normalization processing on the energy consumption data of large carbon emission detection enterprises in DataFrame format;

[0013] Step 4: Establish a monarch butterfly optimization algorithm to optimize the long short-term memory autoencoder model to identify outliers in the energy consumption data of large carbon emission monitoring enterprises in the DataFrame format in step 3;

[0014] Moreover, the specific method of step 1 is:

[0015] Extract energy consumption data of large carbon emission detection enterprises from smart meters through REST API communication protocol.

[0016] Moreover, the specific method of step 2 is:

[0017] The step 2 converts the energy consumption data format of the large carbon emission monitoring enterprise from the JSON format to the DataFrame format in the Pandas library.

[0018] Moreover, the specific steps of step 3 include:

[0019] (1) In order to reduce the adverse effects of missing data on outlier detection, a multi-order Lagrange interpolation method is used to repair the missing data in the data set. The formula is:

[0020]

[0021] In the formula, m1 is the forward point number; m2 is the backward point number; t is the specific time when the electricity meter of the large carbon emission detection enterprise is missing data.

[0022] (2) Normalize the data set to the interval [-1,1], the formula is:

[0023]

[0024] In the formula, x represents the data before normalization; x′ represents the data after normalization; x max and x min are the maximum and minimum values ​​of energy consumption data of large carbon emission monitoring enterprises; y max and min are the upper limit (1) and lower limit (-1) of the normalized target interval, respectively.

[0025] Moreover, the specific steps of step 4 include:

[0026] (1) Select the hyperparameters that need to be optimized for the LSTM-AE model, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate;

[0027] The LSTM-AE model combines the long short-term memory unit LSTM and the autoencoder AE, and is divided into two parts: the encoder and the decoder, in which the encoder and the decoder contain multiple LSTM layers;

[0028] Among them, the long short-term memory unit LSTM consists of three parts: forget gate, input gate and output gate. The forget gate discards irrelevant information, the input gate determines the new information stored in the unit state, and the output gate controls the output of the hidden layer node;

[0029] The autoencoder AE includes an encoder and a decoder;

[0030] During the encoding process, for the high-dimensional input X, the encoder maps it to a low-dimensional space through an activation function;

[0031] In the decoding process, the decoder reversely maps Y to a high-dimensional space through an activation function to achieve data reconstruction;

[0032] (2) Define the fitness function: Accuracy is used as the fitness function, which will evaluate the quality of the model based on its performance on the training data:

[0033]

[0034] Where TP, TN, FP and FN are true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) in the confusion matrix respectively.

[0035] (3) Initializing the monarch butterfly population: Initially, a group of monarch butterflies is randomly generated. Each butterfly represents a set of LSTM-AE hyperparameters, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate. The combination of these hyperparameters constitutes a solution space, and the monarch butterfly population will search for the optimal solution in this solution space.

[0036] (4) Update the monarch butterfly population: The population is divided into sub-population 1, sub-population 2, and sub-population 3. The individuals in the three sub-populations are all affected by the migration operator with local search capability and the adjustment operator with global search capability. Through continuous iteration, the monarch butterfly population will adjust the hyperparameters according to the evaluation results of the fitness function and evolve towards a better solution.

[0037] Among them, the migration operator with local search capability is:

[0038]

[0039] In the formula, When X is t+1 iterations i The position of the k-th monarch butterfly i, which contains a set of LSTM-AE hyperparameters; r1 and r2 are random individuals from populations 1 and 2 respectively; R is a random number between 0 and 1 that conforms to the uniform distribution; p e is the migration period, which is usually set to 1.2; p is the migration rate, which is used to determine the ratio of monarch butterflies in population 1 and population 2, and its value is 5 / 12.

[0040] The adjustment operator with global search capability is:

[0041]

[0042] In the formula, When X is t+1 iterations i The position of monarch butterfly j in the kth dimension; is the optimal individual X at t iterations best The kth dimension of is the kth dimension of the tth generation monarch butterfly randomly selected from population 3; B a is the adjustment rate. In the monarch butterfly algorithm, the adjustment rate B a Equal to the mobility p; S max is the maximum step length of the monarch butterfly; levy is the Levy flight formula;

[0043] (5) Set the stopping criteria: when the maximum number of iterations is reached or the fitness reaches the threshold, the optimization process is stopped;

[0044] (6) Output optimal hyperparameters: After the optimization process is completed, the hyperparameters corresponding to the individuals with the highest fitness in the monarch butterfly population are considered to be the optimal hyperparameters of LSTM-AE, namely, the optimal number of LSTM layers, the number of neurons in the LSTM layer, and the Dropout rate.

[0045] (7) Construct the LSTM-AE model based on the optimal hyperparameters obtained by MBO optimization and perform outlier detection;

[0046] (8) A so-called anomaly score is introduced, and the anomaly score is defined as the absolute value of the difference between the reconstructed energy consumption measurement and the actual measurement relative to the average difference of the entire training data set; each reconstructed sequence of LSTM-AE is compared with the original data. The greater the difference between the reconstructed data and the actual data, the higher the probability of anomaly, which is represented by an anomaly threshold; a threshold based on the anomaly score distribution is defined. For the aggregation level, the threshold is 1.5, corresponding to the 95% quantile of the distribution of all anomaly scores in the training data set; for the device level, the threshold is 0.5, corresponding to the 99% quantile; when the anomaly score of the predicted sequence exceeds the threshold, LSTM-AE issues an anomaly signal.

[0047] After step 4, the following steps are also included:

[0048] Step 5: Build an agent to visualize the outlier detection results of step 4 and trigger an alarm warning;

[0049] Step 6: Conduct a comprehensive evaluation of the outlier detection results of step 4 using multiple evaluation indicators.

[0050] Moreover, the agent in step 5 includes two sub-agents, one responsible for visualizing the outlier detection results; and the other responsible for triggering an alarm when an outlier is detected and implementing an outlier alarm through email, SMS and application.

[0051] Moreover, the step 6 evaluates the outlier detection efficiency of the method through four evaluation indicators: confusion matrix, accuracy, recall rate, and precision.

[0052] An outlier detection system for large-scale carbon emission monitoring enterprises based on MBO-LSTM-AE, including:

[0053] Data acquisition module, data input module, data preprocessing module, model building module, intelligent agent module and evaluation module;

[0054] The data collection module is used to collect energy consumption data of large carbon emission detection enterprises and send the energy consumption data of large carbon emission detection enterprises to a cloud-based energy management system;

[0055] The data input module is used to convert the data format of the collected energy consumption data of large carbon emission detection enterprises;

[0056] The data preprocessing module is used to interpolate missing values ​​and normalize the energy consumption data of large carbon emission detection enterprises in DataFrame format;

[0057] Model building module, used to build a monarch optimization algorithm to optimize the long short-term memory autoencoder model and identify outliers in the energy consumption data of large carbon emission monitoring enterprises in DataFrame format;

[0058] The intelligent agent module is used to construct an intelligent agent, visualize the outlier detection results and trigger an alarm warning;

[0059] The evaluation module is used to conduct an all-round evaluation of the outlier detection results through multiple evaluation indicators.

[0060] Advantages and beneficial effects of the present invention:

[0061] 1. In order to realize accurate outlier detection of energy consumption data of large-scale carbon emission monitoring enterprises (such as steel mills), the present invention proposes an unsupervised energy consumption data outlier detection method based on the Monarch Butterfly Optimization (MBO) optimized long short-term memory based Autoencoder (LSTM-AE). There is no need to mark the abnormal event type in advance to train the model. The complex patterns in energy consumption data can be unsupervisedly learned and captured, and outlier detection can be realized efficiently, so that large-scale carbon emission monitoring enterprises can identify outliers in energy consumption data and the causes of abnormalities.

[0062] 2. The present invention proposes an unsupervised outlier detection method based on the Monarch Butterfly Optimization (MBO) algorithm to optimize the long short-term memory autoencoder (Long Short-term Memory based Autoencoder, LSTM-AE), which is used to identify outliers in energy consumption data of large carbon emission monitoring enterprises at the aggregation and device levels. The present invention includes six modules: data acquisition module, data input module, data preprocessing module, model building module, intelligent agent module and evaluation module. This design makes the system easier to understand, maintain and expand, while also providing flexibility so that each module can operate independently or be replaced to meet the specific needs of different enterprises and data sources. This method uses the Monarch Butterfly Optimization Algorithm to optimize the long short-term memory autoencoder (MBO-LSTM-AE) model as the core of outlier detection. Compared with traditional methods, the MBO-LSTM-AE model can perform outlier detection without labeled data, which reduces the difficulty of deployment and maintenance, and is particularly suitable for the actual scenarios of carbon emission monitoring enterprises. In addition, as a deep learning method, MBO-LSTM-AE is able to capture complex patterns in time series data, thereby improving the accuracy of outlier detection in energy consumption data of large carbon emission monitoring enterprises. This enables the system to identify potential problems in a timely manner, helping enterprises to take effective energy-saving and emission reduction measures. Ultimately, this outlier detection method for energy consumption data of large carbon emission monitoring enterprises is able to more accurately capture abnormal patterns in time series data, which is crucial for large carbon emission monitoring enterprises, so that the ability to quickly and accurately identify outliers helps to detect potential problems early, reduce energy waste in the production process, and improve energy utilization efficiency. Experiments show that this method can accurately detect outliers at both the aggregate and device levels, and has high engineering significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a process flow chart of the present invention;

[0064] Figure 2 It is a graph of abnormal value detection results of the overall table data of the entire metal processing plant of the present invention;

[0065] Figure 3 This is a graph of abnormal value detection results of the laser drilling machine equipment in the factory of the present invention. DETAILED DESCRIPTION

[0066] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0067] A large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE, such as Figure 1 As shown, the following steps are included:

[0068] Step 1: Collect energy consumption data of large carbon emission detection enterprises, and send the energy consumption data (original electricity meter data) of large carbon emission detection enterprises to a cloud-based energy management system;

[0069] The specific method of step 1 is:

[0070] The energy consumption data of large carbon emission detection enterprises is extracted from smart meters through the REST API communication protocol, and then transmitted to the cloud-based energy management system.

[0071] In this embodiment, the energy consumption data of a large carbon emission detection enterprise is sent to a cloud-based energy management system through step 1. At this time, the system uses REST API as a communication protocol to extract data from smart meters and then transmit the data to the cloud-based energy management system.

[0072] Step 2: Convert the data format of the energy consumption data of large carbon emission detection enterprises collected in step 1;

[0073] The specific method of step 2 is:

[0074] The step 2 converts the energy consumption data format of the large carbon emission monitoring enterprise from the JSON format to the DataFrame format in the Pandas library.

[0075] In this embodiment, after converting the data into DataFrame format, you can easily use the various functions provided by Pandas to perform data manipulation, screening, statistics, and analysis. In addition, the Pandas data frame organizes data in a tabular form, making the data easier to understand and manipulate.

[0076] Step 3: Perform missing value interpolation and normalization processing on the energy consumption data of large carbon emission detection enterprises in DataFrame format;

[0077] The specific steps of step 3 include two parts: missing value interpolation and normalization, which can be implemented by DPUs.

[0078] The specific steps include:

[0079] (1) In order to reduce the adverse effects of missing data on outlier detection, a multi-order Lagrange interpolation method is used to repair the missing data in the data set. The formula is:

[0080]

[0081] In the formula, m1 is the forward point number; m2 is the backward point number; t is the specific time when the electricity meter of the large carbon emission detection enterprise is missing data.

[0082] (2) Normalize the data set to the interval [-1,1], the formula is:

[0083]

[0084] In the formula, x represents the data before normalization; x′ represents the data after normalization; x max and x min are the maximum and minimum values ​​of energy consumption data of large carbon emission monitoring enterprises; y max and min are the upper limit (1) and lower limit (-1) of the normalized target interval, respectively.

[0085] Step 4: Establish a Monarch Butterfly Optimization (MBO) optimized Long Short-term Memory based Autoencoder (LSTM-AE) model to identify outliers in the energy consumption data of large carbon emission monitoring enterprises in the DataFrame format in step 3;

[0086] The specific steps of step 4 include:

[0087] (1) Select the hyperparameters that need to be optimized for the LSTM-AE model, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate;

[0088] The choice of hyperparameters directly affects the performance of the model. By optimizing these parameters, the model's ability to fit the data and generalize can be improved.

[0089] The LSTM-AE model combines the characteristics of long short-term memory unit (LSTM) and autoencoder (AE), and can be divided into two parts: encoder and decoder, in which the encoder and decoder contain multiple LSTM layers.

[0090] Among them, the long short-term memory unit (LSTM) consists of three parts: the forget gate, the input gate and the output gate. The forget gate can discard irrelevant information, the input gate determines the new information stored in the unit state, and the output gate controls the output of the hidden layer node.

[0091] The autoencoder (AE) specifically includes an encoder and a decoder. During the encoding process, for a high-dimensional input X, the encoder maps it to a low-dimensional space through an activation function.

[0092] During the decoding process, the decoder reversely maps Y to a high-dimensional space through an activation function to achieve data reconstruction.

[0093] (2) Define the fitness function: Accuracy is used as the fitness function, which will evaluate the quality of the model based on its performance on the training data:

[0094]

[0095] Where TP, TN, FP and FN are true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) in the confusion matrix respectively.

[0096] In this embodiment, the fitness function should be designed so that the performance of the model gradually improves during the optimization process.

[0097] The confusion matrix is ​​as follows:

[0098] The predicted value is 1 The predicted value is 0 The true value is 1 TN FP The true value is 0 FN TP

[0099] (3) Initializing the monarch butterfly population: Initially, a group of monarch butterflies is randomly generated. Each butterfly represents a set of LSTM-AE hyperparameters, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate. The combination of these hyperparameters constitutes a solution space, and the monarch butterfly population will search for the optimal solution in this solution space.

[0100] (4) Update the monarch butterfly population: The population is divided into sub-population 1, sub-population 2, and sub-population 3. The individuals in the three sub-populations are all affected by the migration operator with local search capability and the adjustment operator with global search capability. Through continuous iteration, the monarch butterfly population will adjust the hyperparameters according to the evaluation results of the fitness function and evolve towards a better solution.

[0101] Among them, the migration operator with local search capability is:

[0102]

[0103] In the formula, When X is t+1 iterations i The position of the k-th monarch butterfly i in the k-th dimension contains a set of LSTM-AE hyperparameters (the number of LSTM layers, the number of neurons in the LSTM layer, and the Dropout rate); r1 and r2 are random individuals from populations 1 and 2 respectively; R is a random number between 0 and 1 that conforms to the uniform distribution; p e is the migration period, which is usually set to 1.2; p is the migration rate, which is used to determine the ratio of monarch butterflies in population 1 and population 2, and its value is 5 / 12.

[0104] The adjustment operator with global search capability is:

[0105]

[0106] In the formula, When X is t+1 iterations i The position of monarch butterfly j in the kth dimension; is the optimal individual X at the tth iteration best The kth dimension of is the kth dimension of the tth generation monarch butterfly randomly selected from population 3; B a is the adjustment rate. In the monarch butterfly algorithm, the adjustment rate B a Equal to the mobility p; S max is the maximum step length of the monarch butterfly; levy is the Levy flight formula.

[0107] (5) Set the stopping criteria: when the maximum number of iterations is reached or the fitness reaches the threshold, the optimization process is stopped;

[0108] (6) Output optimal hyperparameters: After the optimization process is completed, the hyperparameters corresponding to the individuals with the highest fitness in the monarch butterfly population are considered to be the optimal hyperparameters of LSTM-AE, namely, the optimal number of LSTM layers, the number of neurons in the LSTM layer, and the Dropout rate.

[0109] In this embodiment, the specifications of the model and related hyperparameters are shown in Table 1.

[0110] Table 1 Model specifications and related hyperparameters

[0111]

[0112] (7) The LSTM-AE model is constructed based on the optimal hyperparameters obtained by MBO optimization, and outlier detection is performed.

[0113] (8) In order to be able to detect anomalies in energy consumption with the trained LSTM-AE, a so-called anomaly score is introduced. The anomaly score is defined as the absolute value of the difference between the reconstructed energy consumption measurement and the actual measurement relative to the average difference of the entire training data set; each reconstructed sequence of the LSTM-AE is compared with the original data. The greater the difference between the reconstructed data and the actual data, the higher the probability of anomaly, which is represented by an anomaly threshold; a threshold based on the anomaly score distribution is defined. For the aggregation level, the threshold is 1.5, corresponding to the 95% quantile of the distribution of all anomaly scores in the training data set; for the device level, the threshold is 0.5, corresponding to the 99% quantile; when the anomaly score of the predicted sequence exceeds the threshold, the LSTM-AE issues an anomaly signal.

[0114] Step 5: Build an agent to visualize the outlier detection results of step 4 and trigger an alarm warning;

[0115] The agent in step 5 includes two sub-agents, one responsible for visualizing the outlier detection results; and the other responsible for triggering an alarm when an outlier is detected and implementing an outlier alarm through email, SMS and application.

[0116] In this embodiment, the overall processing process of the outlier detection method for energy consumption data of a large carbon emission monitoring enterprise based on LSTM-AE is as follows: Figure 1 The trained LSTM-AE is applied to the sampled test datasets at the aggregate and device levels. Figure 2 Depicts the load curve and detected anomalies at the aggregate level during the test data period, with the blue line representing the load curve and each red dot representing an anomaly. Results are shown at the device level for a shorter period of time than at the aggregate level to better identify details. Figure 3 The load profiles and detected anomalies at the device level from 2020-10-25 to 2020-10-26 are shown in blue and red.

[0117] Step 6: Conduct a comprehensive evaluation of the outlier detection results of step 4 using multiple evaluation indicators.

[0118] The step 6 evaluates the outlier detection efficiency of the method through four evaluation indicators, namely, confusion matrix, accuracy, recall rate, and precision.

[0119] The confusion matrix is ​​a table used to evaluate the performance of a classification model based on the relationship between the model's predictions and the actual labels. Accuracy is already given.

[0120] The formulas for recall and precision are:

[0121]

[0122] An outlier detection system for large-scale carbon emission monitoring enterprises based on MBO-LSTM-AE, including:

[0123] Data acquisition module, data input module, data preprocessing module, model building module, intelligent agent module and evaluation module;

[0124] The data collection module is used to collect energy consumption data of large carbon emission detection enterprises and send the energy consumption data of large carbon emission detection enterprises to a cloud-based energy management system;

[0125] The data input module is used to convert the data format of the collected energy consumption data of large carbon emission detection enterprises;

[0126] The data preprocessing module is used to interpolate missing values ​​and normalize the energy consumption data of large carbon emission detection enterprises in DataFrame format;

[0127] Model building module, used to build a monarch optimization algorithm to optimize the long short-term memory autoencoder model and identify outliers in the energy consumption data of large carbon emission monitoring enterprises in DataFrame format;

[0128] The intelligent agent module is used to construct an intelligent agent, visualize the outlier detection results and trigger an alarm warning;

[0129] The evaluation module is used to conduct an all-round evaluation of the outlier detection results through multiple evaluation indicators.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

Claims

1. A large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE, characterized by: The following steps are involved: Step 1: Collect energy consumption data of large carbon emission detection enterprises; Step 2: Convert the data format of the energy consumption data of large carbon emission detection enterprises collected in step 1; Step 3: Perform missing value interpolation and normalization processing on the energy consumption data of large carbon emission detection enterprises in DataFrame format; Step 4: Establish a monarch butterfly optimization algorithm to optimize the long short-term memory autoencoder model to identify outliers in the energy consumption data of large carbon emission monitoring enterprises in the DataFrame format in step 3.

2. According to claim 1, a large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE is characterized in that: The specific method of step 1 is: Extract energy consumption data of large carbon emission detection enterprises from smart meters through REST API communication protocol.

3. According to claim 1, a large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE is characterized in that: The specific method of step 2 is: The step 2 converts the energy consumption data format of the large carbon emission monitoring enterprise from the JSON format to the DataFrame format in the Pandas library.

4. According to claim 1, a large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE is characterized in that: The specific steps of step 3 include: (1) In order to reduce the adverse effects of missing data on outlier detection, a multi-order Lagrange interpolation method is used to repair the missing data in the data set. The formula is: In the formula, m1 is the forward point number; m2 is the backward point number; t is the specific time when the meter of a large carbon emission detection enterprise is missing data; (2) Normalize the data set to the interval [-1,1], the formula is: In the formula, x represents the data before normalization; x′ represents the data after normalization; x max and x min are the maximum and minimum values ​​of energy consumption data of large carbon emission monitoring enterprises; y max and min are the upper limit (1) and lower limit (-1) of the normalized target interval, respectively.

5. According to claim 1, a large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE is characterized in that: The specific steps of step 4 include: (1) Select the hyperparameters that need to be optimized for the LSTM-AE model, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate; The LSTM-AE model combines the long short-term memory unit LSTM and the autoencoder AE, and is divided into two parts: the encoder and the decoder, in which the encoder and the decoder contain multiple LSTM layers; Among them, the long short-term memory unit LSTM consists of three parts: forget gate, input gate and output gate. The forget gate discards irrelevant information, the input gate determines the new information stored in the unit state, and the output gate controls the output of the hidden layer node; The autoencoder AE includes an encoder and a decoder; During the encoding process, for the high-dimensional input X, the encoder maps it to a low-dimensional space through an activation function; During the decoding process, the decoder reversely maps Y to a high-dimensional space through an activation function to achieve data reconstruction; (2) Define the fitness function: Accuracy is used as the fitness function, which will evaluate the quality of the model based on its performance on the training data: Where: TP, TN, FP and FN are true positive examples, true negative examples, false positive examples and false negative examples in the confusion matrix respectively; (3) Initializing the monarch butterfly population: Initially, a group of monarch butterflies is randomly generated. Each butterfly represents a set of LSTM-AE hyperparameters, including the number of LSTM layers, the number of neurons in the LSTM layer, and the dropout rate. The combination of these hyperparameters constitutes a solution space, and the monarch butterfly population will search for the optimal solution in this solution space. (4) Update the monarch butterfly population: The population is divided into sub-population 1, sub-population 2, and sub-population 3. The individuals in the three sub-populations are all affected by the migration operator with local search capability and the adjustment operator with global search capability. Through continuous iteration, the monarch butterfly population will adjust the hyperparameters according to the evaluation results of the fitness function and evolve towards a better solution. Among them, the migration operator with local search capability is: In the formula, When X is t+1 iterations i The position of the k-th monarch butterfly i, which contains a set of LSTM-AE hyperparameters; r1 and r2 are random individuals from populations 1 and 2 respectively; R is a random number between 0 and 1 that conforms to the uniform distribution; p e is the migration period, which is usually set to 1.2; p is the migration rate, which is used to determine the ratio of monarch butterflies in population 1 and population 2, and its value is 5 / 12; The adjustment operator with global search capability is: In the formula, When X is t+1 iterations i The position of monarch butterfly j in the kth dimension; is the optimal individual X at the tth iteration best The kth dimension of is the kth dimension of the tth generation monarch butterfly randomly selected from population 3; B a is the adjustment rate. In the monarch butterfly algorithm, the adjustment rate B a Equal to the mobility p; S max is the maximum step length of the monarch butterfly; levy is the Levy flight formula; (5) Set the stopping criteria: when the maximum number of iterations is reached or the fitness reaches the threshold, the optimization process is stopped; (6) Outputting the optimal hyperparameters: After the optimization process is completed, the hyperparameters corresponding to the individuals with the highest fitness in the monarch butterfly population are considered to be the optimal hyperparameters of LSTM-AE, namely, the optimal number of LSTM layers, the number of neurons in the LSTM layer, and the Dropout rate; (7) Construct the LSTM-AE model based on the optimal hyperparameters obtained by MBO optimization and perform outlier detection; (8) A so-called anomaly score is introduced, and the anomaly score is defined as the absolute value of the difference between the reconstructed energy consumption measurement and the actual measurement relative to the average difference of the entire training dataset; each reconstructed sequence of LSTM-AE is compared with the original data. The greater the difference between the reconstructed data and the actual data, the higher the probability of anomaly, which is represented by the anomaly threshold; a threshold based on the anomaly score distribution is defined; for the aggregation level, the threshold is 1.5, corresponding to the 95% quantile of the distribution of all anomaly scores of the training dataset; for the device level, the threshold is 0.5, corresponding to the 99% quantile; when the anomaly score of the predicted sequence exceeds the threshold, LSTM-AE sends an anomaly signal.

6. According to claim 1, a large-scale carbon emission monitoring enterprise outlier detection method based on MBO-LSTM-AE is characterized in that: After step 4, the following steps are also included: Step 5: Build an agent to visualize the outlier detection results of step 4 and trigger an alarm warning; Step 6: Conduct a comprehensive evaluation of the outlier detection results of step 4 using multiple evaluation indicators.

7. According to claim 6, a large-scale carbon emission monitoring enterprise abnormal value detection method based on MBO-LSTM-AE is characterized in that: The agent in step 5 includes two sub-agents, one responsible for visualizing the outlier detection results; and the other responsible for triggering an alarm when an outlier is detected and implementing an outlier alarm through email, SMS and application.

8. According to claim 6, a large-scale carbon emission monitoring enterprise abnormal value detection method based on MBO-LSTM-AE is characterized in that: The step 6 evaluates the outlier detection efficiency of the method through four evaluation indicators: confusion matrix, accuracy, recall rate, and precision.

9. A large-scale carbon emission monitoring enterprise outlier detection system based on MBO-LSTM-AE, characterized by: include: Data acquisition module, data input module, data preprocessing module, model building module, intelligent agent module and evaluation module; The data collection module is used to collect energy consumption data of large carbon emission detection enterprises and send the energy consumption data of large carbon emission detection enterprises to a cloud-based energy management system; The data input module is used to convert the data format of the collected energy consumption data of large carbon emission detection enterprises; The data preprocessing module is used to interpolate missing values ​​and normalize the energy consumption data of large carbon emission detection enterprises in DataFrame format; Model building module, used to build a monarch optimization algorithm to optimize the long short-term memory autoencoder model and identify outliers in the energy consumption data of large carbon emission monitoring enterprises in DataFrame format; The intelligent agent module is used to construct an intelligent agent, visualize the outlier detection results and trigger an alarm warning; The evaluation module is used to conduct an all-round evaluation of the outlier detection results through multiple evaluation indicators.

Images