Multi-head attention fused variational auto-encoder heat metering data anomaly detection method

Abstract

The invention belongs to the technical field of industrial data analysis, and discloses a variational auto-encoder heat metering data anomaly detection method fusing multi-head attention, which comprises the steps of collecting original heat metering data to construct a feature data set, taking feature data in the feature data set under a normal working condition as a training set, and taking the remaining feature data as a test set; taking a variational auto-encoder as an anomaly detection model; training an anomaly detection model by using the training set, and calculating a loss function in the training process to update parameters of the anomaly detection model; after training is finished, the test set is used for testing the anomaly detection model, and if the test is passed, the anomaly detection model is output for real-time anomaly detection of the heat metering data; otherwise, training the anomaly detection model again. The method can better adapt to a complex heat supply system environment, the dependence on labels is reduced, and the anomaly detection accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial data analysis, and specifically relates to a method for detecting anomalies in heat metering data by using a variational autoencoder with multi-head attention. In particular, it relates to the use of deep learning technology for time series data processing and anomaly detection. Specifically, the present invention is applied to a heat metering system, and uses an advanced neural network model: combining a multi-head attention mechanism, a gated recurrent unit (GRU), and a variational autoencoder (VAE) to detect and analyze abnormal patterns in heat metering data, so as to improve the efficiency and reliability of the energy management system. Background Art

[0002] As the scale of heating systems continues to expand, various potential problems in their operation are gradually emerging. Heating systems may face a series of problems such as boiler failure, heat exchanger damage, incomplete combustion, and water loss problems in the heating network, such as network leakage, sewage discharge, and user water discharge. In addition, the secondary network connects many users with different needs, involving a large and extensive pipeline network, which makes the situation more complicated.

[0003] Therefore, it is particularly important to effectively monitor and diagnose abnormal data of single-user heat meters in the secondary pipe network. Accurate anomaly detection not only helps to timely adjust and optimize energy use, improve energy efficiency, and reduce waste, but also has great significance for reducing overall carbon emissions.

[0004] Currently, anomaly detection of user-side heat meter data usually relies on rules or traditional statistical methods. These methods can be roughly divided into two categories:

[0005] 1. Rule-based anomaly detection: This type of method relies on expert experience or preset rules to identify anomalies. Although it has a certain degree of accuracy in some scenarios, it is slow to respond to complex and changing systems. Especially in heating systems, traditional fixed rules are not flexible enough when faced with thermal inertia, dynamic changes and complex user needs.

[0006] 2. Model-based statistical methods: This type of method detects anomalies by building statistical models. However, since traditional models mostly use linear or simple nonlinear methods, it is difficult to handle the multi-dimensional and complex time series information in heat meter data, and cannot cope with the nonlinear characteristics and complex time dependencies in the data.

[0007] With the development of deep learning technology, data-driven anomaly detection methods have gradually emerged, especially anomaly detection applied to multi-dimensional time series data. Although deep learning technology has shown superior performance in some scenarios, it still faces the following challenges in practical applications:

[0008] 1. Dependence on a large amount of labeled data: Deep learning models usually require a large amount of data with abnormal labels for training, but in heating systems, abnormal data is often scarce and difficult to label. This strong dependence on labeled data limits the universality of the model in practical applications.

[0009] 2. Difficulty in capturing rare anomalies: The abnormal situations in the heating system are complex and diverse, and some anomalies may never appear in historical data. Traditional supervised learning models are difficult to detect these rare anomalies.

[0010] Therefore, current anomaly detection methods, whether traditional rule-based methods or modern methods relying on deep learning, have obvious limitations. Summary of the invention

[0011] The purpose of the present invention is to provide a variational autoencoder heat metering data anomaly detection method integrating multi-head attention, which can better adapt to the complex heating system environment, reduce dependence on labels, and improve the accuracy of anomaly detection.

[0012] To achieve the above object, the technical solution adopted by the present invention is: A method for detecting anomaly of thermal metering data by using a variational autoencoder fused with multi-head attention, the method comprising: Collect the original heat metering data to build a feature data set, take the feature data under normal working conditions in the feature data set as the training set, and the rest as the test set; A variational autoencoder is used as an anomaly detection model, wherein the encoder of the variational autoencoder includes a layer of gated recurrent units, the decoder of the variational autoencoder includes two layers of gated recurrent units connected in sequence, and the encoder is connected to a multi-head attention mechanism layer; The anomaly detection model is trained using the training set, and the loss function is calculated during the training process to update the parameters of the anomaly detection model, wherein the loss function includes a reconstruction loss, a KL divergence, and an entropy regularization term based on the attention weight output by the multi-head attention mechanism layer; After the training is completed, the anomaly detection model is tested using the test set. If the test passes, the anomaly detection model is output to perform real-time anomaly detection of thermal metering data; otherwise, the anomaly detection model is retrained.

[0013] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0014] Preferably, the collecting of original heat metering data to construct a feature data set includes: Acquire the original heat metering data of the current time step from the heat meter of each household, wherein the original heat metering data includes supply water temperature, return water temperature, flow rate and thermal power; Calculate the temperature difference between the supply water temperature and the return water temperature, and the thermal power per square meter. At the same time, select the supply water temperature and flow rate of the previous time step as the lag time characteristics, which are recorded as the supply water temperature lag and flow rate lag respectively. The supply water temperature hysteresis, return water temperature, temperature difference, flow rate hysteresis and thermal power per square meter are taken as characteristic values, and the flow rate is taken as the target value to obtain the characteristic data set.

[0015] Preferably, the gated recurrent unit of the encoder includes 64 hidden units, and the activation function is the tanh function; the first layer of the gated recurrent unit of the decoder includes 64 hidden units, and the activation function is the tanh function; the second layer of the gated recurrent unit of the decoder includes 6 hidden units, and the activation function is the relu function.

[0016] Preferably, the multi-head attention mechanism in the multi-head attention mechanism layer has 4 attention heads.

[0017] Preferably, the loss function calculation formula is as follows: In the formula, represents the final loss function, represents the reconstruction loss, represents the KL divergence, represents the entropy regularization term of the attention weights output by the multi-head attention mechanism layer, is a hyperparameter; In the formula, is the sample size, Indicates The target amount in the input samples, Indicates that the anomaly detection model is targeted at The reconstruction of the output of the input sample is is the total number of dimensions of latent variables, represents the first hidden variable after being encoded by the encoder The standard deviation of the dimension, represents the first hidden variable after being encoded by the encoder The mean of the dimensions, represents the attention weight.

[0018] Preferably, the output anomaly detection model performs real-time anomaly detection of heat metering data, including: Obtain real-time raw thermal metering data and convert it into real-time characteristic data; Inputting the real-time feature data into an anomaly detection model to obtain a real-time reconstruction amount output by the anomaly detection model; The reconstruction residual is calculated based on the real-time feature data and the real-time reconstruction amount, and the dynamic threshold is set using the SPOT algorithm. If the reconstruction residual value exceeds the dynamic threshold, it is judged that the current working condition is abnormal; otherwise, it is a normal working condition.

[0019] The present invention provides a method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention, which has the following beneficial effects compared with the prior art: 1. Reduce reliance on labels: The method of the present invention adopts the idea of ​​regression, establishes a model through supervised learning of normal data, and identifies abnormal points that deviate from the normal pattern without relying on abnormal samples. The model shows stronger adaptability and wide applicability in areas where abnormal labels are scarce or difficult to obtain (such as heating systems), solving the traditional method's need for large-scale labeled data.

[0020] 2. Performance improvement: The method of the present invention combines the multi-head attention mechanism and the GRU module to effectively capture the dependencies in complex multi-dimensional time series data, so that the model has stronger feature extraction capabilities when processing nonlinear and dynamically changing heating data. Compared with traditional linear or simple nonlinear models, the method of the present invention performs better in detection efficiency and accuracy, and significantly improves the sensitivity to anomalies and the stability of detection.

[0021] 3. Flexibility and scalability: The method of the present invention enhances the generalization ability of unseen data based on variational autoencoders, and at the same time controls the complexity of the attention mechanism and improves the model's adaptability by introducing a custom loss function with entropy regularization. This design enables the model to flexibly cope with data sets of different sizes and a variety of operating environments, improving data processing efficiency and response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a method for detecting anomaly of thermal metering data by a variational autoencoder integrating multi-head attention according to the present invention; Figure 2 It is a structural schematic diagram of the anomaly detection model of the present invention; Figure 3 It is a loss convergence curve diagram of the anomaly detection model on the training set and the test set in the experiment of the present invention; Figure 4-Figure 7 It is the probability density distribution diagram of the predicted value and the true value under the test set corresponding to different households in the experiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0025] The present invention provides a variational autoencoder thermal metering data anomaly detection method fused with multi-head attention, which is used for anomaly detection of thermal metering data. By reconstructing the data under normal working conditions, the training model can learn the normal behavior of the system, so as to identify abnormal points that deviate from the normal mode without relying on abnormal samples. Its purpose is to provide a method that can effectively identify and process abnormal patterns in large-scale and complex time series data, especially in the field of thermal energy supply and management, which can improve the accuracy and efficiency of anomaly detection.

[0026] like Figure 1 As shown, the variational autoencoder thermal metering data anomaly detection method of this embodiment fused with multi-head attention includes the following steps: 1) Data collection: The original heat metering data is obtained from the heat meter of each household. The original heat metering data includes multi-dimensional information such as supply water temperature, return water temperature, flow rate (instantaneous flow rate), thermal power, accumulated heat, etc., and is transmitted to the cloud system for centralized storage and monitoring.

[0027] 2) Feature data set construction: A feature data set is constructed based on the original thermal metering data. The feature data under normal working conditions in the feature data set are selected according to prior knowledge as the training set, and the remaining data that may contain anomalies are used as the test set.

[0028] Based on the original heat metering data, a new feature is constructed: the temperature difference between the supply water temperature and the return water temperature (Temp_Difference). The heat power per square meter is obtained by dividing the heat power by the actual residential area of ​​the household. At the same time, some characteristic values ​​of the previous time step are selected as lag time features, and the lag time features of the water supply temperature and flow are added accordingly, which are recorded as the water supply temperature lag and flow lag, respectively.

[0029] The supply water temperature hysteresis, return water temperature, temperature difference, flow hysteresis, thermal power and accumulated heat are selected as characteristic values, and the flow is selected as the target quantity. Linear normalization is performed on each dimension, and the sliding window generator class (Window_Generator) is used to generate data slices with a window size of 30 to obtain the characteristic data set.

[0030] 3) Model training: Take the variational autoencoder as the anomaly detection model, use the training set to train the anomaly detection model, and calculate the loss function during the training process to update the parameters of the anomaly detection model.

[0031] 3.1) Structure of anomaly detection model.

[0032] This embodiment uses a GRU-VAE (Multi-Head Attention GRU-Variational Autoencoder) deep learning model that integrates a multi-head attention mechanism as an anomaly detection model. The specific structure diagram is as follows: Figure 2 As shown in Figure 2. Variational Autoencoder VAE is a deep generative model based on the autoencoder structure. The basic structure of VAE consists of two parts: encoder and decoder. Its core idea is to map the input data to a latent space and generate new data from this space.

[0033] The role of the encoder is to map high-dimensional input data to a distribution in the latent space. This is usually achieved through a neural network, which outputs the mean and variance parameters of the latent variable to describe the Gaussian distribution. This mapping makes VAE different from traditional autoencoders, which only output a certain latent representation. The task of the decoder is to reconstruct the input data based on the latent representation output by the encoder.

[0034] In the original variational autoencoder (VAE), the encoder and decoder usually use a simple fully connected neural network (FCNN) structure. This structure performs well when processing independent and identically distributed (IID) data, and can effectively map the input to the latent space and reconstruct data similar to the input. However, for time series data with time dependence, this structure has obvious shortcomings:

[0035] 1. Lack of time dependency processing: Fully connected networks cannot capture the time dependencies in the input data, especially when processing time series like heat metering data, the correlation of data at different moments is important, and traditional fully connected networks ignore these features.

[0036] 2. Limited expressiveness: Time series data often contain complex dynamic patterns, and relying solely on simple encoder and decoder structures may not be sufficient to effectively capture these complex temporal dependencies.

[0037] Therefore, in this embodiment, a gated recurrent unit (GRU) is used as the encoder and decoder of the variational autoencoder. The basic idea is to use the sequence processing capability of GRU to learn the potential representation of the input data. The reasons are as follows:

[0038] 1. Enhanced time series modeling capabilities: GRU has a specially designed gating mechanism (update gate and reset gate) that can capture and retain long-term dependencies in time series and address the shortcomings of fully connected networks in processing time correlations.

[0039] 2. Improve model stability: In the decoder part, a two-layer GRU structure is introduced. The first layer GRU is used to capture the temporal dynamics of the potential representation, and the second layer GRU further refines the reconstruction process, enhancing the model's ability to capture complex patterns and improving the quality and consistency of generated data.

[0040] 3. Optimize the learning process: The design of multi-layer GRU helps stabilize the model training process and improve the model's expressiveness on long-term dependent data. This solves the problem of insufficient expressiveness and generation quality of the original fully connected network when processing complex time series data.

[0041] In this embodiment, the GRU layer of the encoder contains 64 hidden units, and the activation function is the default tanh function; correspondingly, the first GRU layer of the decoder also contains 64 hidden units and uses the tanh activation function, and the second GRU layer contains 6 hidden units and uses the relu function activation. For the resampling process of VAE, the standard VAE implementation is used, and resampling is performed by calculating the mean and standard deviation without additional modification.

[0042] In addition, this embodiment adds a multi-head attention mechanism layer after the GRU encoder, so that the GRU first processes the dynamic characteristics of the time series, and then captures dependencies and features at different levels through the multi-head attention mechanism. By introducing the attention mechanism, the model can distribute attention weights over a larger range, thereby better capturing these fine-grained time dependencies. In addition, this embodiment introduces a multi-head attention mechanism, which is intended to operate simultaneously in different representation spaces of the model, giving the model higher flexibility and expressiveness.

[0043] The multi-head attention mechanism learns multi-level features from different subspaces by calculating multiple different attention heads in parallel, making the model more diverse and comprehensive in capturing different levels of data features while avoiding information loss. Attention weights are assigned by calculating the correlation between the query, key, and value, so that different parts can be selectively focused on when processing sequence data. Then the standard scaled dot-product attention is used to adjust the attention weights, and the calculation formula is:

[0044] in, is the attention weight, is the query matrix, is the key matrix, V is the value matrix, is the dimension of the key, used to scale the dot product to stabilize the softmax function.

[0045] In this embodiment, the number of attention heads is 4, so the formula of the multi-head attention mechanism can be expressed as: in, is the final output transformation matrix, Represents a splicing operation, , They are The transformation matrix of query, key, and value for each attention head.

[0046] The reason for choosing four attention heads is mainly based on the consideration of balancing model complexity and computational efficiency. First, multiple attention heads can extract feature information from different subspaces and enhance the expressiveness of the model, but too many heads may increase the computational burden and reduce training efficiency. Experimental verification shows that four heads can effectively capture the main feature dependencies in the data while maintaining the stability and efficiency of model training. This design also ensures that the model can achieve higher generalization capabilities with less computing resources when processing long sequence data.

[0047] In this embodiment, the output shape of the multi-head attention layer is [(None, 30, 64), (None, 4, 30,30)]. Among them, (None, 30, 64) represents the final feature representation after multi-head attention processing: None represents the batch size, which can be flexibly changed according to the input; 30 refers to the number of time steps, indicating that the input time series data contains 30 time steps, and 64 corresponds to the feature dimension at each time step. (None, 4, 30, 30) is the attention weight matrix, which represents the correlation weight calculated by each attention head for each pair of time steps in 30 time steps: 4 represents the number of attention heads, the first 30 represents the time step, which is consistent with the input time step, and the second 30 represents the size of the attention weight matrix.

[0048] The multi-head attention mechanism layer is followed by the "GlobalAveragePooling" layer, which averages the time dimension of the multi-head attention results, which helps to reduce the data dimension and extract global information. Then two fully connected layers with the same configuration are connected to output the mean of the learned variational autoencoder latent space ( ) and variance ( ). Then connect the sampling layer and use the predicted and To adjust the noise, so as to generate Hidden variables of distribution The global average pooling layer of this embodiment is used to globally compress the features output by the multi-head attention layer, and evenly merge the features of 30 time steps into a 64-dimensional feature vector. This layer ensures that the output features of the attention mechanism can be further passed to the subsequent fully connected layer for processing, which simplifies the output dimension of the model and retains the main global feature information.

[0049] 3.2) Loss function design.

[0050] In the variational autoencoder, the loss function mainly consists of two parts: reconstruction loss and KL divergence. Reconstruction loss is used to measure the similarity between the model output (reconstructed data) and the original input data. In the model of this embodiment, the loss is calculated using mean square error (MSE). The formula is:

[0051] in is the sample size, Indicates The target amount in the input samples, Indicates that the anomaly detection model is targeted at The smaller the reconstruction loss, the closer the output generated by the model is to the input data, thus effectively retaining the characteristics of the original data.

[0052] KL divergence (Kullback-Leibler divergence) loss is a core component of variational autoencoders and is used to quantify the distribution of latent variables learned by the model. With prior distribution The KL divergence formula is:

[0053] in, is the total number of dimensions of latent variables, represents the first hidden variable after being encoded by the encoder The standard deviation of the dimension, represents the first hidden variable after being encoded by the encoder The loss ensures that the potential representation learned by the model is consistent with the prior distribution and helps generate reasonable samples.

[0054] This embodiment introduces the entropy regularization of attention weights into the original loss function, aiming to regularize the weight distribution of the multi-head attention mechanism, encourage the model to use all possible attention paths, avoid over-reliance on specific paths, and enhance the expressiveness of the model. Entropy is an indicator of the uniformity of distribution. The higher the entropy value, the more uniform the distribution. The formula is:

[0055] in Represents the attention weight. Each attention head has its own attention weight distribution. The weight represents the degree of attention of the model on different input positions. The sum is the sum of all attention weights. This embodiment encourages the model to disperse its attention by maximizing entropy so that it does not overly focus on a few inputs.

[0056] Combining the above three parts, we get the final loss function of the model, which is used for optimization during the training process. The formula is as follows: Here is a hyperparameter used to adjust the contribution of the entropy regularization term to the total loss. Through this design, the model can not only effectively reconstruct the input data and learn the potential representation, but also capture the complex dependencies in the time series data through the attention mechanism, thereby improving the performance of anomaly detection.

[0057] The present invention introduces entropy regularization of attention weights. This method not only enriches the learning ability of the model, but also solves the limitations of traditional variational autoencoders in dealing with complex dependencies. By encouraging diverse attention paths, the model can better capture long-range dependencies and multi-level features in the data while retaining the sensitivity of the order of time series data, thereby effectively improving the accuracy and robustness of anomaly detection.

[0058] The anomaly detection model of the present invention uses the GRU layer as a codec to process time series data. The advantage is that it can effectively capture the temporal dynamic characteristics in the sequence and reduce the gradient vanishing problem in the training process through a gating mechanism, thereby maintaining long-term dependent information. And a multi-head attention layer is added after the encoder to enhance the model's ability to capture complex time dependencies in heating data. This mechanism allows the model to focus on multiple time points and features at the same time, so as to better handle the nonlinear and dynamic changes of time series data. At the same time, a composite loss function including reconstruction loss, KL divergence and attention weight entropy is specially designed to optimize the overall performance of the model, ensuring that while improving the accuracy of anomaly detection, it can also maintain high sensitivity to new anomaly types. Therefore, the anomaly detection model of the present invention shows significant advantages in anomaly detection of heat metering data, and can achieve higher detection accuracy and lower false alarm rate. In addition, the model has good scalability and flexibility, and can easily adapt to different types of data sets and a variety of operating environments, significantly improving the efficiency and response speed of data processing.

[0059] 4) Anomaly detection: After training, the anomaly detection model is tested using the test set. If the test passes, the anomaly detection model is output to perform real-time anomaly detection of thermal metering data; otherwise, the anomaly detection model is retrained.

[0060] The application process of the anomaly detection model is as follows: obtain real-time original heat metering data and convert it into real-time feature data; input the real-time feature data into the anomaly detection model to obtain the real-time reconstruction amount output by the anomaly detection model; calculate the reconstruction residual based on the real-time feature data and the real-time reconstruction amount (the reconstruction residual of this embodiment uses the difference between the real-time feature data and the real-time reconstruction amount), and use the SPOT algorithm to set a dynamic threshold. If the reconstruction residual value exceeds the dynamic threshold, it is judged that the current working condition is abnormal; otherwise, the current working condition is normal.

[0061] By learning the characteristic distribution under normal working conditions, the anomaly detection model of the present invention can effectively improve the detection ability of unknown anomalies. This feature makes the method more widely applicable in practical applications, especially in the field of heating systems where abnormal samples are difficult to obtain, and has important practical application value and broad market prospects.

[0062] In order to prove the effectiveness of the method of the present invention, an experimental description is given below.

[0063] 1) Model evaluation indicators.

[0064] In the multivariate time series anomaly detection task, the goal is to predict the overall situation of each test sample, especially to determine whether each test sample is abnormal at a specified timestamp and give a corresponding label. However, time series anomaly detection tasks usually face the challenge of severe imbalance of data sets, that is, the number of abnormal data samples is far less than that of normal data samples. Therefore, in order to accurately evaluate the performance of the model, evaluation indicators based on the confusion matrix are usually used, including precision, recall, and F1 value.

[0065] Define a confusion matrix. The four elements in the confusion matrix are defined as follows: TP (True Positive) indicates that the true category of the sample is abnormal (positive class) and the model correctly predicts it as abnormal; FN (False Negative) indicates that the true category of the sample is abnormal (positive class), but the model incorrectly predicts it as normal; FP (False Positive) indicates that the true category of the sample is normal (negative class), but the model incorrectly predicts it as abnormal; TN (True Negative) indicates that the true category of the sample is normal (negative class), and the model correctly predicts it as normal. Among them, the positive class represents abnormal events, and the negative class represents normal situations.

[0066] Although precision, recall, and F1 value provide a comprehensive evaluation of the classification performance of the anomaly detection task, this experiment also uses R 2 Indicators are used to evaluate the model’s ability to reconstruct time series data. 2 Or the coefficient of determination, is an indicator used to measure the effect of model reconstruction, especially suitable for regression tasks. Therefore, in this experiment, R 2 It is used to measure the performance of the model in reconstructing time series data, while precision, recall and F1 value are used to evaluate the classification ability of the model in detecting abnormal data.

[0067] 2) Experimental data.

[0068] A new type of heat meter was installed in a residential area in Liaocheng, Shandong Province. The meter has built-in supply and return temperature, flow and heat measurement functions. That is, the heat meter can uniformly collect the supply water temperature (℃), return water temperature (℃), instantaneous flow (℃) in the pipeline according to the set time. )、heat power(kw)、monthly flow( ), monthly heat (kWh), cumulative heat (kWh), cumulative water (t), valve opening, etc. The original heat metering data of each household in the community during the 2023-2024 heating season were collected, and a feature data set was constructed for the experiment. A feature data set was constructed from the original heat metering data of one household.

[0069] In the experiment, each feature data set was divided into a training set and a validation set in a ratio of 8:2, and all data containing anomalies were independently set as test sets. Since the feature data set of this experiment has obvious time series characteristics, the training set, validation set, and test set are strictly divided in chronological order to ensure that the model does not leak future information during the validation and testing process. At the same time, the early stopping strategy was adopted during the training of the model. When the performance index on the validation set no longer improves in 10 consecutive training rounds, the training is stopped to prevent the model from overfitting and accelerate the training process. In addition, all experiments were run multiple times in the same environment to ensure the stability and repeatability of the results.

[0070] 3) Experimental setup.

[0071] The detailed hardware and software environment of this experiment is shown in Table 1: Table 1 Experimental environment name parameter Operating system version Windows 10.0.22621 CPU 12th Gen Intel(R) Core(TM) i5-12500H GPU NVIDIA GeForce RTX 2050 Memory 16 GB Python Version Python 3.7.0 Pytorch Version 1.8.0 CUDA Version 11.7 4) Experimental results.

[0072] like Figure 3 The figure shows the changing trend of the loss value with the rounds during model training. It can be seen from the figure that: in the initial stage, the loss of the training set and the test set decreases rapidly, indicating that the model is learning quickly and gradually fitting the data; after about 25 epochs, the losses of the training set and the test set tend to be stable, indicating that the model gradually converges to a stable state; at the 29th epoch, the loss of the test set and the loss of the training set both decrease significantly. The possible reason is that the introduced multi-head attention mechanism helps the model better capture important features, thereby leading to a sudden improvement in performance at a specific training stage; in the final stage, both losses tend to be stable, and the early stopping mechanism is triggered when training to the 177th epoch, avoiding too many training rounds and thus reducing the training time without causing overfitting.

[0073] Figure 4-Figure 7 The comparison of the probability density distribution (Kernel Density Estimation, KDE) of the model prediction value (reconstruction amount) and the true value under different household data is shown in the figure. Figure 4 is the density map corresponding to household a, Figure 5 is the density map corresponding to household b, Figure 6is the density map corresponding to household c, Figure 7 is the density map corresponding to household d. The KDE map can be used to intuitively evaluate the model's fit to the data distribution. The solid line in the figure represents the distribution of the true FLOW value, and the dotted line represents the distribution of the FLOW value predicted by the model.

[0074] As can be seen from the figure, the model fits the data distribution well in most areas. Especially in the main peak area (around 0.5), the distribution of the predicted value and the true value almost overlap, indicating that the model can well capture the characteristic distribution under normal heating mode. In addition, at the tail of the distribution (around 1.5), the density between the predicted value and the true value is low, indicating that the model fails to handle rare abnormal data well.

[0075] In this comparative experiment, the performance of three models was evaluated: BiGRU (Bidirectional Gated Recurrent Unit), CNN-LSTM, and GRU-VAE of the present invention. Precision, recall, F1, and determination coefficient were used as the main evaluation indicators. The experimental results are shown in Table 2, where the BiGRU model has a precision of 68.11%, a recall of 64.05%, an F1 score of 64.73%, and a determination coefficient R 2 The overall performance is relatively low, indicating that it has limited ability to capture data features and predict anomalies. The CNN-LSTM model has improved in all indicators, with a precision of 79.03%, a recall of 76.03%, an F1 score of 76.56%, and a determination coefficient R 2 The GRU-VAE model showed significant advantages, with an accuracy of 92.16%, a recall of 70.39%, an F1 score of 77.22%, and a determination coefficient R 2 The value of the proposed model increases from 0.9795 to 0.9795, indicating that its accuracy and reliability in anomaly detection are significantly higher than those of other models.

[0076] Table 2 Comparative experimental results Model Precision(%) Recall(%) F1(%) <![CDATA[R 2 ]]> BiGRU 68.11 64.05 64.73 0.5988 CNN-LSTM 79.03 76.03 76.56 0.6700 GRU-VAE 92.16 70.39 77.22 0.9795 In general, compared with the two baseline models (BiGRU and CNN-LSTM), the GRU-VAE model of the present invention performs better in various indicators, especially in accuracy and determination coefficient, showing its powerful ability in processing complex time series data.

[0077] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention, characterized in that: The variational autoencoder thermal metering data anomaly detection method integrating multi-head attention includes: Collect the original heat metering data to build a feature data set, take the feature data under normal working conditions in the feature data set as the training set, and the rest as the test set; A variational autoencoder is used as an anomaly detection model, wherein the encoder of the variational autoencoder includes a layer of gated recurrent units, the decoder of the variational autoencoder includes two layers of gated recurrent units connected in sequence, and the encoder is connected to a multi-head attention mechanism layer; The anomaly detection model is trained using the training set, and the loss function is calculated during the training process to update the parameters of the anomaly detection model, wherein the loss function includes a reconstruction loss, a KL divergence, and an entropy regularization term based on the attention weight output by the multi-head attention mechanism layer; After the training is completed, the anomaly detection model is tested using the test set. If the test passes, the anomaly detection model is output to perform real-time anomaly detection of thermal metering data; otherwise, the anomaly detection model is retrained.

2. The method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention according to claim 1 is characterized in that: The collecting of original heat metering data to construct a feature data set includes: Acquire the original heat metering data of the current time step from the heat meter of each household, wherein the original heat metering data includes supply water temperature, return water temperature, flow rate and thermal power; Calculate the temperature difference between the supply water temperature and the return water temperature, and the thermal power per square meter. At the same time, select the supply water temperature and flow rate of the previous time step as the lag time characteristics, which are recorded as the supply water temperature lag and flow rate lag respectively. The supply water temperature hysteresis, return water temperature, temperature difference, flow rate hysteresis and thermal power per square meter are taken as characteristic values, and the flow rate is taken as the target value to obtain the characteristic data set.

3. The method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention according to claim 1 is characterized in that: The gated recurrent unit of the encoder includes 64 hidden units, and the activation function is the tanh function; the first layer of the gated recurrent unit of the decoder includes 64 hidden units, and the activation function is the tanh function; the second layer of the gated recurrent unit of the decoder includes 6 hidden units, and the activation function is the relu function.

4. The method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention according to claim 1 is characterized in that: The multi-head attention mechanism in the multi-head attention mechanism layer has 4 attention heads.

5. The method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention according to claim 1 is characterized in that: The loss function calculation formula is as follows: In the formula, represents the final loss function, represents the reconstruction loss, represents the KL divergence, represents the entropy regularization term of the attention weights output by the multi-head attention mechanism layer, is a hyperparameter; In the formula, is the sample size, Indicates The target amount in the input samples, Indicates that the anomaly detection model is targeted at The reconstruction of the output of the input sample is is the total number of dimensions of latent variables, represents the first hidden variable after being encoded by the encoder The standard deviation of the dimension, represents the first hidden variable after being encoded by the encoder The mean of the dimensions, represents the attention weight.

6. The method for detecting anomaly in thermal metering data by using a variational autoencoder with multi-head attention according to claim 1, characterized in that: The output anomaly detection model performs real-time anomaly detection of heat metering data, including: Obtain real-time raw thermal metering data and convert it into real-time characteristic data; Inputting the real-time feature data into an anomaly detection model to obtain a real-time reconstruction amount output by the anomaly detection model; The reconstruction residual is calculated based on the real-time feature data and the real-time reconstruction amount, and the dynamic threshold is set using the SPOT algorithm. If the reconstruction residual value exceeds the dynamic threshold, it is judged that the current working condition is abnormal; otherwise, it is a normal working condition.

Images