IoT data online prediction system based on memory replay variational autoencoder

Abstract

The application provides an IoT data online prediction system based on a memory replay variational autoencoder, which comprises a prediction module and a training module; the prediction module is used for inputting IoT data to be predicted into a trained memory replay VAE to obtain a prediction result; the training module is used for training the memory replay VAE, wherein the memory replay VAE comprises an encoder and a generator; the training process of the memory replay VAE is as follows: first sample data are input into the encoder to obtain first sample latent factors and a first sample prediction result; the generator obtains first sample replay data based on the first sample latent factors; second sample data and the first sample replay data are input into the encoder to obtain fusion sample latent factors and a corresponding prediction result; a loss function is calculated based on labels and the obtained prediction result; and the training is completed when the loss is minimum. The application combines an OLVAE with an attention mechanism and a brain replay mechanism, alleviates the forgetting of old knowledge by the encoder, and realizes efficient prediction of IoT data.

Description

Technical Field

[0001] This invention belongs to the field of sensor time series technology, and particularly relates to an online prediction system for IoT data based on a memory-replay variational autoencoder. Background Technology

[0002] In recent years, the widespread application of various sensors has led to a dramatic increase in the generation of time-series data. In fields such as intelligent environmental monitoring, industrial monitoring, and data center monitoring, the importance of real-time and intelligent analysis is becoming increasingly prominent.

[0003] Time-series data observed by sensors are affected by a variety of unknown factors, such as physical or chemical changes in the environment, contaminant adhesion, and sensor performance degradation, causing most flow cytometry data to be non-stationary and prone to dynamic distribution drift. This phenomenon is called concept drift in machine learning, meaning that data changes in unpredictable ways over time.

[0004] Deep learning models excel at capturing time-series patterns, but offline-trained models struggle to adapt to data streams with varying distributions. First, their representational capabilities are limited by a predefined, fixed structure, allowing adjustment only of parameters between hidden layers within that structure, neglecting the importance of adjusting different hidden layers and hindering flexible model structure adjustments. Second, when sensor data sampling rates are high, performance degradation occurs if the model cannot quickly adapt to this new data. Finally, this approach cannot adjust model complexity, hindering effective model selection.

[0005] The aforementioned problems with offline learning will accumulate decision biases, affecting the model's ability to learn complex patterns. Furthermore, the "one-time review" strategy of streaming data analysis makes errors difficult to correct, which will have serious adverse effects on sensor monitoring systems such as data centers and smart oceans, and may even threaten system security.

[0006] Online learning methods can learn from data streams quickly and in real time. However, the current challenge is how to enable the model to learn from shallow to deep as the data stream progresses, thereby effectively alleviating the concept drift problem and overcoming overfitting when the amount of data is small and underfitting when the amount of data is large. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes an online prediction system for IoT data based on a memory replay variational autoencoder. By co-training the encoder and generator and combining attention and brain replay mechanisms, the system mitigates the encoder's forgetting of old knowledge, thereby achieving efficient prediction of IoT time series data.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides an online prediction system for IoT data based on a memory-replay variational autoencoder, comprising:

[0010] The prediction module is used to acquire IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result.

[0011] The training module is used to train the memory replay VAE; the memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows:

[0012] Input the first sample data into the encoder to obtain the first sample latent factors and the first sample prediction results;

[0013] Input the latent factors of the first sample into the generator to obtain the replay data of the first sample;

[0014] The second sample data and the first sample replay data are input into the encoder to obtain the latent factors of the fusion category, as well as the prediction results of the first and second samples;

[0015] Based on the prediction results and labels of the first sample, as well as the prediction results and labels of the second sample, the loss function is calculated. Training is complete when the loss is minimized.

[0016] Preferably, the encoder includes a hidden layer and an output layer; the hidden layer includes an attention mechanism for adjusting the participation weights of the hidden layer to obtain a prediction result based on the input IoT data; the output layer is used to obtain the latent factors of the IoT data based on the input IoT data.

[0017] The generator is used to generate replay IoT data based on potential factors.

[0018] Preferably, the hidden layer includes an attention mechanism to adjust the participation weights of the hidden layer and obtain a prediction result based on the input IoT data, specifically including:

[0019] α j =weight(p j )

[0020]

[0021] Where, `prediction(*)` is a feedforward network that transforms the hidden representation into local predictions, and `weight(*)` is a feedforward network with a softmax function that computes the weights of each hidden layer; the weights of all layers are normalized to... The weight(*) is designed to determine the degree of necessity for each substructure in the final prediction.

[0022] Preferably, the replay data represents the distribution characteristics of potential factors.

[0023] Secondly, the present invention provides an online prediction method for IoT data based on a memory-replay variational autoencoder, comprising:

[0024] Acquire the IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result;

[0025] The memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows:

[0026] Input the first sample data into the encoder to obtain the first sample latent factors and the first sample prediction results;

[0027] Input the latent factors of the first sample into the generator to obtain the replay data of the first sample;

[0028] The second sample data and the first sample replay data are input into the encoder to obtain the latent factors of the fusion category, as well as the prediction results of the first and second samples;

[0029] Based on the prediction results and labels of the first sample, as well as the prediction results and labels of the second sample, the loss function is calculated. Training is complete when the loss is minimized.

[0030] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the IoT data online prediction method based on a memory replay variational autoencoder described in the second aspect.

[0031] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the IoT data online prediction method based on a memory replay variational autoencoder described in the second aspect.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. This invention proposes an online deep learning model based on variational autoencoders, called the OLVAE model. By integrating an attention mechanism, this model can dynamically adjust the contributions of each substructure, thereby flexibly adjusting the model's complexity and representational power.

[0034] 2. This invention proposes a "brain generation and replay mechanism" that generates data representing the distribution of past data to reactivate neural activity patterns representing past memories, reducing storage requirements while improving the time series prediction performance of sensors to address the challenges of concept drift.

[0035] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0037] Figure 1 This is a schematic diagram of the overall structure of the memory playback VAE (OLVAE) provided in Embodiment 1 of the present invention;

[0038] Figure 2 This is the attention-based prediction fusion path provided in Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of an online learning paradigm provided in Embodiment 1 of the present invention;

[0040] Figure 4 This is a schematic diagram of the prediction results of the OLVAE model provided in Embodiment 1 of the present invention on multiple sensors;

[0041] Figure 5 This is a schematic diagram illustrating the evolution of weights at each layer in the predicted CPU power provided in Embodiment 1 of the present invention.

[0042] Figure 6 This is a schematic diagram comparing the inference times of OLVAE, LSTM, and NBeats as provided in Embodiment 1 of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] Example 1

[0045] This embodiment provides an online prediction system for IoT data based on a memory-replay variational autoencoder, including:

[0046] The prediction module is used to acquire IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result.

[0047] The training module is used to train the memory replay VAE; the memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows:

[0048] Input the first sample data into the encoder to obtain the first sample latent factors and the first sample prediction results;

[0049] Input the latent factors of the first sample into the generator to obtain the replay data of the first sample;

[0050] The second sample data and the first sample replay data are input into the encoder to obtain the latent factors of the fusion category, as well as the prediction results of the first and second samples;

[0051] Based on the prediction results and labels of the first sample, as well as the prediction results and labels of the second sample, the loss function is calculated. Training is complete when the loss is minimized.

[0052] The overall framework of the memory playback VAE provided in this embodiment is as follows: Figure 1 As shown, next, combined with Figure 1 Please provide a detailed explanation.

[0053] 1) VAE-based encoder-generator path:

[0054] First, the encoder uses historical input data samples X1 = {x i ,x i+1 ,...,x i+hw ,...,x T Estimate the mean μ and variance σ 2 To determine the potential representation distribution of historical input data.

[0055] The generator samples latent factors z from the latent variable space and feeds them into the generator network. The generator is responsible for generating data representing the distribution characteristics of past data based on the latent factors z, that is, generating memory replay data X1'={x' i ,x' i+1 ,...,x' i+hw ,...,x' i+T}

[0056] Figure 1 This demonstrates the brain's generation and replay mechanism, with the replayed data X1' used for subsequent training. This embodiment aims to use latent factors z sampled from the latent variable space to help calibrate the probability distribution of the generated X1', ensuring it follows the probability distribution of the historical input data X1. Generally, the larger the latent variable space, the more historical probability distribution information can be retained.

[0057] The prior distribution p(z) of the latent factor z is assumed to be a Gaussian distribution. Since p(z) cannot be directly obtained in this embodiment, the encoder is responsible for inferring this distribution by modeling the posterior distribution p(z|x) when given input x. In other words, when given input x, the encoder estimates the Gaussian distribution parameters with mean μ and standard deviation σ. This process can be understood as representing the original data in a low-dimensional space of a Gaussian distribution. Considering that only a finite amount of data is provided, the estimated posterior probability p(z|x) is only an approximation. Therefore, the estimated distribution from the encoder is represented as q(z|x) to distinguish it from the true posterior distribution p(z|x). During model training, the encoder is optimized to approximate the true posterior probability.

[0058] In the brain, a mechanism considered crucial for maintaining memories is the reactivation of neuronal activity patterns representing memories, which is essential for stabilizing new memories. The brain appears to have implemented an efficient and scalable continuous learning algorithm, with this memory replay regulated by the hippocampus. Inspired by brain neuroscience, this embodiment proposes a "brain generation and replay mechanism."

[0059] First, let the generator and predictor share the initial encoder ( Figure 1 (Blue section) This establishes a bidirectional feedback loop between them. The generator learns by using the latent representations obtained from the encoder. Similarly, the predictor also uses the encoder to obtain representations for prediction; therefore, the generator and predictor share the encoder. This bidirectional feedback refers to the interaction between the encoder and generator, used for generator training, specifically manifested as… Figure 1 The black arrow in the image.

[0060] Through the feedback connection modulated by the network's own context, the generator ( Figure 1 The orange section is responsible for generating data Z' representing the distribution characteristics of past data (produced from the generator's first sample data input), and then passing the current data and the data X' representing the distribution characteristics of past data to the encoder to obtain the most likely label. (Predicted values). These data It is used as a replay dataset for subsequent training, rather than replaying the original training data.

[0061] Specifically, to predict the first sample data X1, the first sample data X1 is input into the encoder, and the prediction result of the first sample is obtained based on the first sample label Y1. Simultaneously, the encoder acquires the latent factors z1 of the first sample data X1, and the generator generates the first sample replay data X1' based on the latent factors z1.

[0062] Because the distributions of the first and second sample data are inconsistent, and the encoder learns only current knowledge, it gradually forgets past knowledge. Therefore, when predicting the second sample data, only the second sample data is input into the encoder for training. The encoder focuses on learning the data distribution of the current stage and gradually forgets the data distribution of the previous stage (i.e., the first sample data). Although the prediction effect is good in the current stage, the prediction effect will deteriorate when encountering the first sample data again in the next cycle.

[0063] To overcome the aforementioned catastrophic forgetting problem, this embodiment, when predicting the second sample data X2, inputs both the second sample data X2 and the first sample replay data X1' into the encoder, and obtains the first sample prediction result based on the first sample label Y1 and the second sample label Y2 respectively. Second Sample Prediction Results In addition, the encoder obtains the latent factors z1 of the first sample data and the latent factors z2 of the second sample data. The generator generates the second sample replay data X2' based on the latent factors z2 of the second sample data for subsequent training.

[0064] Each time the model is trained based on data from a new stage, historical replay data and current data from the new stage are input into the Memory Replay VAE (OLVAE). This allows the encoder to learn new knowledge from the current stage while also learning old knowledge from the past, avoiding forgetting past knowledge and improving the accuracy of the model's predictions of time series data.

[0065] In this model, data from different phases within the same period can be used as IoT time-series sample data for training. This embodiment uses seasonal data: relevant data from spring, summer, autumn, and winter phases as the first, second, third, and fourth sample data for training. After the generator generates historical simulation data (replay data), it is combined with the current data for encoder training, essentially reactivating the encoder. Since encoders tend to forget old knowledge after learning new knowledge in the current phase, this method can reactivate the encoder's past memories.

[0066] This process is similar to how the human brain's memory of an event becomes blurred in winter due to the passage of time after experiencing it in the summer. Subsequently, by consciously reviewing memory fragments related to summer, the original memory is reawakened and strengthened, which is essentially the same as reactivating that experience in memory.

[0067] In this embodiment, since the data generated by the generator has already extracted features from historical data, theoretically there is no need to store historical data. Only less data, i.e., potential factors, is needed to achieve the desired goal, thus eliminating the need to worry about data storage. This improvement mimics the structure of the brain, where the generator responsible for playback transmits signals to the encoder.

[0068] Therefore, when the encoder calculates μ and σ, it can sample z from the distribution and feed z further into the generator to generate data that represents the characteristics of the historical data distribution.

[0069] Due to the random sampling in this process, the gradient information flow in the architecture is interrupted, and the traditional backpropagation chain rules cannot be directly applied. Therefore, the VAE model employs a reparameterization technique to capture the properties of classic neural networks during model training. In reparameterization, the VAE uses random sampling to generate a set of variables ε that satisfy a normal distribution. 2 Sampling z in N(0,1) is equivalent to sampling ε from N(0,1), and then letting z = μ + σ. 2 *ε. Therefore, gradient descent can be applied. Finally, the generator decodes z and generates the replay sample X'.

[0070] 2) Attention-based prediction fusion path:

[0071] In addition to the encoder-generator path mentioned above for capturing the potential distribution of historical data, this embodiment also needs to establish an inference path for sensor data prediction, which is the core objective of this paper.

[0072] like Figure 2 As shown, this embodiment establishes an inference path on the encoder, utilizing and fusing information from each layer of the encoder. Furthermore, the inference network dynamically adjusts itself by introducing an attention mechanism onto the information extracted from each layer.

[0073] Suppose that the encoding path of the OLVAE model contains k hidden layers. Determining the appropriate k is challenging because if there is little data, a larger k tends to cause an "overfitting" problem, while if there is a large amount of data, a smaller k tends to cause an "underfitting" problem.

[0074] Considering the above issues, this embodiment chooses to set a relatively large k to ensure the model's representational capability. Simultaneously, an adjustment mechanism is introduced into the prediction path to ensure that the model's representational capability can be adjusted with the data flow. Without this adjustment mechanism, the model's representational capability is constant and is the sum of the capacities of each hidden layer, meaning all hidden layers contribute equally to the encoding. Introducing the adjustment mechanism allows the hidden layers to dynamically adapt to the amount of data to be processed.

[0075] To iteratively update the model, this embodiment incorporates a key technique called reweighting, which dynamically adjusts the contribution of each layer to the final prediction. Through reweighting, the weights of all layers are recalculated and normalized after processing each input sample. This embodiment adds an attention network to learn the attention weights of the hidden layers of the inference network, as shown in Equation (1):

[0076] α j =weight(p j (1)

[0077]

[0078] Where prediction(*) is a feedforward network that transforms the hidden representation into local predictions, and weight(*) is a feedforward network with a softmax function that computes the weights of each hidden layer. The weights of all layers are normalized to... The weight(*) is designed to determine the degree of necessity for each substructure in the final prediction.

[0079] Unlike traditional feedforward neural network models, whose final prediction is derived from the representation of the last layer, OLVAE's final prediction is obtained by weighting the predictions obtained from each hidden layer. The lower the weight, the less contribution that layer makes to predicting the current data. The model's final prediction is a fusion of sub-layer predictions, i.e., a weighted sum of predictions from multiple layers, as shown in Equation (2):

[0080]

[0081] Where p j It is the prediction of the j-th hidden layer representation.

[0082] It's worth noting that if a smaller number of blocks are sufficient to learn the data flow, the model can achieve fast convergence. Using attention mechanisms allows for dynamic tuning of networks with multiple layers, thus introducing plasticity to the model's representational capabilities.

[0083] 3) Loss function and parameter optimization:

[0084] For sensor data prediction, the most important learning objective is to minimize the prediction bias. Therefore, this embodiment first needs to define the prediction bias in the objective function. The prediction bias function is defined as follows:

[0085]

[0086] The loss function here can be any convex function; this embodiment uses the mean squared error loss function (MSE). In online prediction and online learning, this loss is evaluated for each sample in real time.

[0087] In addition to the prediction loss function, the training of the OLVAE model requires guidance from two other loss functions. One of them is the Kullback-Leibler divergence loss (KL loss), which measures the difference between the posterior distribution q(z|x) and the standard normal distribution p(z). Given a hidden variable space of dimension m with known mean μ and variance σ, the KL loss is defined as follows:

[0088]

[0089] The other is the replay loss L. replay This is used to guide the generation of inputs that represent previous learning, and can be a standard prediction loss. The replay loss is defined as follows:

[0090]

[0091] Finally, in order to optimize the model from the above three perspectives, this embodiment needs to minimize the above three losses at the same time. That is, the loss of OLVAE is the sum of the three losses: prediction loss, KL loss and replay loss, as shown in formula (6).

[0092] Loss = L p +L kL +L replay (6)

[0093] In practice, after each prediction round, the model adjusts based on the immediate loss to prepare for subsequent rounds. For traditional feedforward neural networks, the error derivative is backpropagated from the last output layer to the first layer. However, in the OLVAE model, the error derivative is backpropagated to each prediction block to adjust the corresponding parameters. In each iteration, the parameters W of each block are adjusted. i Update according to the following equation:

[0094]

[0095] 4) Online learning paradigms in data flow:

[0096] The preceding sections described the loss function for guided learning. Here, this embodiment clarifies the online learning paradigm of the proposed model within the sensor data stream, such as... Figure 3 As shown.

[0097] In this study, this embodiment employs a real-time prediction and learning paradigm. In practical applications, the OLVAE model iteratively updates based on the loss and gradient generated for each incoming data point. Let X = x0, x1, ..., x t ,...,x TThis represents time-series data generated in real time by the sensor, where the time index T is unbounded. In the online learning paradigm, this time series is not available all at once, but rather received sequentially. When performing prediction for the next time t, this embodiment can only access historical data x0, x1, ..., xt. t-1 This embodiment uses the most recent past data at time t for prediction, and then the actual value x is... t It can be used to measure prediction error. By observing x... t This real-time loss is calculated by comparing it with the predicted value, and necessary parameter adjustments are made for the subsequent round of prediction.

[0098] 5) Experimental results:

[0099] Validate the predictive performance of the OLVAE model on a selected sensor dataset. Figure 4 As shown in the figure, this embodiment plots the original and predicted data of the OLVAE model for each experimental dataset. The OLVAE model can flexibly adapt to changes in the distribution of sensor data and exhibits excellent prediction performance when faced with significant "concept drift". This experiment demonstrates that OLVAE is a robust prediction method suitable for sensor time series data with drift distribution.

[0100] In addition, such as Figure 5 As shown in (a), by printing the model weight evolution process for CPU Power, it can be observed that the attention weights of the three layers are not static, but are reweighted during online learning. In the early stages of the data flow, the contribution of each layer to power prediction is relatively stable. This is reasonable, as power also exhibits relatively stable behavior in the early stages. The first layer has a smaller effect compared to the other two layers. The deeper layers contribute more to the representation data distribution in the early stages of the data flow, especially the third layer. As the data distribution changes, this embodiment shows that the influence of the third layer increases. Conversely, the influence of the first layer gradually decreases. It is noteworthy that the weights of all three layers change significantly between 3000 and 3500. The weight of the second layer remains almost constant for most of the time until it suddenly increases vertically at the end of the data flow. This situation is understandable, as... Figure 5 As shown in (b), this is due to the significant abrupt change in the power curve, which increases the complexity of the data distribution. Therefore, deeper networks should take on more responsibility when modeling the underlying factors of the data. This experimental result verifies the effectiveness of the model in dynamic model selection in this embodiment.

[0101] In real-time prediction of sensor data, the online learning method proposed in this embodiment not only outperforms traditional deep learning methods in prediction performance but also in terms of time cost. Time cost is a crucial factor to consider in the processing and analysis of high-volume sensor data. This embodiment compares the time cost of its method and a benchmark model under different data scales. This embodiment selects data of different scales from a dataset to simulate sensor flow. The data scales are set to 1000, 5000, 10000, 15000, and 20000, respectively. This embodiment selects the NBeats and LSTM models for comparison.

[0102] like Figure 6 As shown, the OLVAE method consistently demonstrates higher inference efficiency than NBeats and LSTM across various data scales, and this advantage increases with increasing data scale. This indicates that the OLVAE model has a competitive advantage in sensor data processing. Theoretically, as a model composed of several fully connected network blocks, NBeats has higher inference efficiency than LSTM. LSTM contains a recursive mechanism, resulting in greater time consumption when processing sequential data. Figure 6 As can be seen, LSTM's inference speed is indeed the slowest.

[0103] This embodiment generates replay data representing the distribution of historical data. This replay data is input simultaneously when predicting current data, allowing the encoder to learn both new and old knowledge, mitigating the model's catastrophic forgetting of old knowledge. Furthermore, by storing replay data representing historical data instead of storing all data directly, storage requirements are significantly reduced. By reactivating past memory patterns, the model can better adapt to changes in data distribution, thereby improving performance in time series forecasting, especially when facing the challenge of concept drift. This mechanism draws inspiration from biological memory consolidation and replay processes, providing a novel and effective solution for time series forecasting.

[0104] The OLVAE model provided in this embodiment integrates an attention mechanism on top of a variational autoencoder, enabling the model to dynamically adjust the contributions of its various substructures. The OLVAE model can automatically adjust its complexity and representational power based on different features of the input data to adapt to different learning tasks. Because it can dynamically adjust complexity and representational power, the OLVAE model exhibits greater flexibility and adaptability when processing complex and varied online data.

[0105] Example 2

[0106] This embodiment discloses an online prediction method for IoT data based on a memory-replay variational autoencoder, including the following steps:

[0107] S1: Obtain the IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result;

[0108] S2: The memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows:

[0109] Input the first category data into the encoder to obtain the first category latent factors and the first category prediction results;

[0110] Input the first category of latent factors into the generator to obtain the first category of replay data;

[0111] The second category data and the first category replay data are input into the encoder to obtain the latent factors of the fused sample, as well as the prediction results of the first and second categories;

[0112] Based on the prediction results and labels of the first category, as well as the prediction results and labels of the first and second categories, a loss function is calculated. Training is complete when the loss is minimized.

[0113] Example 3

[0114] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the IoT data online prediction method based on a memory replay variational autoencoder as described in Embodiment 2 above.

[0115] Example 4

[0116] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the IoT data online prediction method based on a memory replay variational autoencoder as described in Embodiment 2 above.

[0117] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0118] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An online prediction system for IoT data based on a memory-replay variational autoencoder, characterized in that, include: The prediction module is used to acquire IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result. The training module is used to train the memory replay VAE; the memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows: The first sample data is input into the encoder to obtain the first sample latent factors and the first sample prediction result. The encoder includes a hidden layer and an output layer. The hidden layer includes an attention mechanism to adjust the participation weights of the hidden layer and obtain the prediction result based on the input IoT data. The output layer is used to obtain the latent factors of the IoT data based on the input IoT data. The first sample latent factors are input into the generator to obtain the first sample replay data. The generator is used to generate replay IoT data based on the latent factors. The generator samples the latent factors z from the latent variable space and feeds them into the generator network. The generator is responsible for generating data representing the distribution characteristics of past data based on the latent factors z. The second sample data and the first sample replay data are input into the encoder to obtain the latent factors of the fused sample, as well as the prediction results of the first and second samples; Based on the prediction results and labels of the first sample, as well as the prediction results and labels of the second sample, the loss function is calculated. Training is complete when the loss is minimized.

2. The IoT data online prediction system based on memory-replay variational autoencoder as described in claim 1, characterized in that, The hidden layer includes an attention mechanism to adjust the participation weights of the hidden layer and obtain prediction results based on the input IoT data, specifically including: in, It is a feedforward network that transforms hidden representations into local predictions. It is a The function uses a feedforward network to compute the weights of each hidden layer; the weights of all layers are normalized to... The aim is to determine the necessity of each substructure in the final prediction.

3. The IoT data online prediction system based on memory-replay variational autoencoder as described in claim 1, characterized in that, The replay data represents the distribution characteristics of potential factors.

4. A method for online prediction of IoT data based on a memory-replay variational autoencoder, characterized in that, include: Acquire the IoT data to be predicted, input the IoT data to be predicted into the trained memory replay VAE, and obtain the prediction result; The memory replay VAE includes an encoder and a generator; the training process of the memory replay VAE is as follows: The first sample data is input into the encoder to obtain the first sample latent factors and the first sample prediction result. The encoder includes a hidden layer and an output layer. The hidden layer includes an attention mechanism to adjust the participation weights of the hidden layer and obtain the prediction result based on the input IoT data. The output layer is used to obtain the latent factors of the IoT data based on the input IoT data. The first sample latent factors are input into the generator to obtain the first sample replay data. The generator is used to generate replay IoT data based on the latent factors. The generator samples the latent factors z from the latent variable space and feeds them into the generator network. The generator is responsible for generating data representing the distribution characteristics of past data based on the latent factors z. The second sample data and the first sample replay data are input into the encoder to obtain the latent factors of the fused sample, as well as the prediction results of the first and second samples; Based on the prediction results and labels of the first sample, as well as the prediction results and labels of the second sample, the loss function is calculated. Training is complete when the loss is minimized.

5. The method for online prediction of IoT data based on a memory-replay variational autoencoder as described in claim 4, characterized in that, The hidden layer includes an attention mechanism to adjust the participation weights of the hidden layer and obtain prediction results based on the input IoT data, specifically including: in, It is a feedforward network that transforms hidden representations into local predictions. It is a The function uses a feedforward network to compute the weights of each hidden layer; the weights of all layers are normalized to... The aim is to determine the necessity of each substructure in the final prediction.

6. The method for online prediction of IoT data based on a memory-replay variational autoencoder as described in claim 4, characterized in that, The replay data represents the distribution characteristics of potential factors.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the IoT data online prediction method based on memory replay variational autoencoder as described in any one of claims 4-6.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the IoT data online prediction method based on memory replay variational autoencoder as described in any one of claims 4-6.

Images