Time series data intermittent learning method and system based on time attention

Abstract

The invention relates to the fields of passive Internet of Things, artificial intelligence and the like, in particular to a time series data intermittent learning method and system based on time attention. Aiming at the problem of time series data processing in a passive sensing monitoring environment, samples meeting conditions are selected by adopting a variational inference guided adaptive batch selection and dynamic sample re-calibration method, then feature fusion is carried out based on a time attention mechanism in combination with a Bi-GRU algorithm, and the execution sequence of different modules in a machine learning task is considered, so that the time series data processing efficiency is improved. The whole model learning process is divided into a feed-forward layer, a back propagation layer and a feature fusion layer for hierarchical learning, a time-series data intermittent learning prediction mechanism based on time attention is proposed to ensure the precision of time-series data prediction, and a local optimal solution is searched on a limited decision domain to solve the problem of action state explosion.

Description

Technical Field

[0001] The present invention relates to the fields of passive Internet of Things, artificial intelligence, and in particular to an intermittent learning method of time series data based on temporal attention. Background Art

[0002] Passive IoT edge devices continuously obtain energy from the surrounding environment in different ways to maintain their basic functions, thus completely getting rid of the dependence on batteries, but energy interruptions occur. Therefore, the execution of machine learning is intermittent, and it is usually necessary to dynamically select an optimal action (e.g., reading sensors, learning, and classification) for passive sensor monitoring intermittently while the system is running to achieve its overall goal. Battery-free intermittent computing is fundamentally different from traditional computing because intermittent computing devices violate one of the most basic assumptions of stable power supply. In addition, systems that suspend and resume execution based on captured energy have a higher risk of missing the events of interest it wants to detect or learn. Since both energy acquisition and the occurrence of events of interest are intermittent and uncertain, the problem of intermittent learning of edge devices is a key scientific problem facing passive sensor monitoring.

[0003] In practical passive sensor monitoring applications, a large amount of time series data is usually available, and relevant machine learning algorithms (such as LSTM) are used to learn models based on historical observations to accurately predict the future. However, the temporal patterns hidden in historical information play an important role in the accurate prediction of long time series, and dynamic future information, such as calendar events, may have a strong impact on the prediction model. In addition, since energy needs to be obtained from the environment and the power supply system may be interrupted in the passive sensor monitoring environment, machine learning in this environment is an intermittent learning, which requires dynamically selecting an optimal operation (e.g., reading sensors, learning, and classification) at runtime so that its overall goal (i.e., the number and type of samples required to complete learning) is optimal. Therefore, how to ensure the accuracy of time series prediction models for intermittent learning in passive sensor monitoring environments becomes an important issue. Summary of the invention

[0004] Aiming at the problem of time series data processing in passive sensor monitoring environment, the adaptive batch selection and dynamic sample recalibration method guided by variational inference is used to select samples that meet the conditions. Then, the Bi-GRU algorithm is combined with the temporal attention mechanism to perform feature fusion. Considering the execution order of different modules in machine learning tasks, the entire model learning process is divided into feedforward layer, back propagation layer, and feature fusion layer for hierarchical learning. An intermittent learning prediction mechanism for time series data based on temporal attention is proposed to ensure the accuracy of time series data prediction, and to find the local optimal solution in a limited decision domain to solve the problem of action state explosion.

[0005] In order to achieve the above object, the technical solution provided by the present invention is a time series data intermittent learning method based on time attention, comprising the following steps:

[0006] Step 1, obtain the amount of energy obtained by the passive IoT edge device from the outside world at each time step, that is, obtain energy time series data;

[0007] Step 2: Sample selection is performed for energy time series data. The most representative samples are automatically selected from the energy time series data through adaptive batch selection and dynamic sample recalibration methods guided by variational inference.

[0008] Step 3: Build an online multi-layer Bi-GRU model based on the fusion attention mechanism and AdamW optimizer, input the energy number time series data sample in step 2, and output the energy number that can be obtained in the future time step;

[0009] Step 4: Define the energy level according to the amount of energy obtained, as well as the energy level required for each action of the passive IoT edge device at each time step, and model it through a graph neural network, where the nodes in the graph structure represent the action states corresponding to the energy levels, and the edges are the transition relationships between states. The final proximal policy optimization algorithm PPO is used to find all action sequences within a limited range and select the best action sequence.

[0010] Furthermore, the implementation of the variational inference guided adaptive batch selection method in step 2 is as follows;

[0011] For each new sample x in the data stream t , based on historical data Perform variational inference to estimate the contribution of the sample to the current learning task. The goal of variational inference is to infer the selection probability of the sample by maximizing the lower bound of evidence. The specific inference process is as follows:

[0012]

[0013] in, For sample x t Given historical data The probability of selection under, q(θ) is the variational distribution, θ is the parameter, E represents the expectation, which is used to calculate the lower bound of evidence in variational inference;

[0014] In practice, variational inference is performed for each sample x t Calculate a selection probability And decide whether the sample is selected based on the probability.

[0015] Furthermore, the implementation of the dynamic sample recalibration method in step 2 is as follows;

[0016] For each time step t, let For the model to sample x t The predicted value, y t is the true label, and the prediction error of the sample is Based on the error, the weight w of the sample t Dynamically updated via the following formula:

[0017]

[0018] Among them, λ is a hyperparameter that controls the weight update.

[0019] Furthermore, the specific implementation of step 2 is as follows;

[0020] Step 2.1: Input a series of time series sample streams {x1, x2, …, x t}, each sample x t They all carry certain time information and possible tags;

[0021] Step 2.2, variational inference: For each time step t, based on historical data That is, {x1,x2,…,x t-1}) and the current sample x t , calculate the sample x through variational inference t The probability of selection If the probability exceeds a certain preset threshold, it is considered that the sample has a greater contribution to the current learning task and should be selected for learning;

[0022] Step 2.3, dynamic sample recalibration: For each selected sample x t , first by calculating its prediction error in For the model to sample x t The predicted value, y t is the true label; then adjust the sample weight w according to the error t , that is, through the formula:

[0023]

[0024] Step 2.4, sample selection: Based on the selection probability calculated by variational inference and the dynamically adjusted weights, the most representative samples at the current moment are selected to form the current learning batch. And use it for subsequent incremental learning;

[0025] Step 2.5, update the hyperparameter λ through the incremental learning algorithm, using the currently selected sample batch Conduct training;

[0026] Step 2.6, loop execution: With the new sample stream x t+1 With the arrival of the new feature, the whole process continues, and the variational inference and sample recalibration mechanisms are dynamically adjusted according to the changes in the data stream to continuously optimize sample selection and model training.

[0027] Furthermore, in step 3, the specific processing process of the online multi-layer Bi-GRU model is as follows:

[0028] Step 3.1, input data preprocessing: select the time series data samples {x1, x2, …, x T}Standardization and normalization are performed to ensure that the data is within a reasonable range; where T represents the selection of samples of T time steps;

[0029] Step 3.2, Bi-GRU encoding: Input the processed time series data into the Bi-GRU module to obtain the forward and reverse hidden states of each time step, and concatenate the two hidden states as the output;

[0030] Step 3.3, fusion attention mechanism: by fusing the input features and Bi-GRU output features, calculating the attention weights, and performing weighted summation on each time step, a global temporal representation is obtained;

[0031] Step 3.4, prediction: input the weighted time series representation into the fully connected prediction layer and output the final prediction result;

[0032] Step 3.5, optimization training: Use AdamW optimizer to update the optimization model parameters and train the model by minimizing the MSE loss function;

[0033] The update formula of the AdamW optimizer is as follows:

[0034]

[0035] Where: θ t represents the online Bi-GRU model parameters, represents the mean value of the gradient, represents the variance of the gradient, ∈ is a smoothing term used to prevent zero division errors; λ is the weight decay coefficient, which controls the strength of regularization; η is the learning rate;

[0036] The specific loss function L is calculated as follows:

[0037]

[0038] Where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value;

[0039] Step 3.6, Weight Update: In each iteration, the AdamW optimizer updates the model parameters based on the calculated gradients and the regularization term.

[0040] Furthermore, the specific implementation of step 3.2 is as follows:

[0041] For each time step t, Bi-GRU generates two hidden states: one from the forward GRU, i.e., GRU forward , and the other comes from the reverse GRU, i.e. GRU backward ;

[0042]

[0043] Among them, x t is the time series data at time step t, refers to the hidden state generated by the forward GRU, Refers to the hidden state generated by the reverse GRU;

[0044] The output of Bi-GRU is the concatenation of these two hidden states, representing the information at each time step:

[0045]

[0046] Furthermore, the specific implementation of step 3.3 is as follows:

[0047] The input features x at each time step t and Bi-GRU output h t are concatenated and passed through a fully connected layer for nonlinear transformation:

[0048] f t =W f [x t ;h t ]+b f

[0049] Among them, W f and b f is the parameter to be learned, and the concatenated feature vector f t Contains the input data and the timing information extracted by Bi-GRU;

[0050] Based on the fusion feature f t , the importance of each time step is calculated through the attention mechanism, and the attention weight α t Calculated by:

[0051]

[0052] Among them, score(f t ,W att ) is the fusion feature ft and learnable weights W att The linear transformation can be expressed as:

[0053] score(f t ,W att ) = f t W att +b att

[0054] According to the attention weight α t , perform weighted summation on the fusion features of all time steps to obtain the final time series representation h att :

[0055]

[0056] Furthermore, in step 4, the actions taken by the passive IoT edge device include sensing data, executing a layer defined by the model in step 3, and waiting for the next time step, wherein the sensed data refers to the actual task data that the passive IoT edge device senses and is supposed to read; the learning process of the Bi-GRU model is divided into a feedforward layer, a back propagation layer, a feature fusion layer based on time attention, and a prediction optimization layer. The feedforward layer is used to extract preliminary features from the input data and perform nonlinear transformations to generate hidden representations. The input of the back propagation layer is the predicted output of the model, the true label, and the hidden state generated by the feedforward layer. The output is the updated model parameters. The goal of the back propagation layer is to optimize the weights of the model based on the loss function; the input of the feature fusion layer is the transformed content vector of the hidden state and the input feature data, and the output is the fused time series representation for prediction tasks. In the feature fusion layer, the model performs a weighted combination of the hidden state and other features generated by the feedforward layer through the time attention mechanism to generate the final global feature representation; waiting for the next time step means that in order to ensure the coherent execution of subsequent actions, no action is performed in the current time step to accumulate energy.

[0057] Furthermore, the goal of PPO is to maximize the following objective function:

[0058]

[0059] in:

[0060] r t (θ′) is the ratio of the new strategy to the old strategy:

[0061]

[0062] In the formula, is the advantage function, which is calculated by generalized advantage estimation and is used to estimate the advantage of the current action; ∈ is a small hyperparameter that controls the range of policy updates; Et is the expected symbol, which represents the expected operation for all time steps t. The clip function is used to limit the strategy ratio to an acceptable range. CLIP (θ′) is the clipping objective function of PPO, which is used to measure the difference between the current strategy and the old strategy; π θ′ (a t |s t ) is the probability distribution of the current strategy, indicating that in state s t Next, take action a t The probability of is the probability distribution of the old strategy.

[0063] The present invention also provides an intermittent learning system for time series data based on temporal attention, which is characterized by: comprising a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a method for intermittent learning of time series data based on temporal attention as described in the above technical solution.

[0064] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0065] This invention considers the utility of perception data and the execution order of different modules in machine learning tasks, combines temporal attention, uses the Bi-GRU algorithm and intermittent learning method, and proposes an intermittent learning mechanism for passive sensor data based on temporal attention to ensure the accuracy of the learning model. Its main innovation is to perform feature fusion based on the temporal attention mechanism, consider the execution order of different modules in machine learning tasks, divide the entire model learning process into feedforward layer, back propagation layer, and feature fusion layer for hierarchical learning, so as to ensure the performance of the time series data learning model in the passive sensing environment, and find the local optimal solution in a limited decision domain to solve the problem of action state explosion. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 The figure is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0068] Aiming at the problem of intermittent learning prediction of time series data in passive sensor monitoring, the present invention will solve it from the following two aspects.

[0069] On the one hand, from the perspective of time series data prediction, Bi-GRU is used to explicitly learn to construct hidden pattern representations, propagate future information in both forward and backward directions, and consider dynamic future information such as calendar events. In addition, the use of the temporal attention mechanism can better capture the potential patterns in historical data and focus on different parts of history to predict the future, that is, through the multimodal fusion mechanism, it is used to combine the features of different parts of history to better represent the future. However, this method does not consider the intermittent learning problem in passive sensor monitoring to ensure effective model learning, nor does it consider the sample selection problem to reduce computational overhead.

[0070] On the other hand, from the perspective of intermittent learning, the goal of intermittent learning is to enable edge devices that obtain energy to effectively and efficiently perform certain types of machine learning tasks, so that they can intermittently learn data and application semantic tasks related to machine learning. To this end, heuristic algorithms are designed to determine a series of actions to achieve the expected learning goals under energy-constrained conditions, helping intermittent learners decide whether to learn or discard training instances at runtime, which can increase the energy efficiency of the system.

[0071] like Figure 1 As shown, the present invention provides a time series data intermittent learning method based on time attention, comprising the following steps:

[0072] Step 1: Automatically select the most representative samples from the time series data. Through adaptive batch selection and dynamic sample recalibration guided by variational inference, the model can intelligently determine which samples are most critical to the learning task and select these samples for training.

[0073] The time series data here refers to the amount of energy that can be obtained from the outside world at each time step in a passive environment, and sample selection is performed based on the energy time series data.

[0074] In the process of online learning, samples are usually selected using predefined criteria (such as fixed thresholds or similarity metrics). Traditional methods often ignore the potential structure and dynamic changes of data streams. This scheme introduces variational inference and combines historical information of data streams to adaptively select samples.

[0075] For each new sample x in the data stream t Based on historical data, we Perform variational inference to estimate the contribution of the sample to the current learning task. The goal of variational inference is to infer the selection probability of the sample by maximizing the evidence lower bound (ELBO). The specific inference process is as follows:

[0076]

[0077] in, For sample x t Given historical data The probability of selection under , q(θ) is the variational distribution, θ is the sample selection method parameter, which refers to all the parameters that need to be optimized during the learning process. E represents the expectation, which is used to calculate the lower bound of evidence in variational inference. It is obtained by integrating the parameter θ with respect to the variational distribution q(θ).

[0078] In practice, variational inference performs t Calculate a selection probability The probability is used to determine whether the sample is selected. This selection process depends on the importance of the sample in the data stream relative to the historical data, thus achieving dynamic sample selection based on data stream characteristics.

[0079] In order to improve the effectiveness of sample selection, we introduce a dynamic sample recalibration mechanism to make the learning process more focused on important samples by adjusting the weight of each sample in real time. This mechanism dynamically weights each sample according to its prediction error, so that the model prioritizes samples that have a greater impact on prediction.

[0080] For each time step t, let For the method to sample x t The predicted value, y t is the true label, and the prediction error of the sample is Based on the error, the weight w of the sample t Dynamically updated via the following formula:

[0081]

[0082] Among them, λ is a hyperparameter that controls weight update, which is usually small (such as 0.1 or 0.01) to control the sensitivity of error to weight. Error ∈ t The larger the value, the less accurate the model's prediction on the sample. Therefore, the more important the sample is, the higher the weight w is. t The larger the prediction error, the smaller the weight will be. On the contrary, if the prediction error is small, the weight will be reduced, thereby reducing the impact of the sample on the method. For samples with larger weights (i.e., samples with larger errors), they will be given higher priority during the training process, and the model will pay more attention to these samples. For samples with smaller weights (i.e., samples with smaller errors), their influence is lower and may be temporarily ignored. This mechanism dynamically adjusts the sample weights so that the method continuously pays attention to the most challenging or valuable samples in the current data stream during training, avoiding irrelevant or easily predicted samples from occupying too many computing resources. Especially when the data stream changes suddenly or the prediction error of the method is large, the dynamic recalibration mechanism can effectively enhance the flexibility and adaptability of sample selection.

[0083] Overall process

[0084] 1. Input data stream: a series of time series sample streams {x1, x2, ..., x t}, each sample x t Each of them carries certain time information and possible labels (in the case of supervised learning).

[0085] 2. Variational inference: For each time step t, based on historical data (i.e. {x1,x2,…,x t-1}) and the current sample x t , calculate the sample x through variational inference t The probability of selection If the probability exceeds a preset threshold, the sample is considered to have a greater contribution to the current learning task and should be selected for learning.

[0086] 3. Dynamic sample recalibration: For each selected sample x t , first calculate its prediction error through the model Then adjust the sample weight w according to the error t , that is, through the formula:

[0087]

[0088] The larger the weight, the more important the sample is to model training.

[0089] 4. Sample selection: Based on the selection probability calculated by variational inference and the dynamically adjusted weights, the most representative samples at the current moment are selected to form the current learning batch And use it for subsequent incremental learning.

[0090] 5. Model update: The model updates its parameters through an incremental learning algorithm, using the currently selected sample batch 6. Loop execution: With the new sample flow x t+1 With the arrival of the new feature, the whole process continues, and the variational inference and sample recalibration mechanisms are dynamically adjusted according to the changes in the data stream to continuously optimize sample selection and model training.

[0091] By introducing variational inference and dynamic sample recalibration mechanisms, this technical solution makes sample selection not only rely on simple distance metrics or threshold judgments, but also adaptively adjusts according to changes in data streams. Variational inference provides a probabilistic framework to intelligently determine the selection priority of samples, while the dynamic recalibration mechanism dynamically adjusts the importance of samples based on their prediction errors, ensuring that the model prioritizes learning those samples that are most critical to the current task. This method can flexibly respond to changes in data streams in online learning and intermittent learning scenarios, reduce unnecessary computational overhead, and improve learning effects and resource utilization efficiency.

[0092] Step 2: Build an online Bi-GRU model based on the fused attention mechanism and AdamW optimizer.

[0093] This technical solution proposes an innovative online Bi-GRU model that combines the fusion attention mechanism and the AdamW optimizer to improve the accuracy of time series data prediction and the efficiency of training. The model uses the fusion attention mechanism to strengthen the representation of key moments by integrating input features and hidden state information, and introduces the AdamW optimizer for regularization to avoid overfitting and accelerate the convergence of the model.

[0094] The current model uses the energy count time series data selected in step 1. Based on the built-in counter, the energy count of the edge device at each time step is counted. The current model predicts the energy that can be obtained in the future time step.

[0095] First, for the time series data {x1,x2,…,x T}Standardization and normalization (T represents a total of T time steps of samples selected) are performed to ensure that the data is within an appropriate range, which helps to improve the training efficiency and stability of the model. Assume x t It is the input feature at the tth moment. After processing, the data can be used as the input of the Bi-GRU model.

[0096] Bi-GRU is the core part of the model, which is used to encode the input time series data. Bi-GRU can capture the past and future information in the time series data at the same time through the two GRU units, forward and reverse. For each time step t, Bi-GRU generates two hidden states: one from the forward GRU and the other from the reverse GRU.

[0097]

[0098] Among them, x t is the time series data at time step t, refers to the hidden state generated by the forward GRU, Refers to the hidden state produced by the Inverse GRU.

[0099] The output of the Bi-GRU is the concatenation (or weighting) of these two hidden states, representing the information at each time step:

[0100]

[0101] Based on the Bi-GRU output, a fusion attention mechanism is used to further process time series data. This mechanism fuses the input features x at each time step. tAnd the corresponding Bi-GRU output h t Generate fusion feature f t , and calculate the attention weight α for each time step t .

[0102] 1. Fusion operation: input feature x at each time step t and Bi-GRU output h t are concatenated and passed through a fully connected layer for nonlinear transformation:

[0103] f t =W f [x t ;h t ]+b f

[0104] Among them, W f and b f is the parameter to be learned, and the concatenated feature vector f t Contains the input data and the timing information extracted by Bi-GRU.

[0105] 2. Calculate attention weight: based on fusion feature f t , the importance of each time step is calculated through the attention mechanism. Attention weight α t Calculated by:

[0106]

[0107] Among them, score(f t ,W att ) is the fusion feature f t and learnable weights W att The linear transformation of can usually be expressed as:

[0108] score(f t ,W att ) = f t W att +b att

[0109] 3. Weighted summation: According to the attention weight α t , perform weighted summation on the fusion features of all time steps to obtain the final time series representation h att :

[0110]

[0111] The weighted time series representation h att It is input to the prediction layer for final prediction. The prediction layer is usually a fully connected layer that outputs the prediction result.

[0112]

[0113] Among them, W out and b out are the weights and biases of the fully connected layer, is the final predicted value.

[0114] In order to optimize the model parameters and avoid overfitting, this model uses the AdamW optimizer. The AdamW optimizer is an improved version of Adam. It effectively prevents the model from overfitting during training by introducing a weight decay term (L2 regularization). The update formula of the AdamW optimizer is as follows:

[0115]

[0116] in:

[0117] θ t Represents the online Bi-GRU model parameters;

[0118] Represents the mean of the gradient. It is the weighted average of the current gradient, and gradually approaches the true mean of the gradient as the training progresses. Represents the variance of the gradient. It is the weighted average of the squared gradient, which gradually approaches the true gradient variance as the training progresses.

[0119] ∈ is a smoothing term used to prevent division by zero errors;

[0120] λ is the weight decay coefficient, which controls the strength of regularization;

[0121] η is the learning rate;

[0122] The AdamW optimizer helps accelerate model training and improve generalization capabilities by adaptively adjusting the learning rate of each parameter and introducing weight decay during the update process.

[0123] During model training, the mean square error loss function is used to calculate the predicted value. The error between the actual value y. The specific loss function is calculated as follows:

[0124]

[0125] Where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value. The MSE loss function measures the difference between the predicted value and the true value, and the training process aims to minimize this loss.

[0126] The model proceeds as follows:

[0127] 1. Input data preprocessing: Standardize and normalize the selected time series data samples to ensure that the data is within a reasonable range.

[0128] 2. Bi-GRU encoding: Input the processed time series data into the Bi-GRU module to obtain the forward and reverse hidden states of each time step.

[0129] 3. Fusion attention mechanism: By fusing the input features and Bi-GRU output features, calculating the attention weights, and performing weighted summation on each time step, a global temporal representation is obtained.

[0130] 4. Prediction: Input the weighted time series representation into the fully connected prediction layer and output the final prediction result.

[0131] 5. Optimization training: Use AdamW optimizer to optimize model parameters and train the model by minimizing the MSE loss function. 6. Weight update: In each iteration, AdamW optimizer updates the model parameters based on the calculated gradient and regularization term.

[0132] By introducing the fusion attention mechanism, the model can comprehensively consider the input features of each time step and the hidden state of Bi-GRU, so that the model can automatically focus on the most important moments for prediction during the training process. At the same time, the use of AdamW optimizer can effectively reduce the risk of overfitting and accelerate the training process of the model.

[0133] According to the characteristics of Bi-GRU, the entire model learning process is divided into a feedforward layer, a back propagation layer, a feature fusion layer based on time attention, and a prediction optimization layer. When a learning action is decided to be initiated, it is carried out layer by layer in an intermittent manner to complete a learning cycle.

[0134] Then, in the present invention, the entire learning process can be carried out according to the neural network layer {l1,l2,...,l m}Execute forward (feedforward layer) in sequence, then execute backward (back propagation layer), feature fusion layer, and prediction optimization layer to complete a learning cycle. The input of the feedforward layer is time series data, and the output is hidden state. The feedforward layer is mainly responsible for extracting preliminary features from the input data, performing nonlinear transformation, and thus generating hidden representations. The input of the back propagation layer is the predicted output of the model, the true label, and the hidden state generated by the feedforward layer. The output is the updated model parameters. The goal of the back propagation layer is to optimize the weights of the model based on the loss function. The input of the feature fusion layer is the transformed content vector of the hidden state and the input feature data. The output is the fused time series representation, which is used for prediction tasks. In the feature fusion layer, the model uses the temporal attention mechanism to perform a weighted combination of the hidden state and other features generated by the feedforward layer to generate the final global feature representation.

[0135] The energy required for all actions of the edge device is statistically summarized, and the amount of energy required for each action is defined as an energy level and sorted. Only when the current energy level is higher than the energy requirement of the action, the entire passive sensing system will select the next action that can be performed.

[0136] Step 3: Construct a limited decision domain model based on GNN and PPO to solve the problem of action state explosion.

[0137] Process the predicted future time series data and count the energy level of the system at each time step. Divide the actions that the system can take into perception data, a layer defined by the model in step 2, and waiting for the next time step. Perception data refers to the actual task data that the edge device perceives to be read, and waiting for the next time step means that in order to ensure the coherent execution of subsequent actions, it may be necessary not to perform any actions in the current time step to accumulate energy. Define the energy level and energy required for each action, model it through graph neural networks, find all action sequences within a limited range, and finally make decisions through the trained PPO.

[0138] First, the system defines a finite state space. Specifically, the state space consists of the following elements:

[0139] Energy level: Discretize the energy state of the system into multiple levels, such as low, medium, and high, corresponding to different ranges of action selection.

[0140] Data perception point: Data perception point is defined as a set of time nodes that the system can perceive. For example, data perception can be performed only at critical moments to reduce unnecessary calculations.

[0141] Action set: An action set consists of multiple optional actions, such as sensing the data required for the task, executing a layer defined by the model in step 2, and waiting for the next time step.

[0142] In a finite state space, the state may still be high-dimensional. In order to further reduce the computational complexity, we use graph neural networks to reduce the dimensionality of the state. The state space is modeled as a graph, where nodes represent states and edges represent the transition relationship between states. The graph neural network learns the low-dimensional representation of each state node through graph convolution operations.

[0143] The state representation processed by the graph neural network is input into the subsequent reinforcement learning model as the basis for decision making.

[0144] In the limited decision domain after dimensionality reduction, we use the PPO (Proximal Policy Optimization) algorithm for policy optimization. PPO is a reinforcement learning algorithm based on the policy gradient method. It ensures the stability of policy updates through the clipping policy update method. The goal of PPO is to maximize the following objective function:

[0145]

[0146] in:

[0147] r t (θ) is the ratio of the new policy to the old policy:

[0148]

[0149] is the advantage function, which is calculated by the generalized advantage estimation (GAE) and is used to estimate the advantage of the current action.

[0150] The role of the clip function in PPO is to limit the policy ratio to an acceptable range, prevent the policy update from being too large or too small, and ensure the stability of training.

[0151] ∈ is a small hyperparameter that controls the range of policy updates.

[0152] E t is the expected symbol, which represents the expected operation for all time steps t.

[0153] L CLIP (θ) is the clipped objective function of PPO, which is used to measure the difference between the current policy and the old policy and optimize the difference. The purpose is to prevent over-updating the policy each time the policy is updated, and to ensure the stability of the policy update through the clipping policy update method.

[0154] π θ (a t |s t ) is the probability distribution of the current strategy, indicating that in state s t Next, take action a t This probability is calculated by constructing a Gaussian distribution using the mean and standard deviation of the PPO policy network output;

[0155] The significance of this objective function is that if the new strategy is better than the old strategy, the update amplitude will be increased; if it is not good, the update amplitude will be limited through clipping operations to avoid excessive strategy changes.

[0156] Specific process

[0157] 1. Initialize the state space: Initialize the limited state space according to the energy level, data sensing point and action set to ensure that the size of the state space is controllable.

[0158] 2. Build a graph structure: Model the initialized state space as a graph structure, where a node is a state, corresponding to a set of energy levels, actions, etc., and an edge is a conversion relationship between states.

[0159] 3. Use GNN for dimensionality reduction: Reduce the dimensionality of the state space through graph neural network to obtain a low-dimensional representation of each state.

[0160] 4. PPO reinforcement learning training: Use the reduced-dimensional state representation combined with the PPO algorithm for policy optimization. During the training process, the policy algorithm and edge devices interact with the environment based on the currently selected policy and update the policy to maximize the cumulative reward. The algorithm is based on the current state s t Execute action a t , and then returns the immediate reward r t and the next state s t+1 , according to the immediate reward r t To update its strategy. Specifically, PPO maximizes the long-term cumulative reward by optimizing the objective function.

[0161] 5. Strategy application: After sufficient training, use the final strategy to make decisions in the environment and choose the optimal action path.

[0162] Step 4: Use the above time series data prediction model and decision domain to predict time series data and select the best action sequence.

[0163] On the other hand, an embodiment of the present invention also provides an intermittent learning system for time series data based on temporal attention, which is characterized by: comprising a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a method for intermittent learning of time series data based on temporal attention as described in the above technical solution.

[0164] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A time series data intermittent learning method based on temporal attention, characterized in that: The steps include: Step 1, obtain the amount of energy obtained by the passive IoT edge device from the outside world at each time step, that is, obtain energy time series data; Step 2: Sample selection is performed for energy time series data. The most representative samples are automatically selected from the energy time series data through adaptive batch selection and dynamic sample recalibration methods guided by variational inference. Step 3: Build an online multi-layer Bi-GRU model based on the fusion attention mechanism and AdamW optimizer, input the energy number time series data sample in step 2, and output the energy number that can be obtained in the future time step; Step 4: Define the energy level according to the amount of energy obtained, as well as the energy level required for each action of the passive IoT edge device at each time step, and model it through a graph neural network, where the nodes in the graph structure represent the action states corresponding to the energy levels, and the edges are the transition relationships between states. The final proximal policy optimization algorithm PPO is used to find all action sequences within a limited range and select the best action sequence.

2. The time series data intermittent learning method based on temporal attention as claimed in claim 1, characterized in that: The implementation of the variational inference guided adaptive batch selection method in step 2 is as follows; For each new sample x in the data stream t , based on historical data Perform variational inference to estimate the contribution of the sample to the current learning task. The goal of variational inference is to infer the selection probability of the sample by maximizing the lower bound of evidence. The specific inference process is as follows: in, For sample x t Given historical data The probability of selection under, q(θ) is the variational distribution, θ is the parameter, E represents the expectation, which is used to calculate the lower bound of evidence in variational inference; In practice, variational inference is performed for each sample x t Calculate a selection probability And decide whether the sample is selected based on the probability.

3. The time series data intermittent learning method based on temporal attention as claimed in claim 1, characterized in that: The implementation of the dynamic sample recalibration method in step 2 is as follows; For each time step t, let For the model to sample x t The predicted value, y t is the true label, and the prediction error of the sample is Based on the error, the weight w of the sample t Dynamically updated via the following formula: Among them, λ is a hyperparameter that controls the weight update.

4. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: The specific implementation of step 2 is as follows; Step 2.1: Input a series of time series sample streams {x1, x2, …, x t }, each sample x t They all carry certain time information and possible tags; Step 2.2, variational inference: For each time step t, based on historical data That is, {x1,x2,…,x t-1 }) and the current sample x t , calculate the sample x through variational inference t The probability of selection If the probability exceeds a certain preset threshold, it is considered that the sample has a greater contribution to the current learning task and should be selected for learning; Step 2.3, dynamic sample recalibration: For each selected sample x t , first by calculating its prediction error in For the model to sample x t The predicted value, y t is the true label; then adjust the sample weight w according to the error t , that is, through the formula: Step 2.4, sample selection: Based on the selection probability calculated by variational inference and the dynamically adjusted weights, the most representative samples at the current moment are selected to form the current learning batch. And use it for subsequent incremental learning; Step 2.5, update the hyperparameter λ through the incremental learning algorithm, using the currently selected sample batch Conduct training; Step 2.6, loop execution: With the new sample stream x t+1 With the arrival of the new feature, the whole process continues, and the variational inference and sample recalibration mechanisms are dynamically adjusted according to the changes in the data stream to continuously optimize sample selection and model training.

5. The time series data intermittent learning method based on temporal attention as claimed in claim 1, characterized in that: In step 3, the specific processing process of the online multi-layer Bi-GRU model is as follows: Step 3.1, input data preprocessing: select the time series data samples {x1, x2, …, x T }Standardization and normalization are performed to ensure that the data is within a reasonable range; where T represents the selection of samples of T time steps; Step 3.2, Bi-GRU encoding: Input the processed time series data into the Bi-GRU module to obtain the forward and reverse hidden states of each time step, and concatenate the two hidden states as the output; Step 3.3, fusion attention mechanism: by fusing the input features and Bi-GRU output features, calculating the attention weights, and performing weighted summation on each time step, a global temporal representation is obtained; Step 3.4, prediction: input the weighted time series representation into the fully connected prediction layer and output the final prediction result; Step 3.5, optimization training: Use AdamW optimizer to update the optimization model parameters and train the model by minimizing the MSE loss function; The update formula of the AdamW optimizer is as follows: Where: θ t represents the online Bi-GRU model parameters, represents the mean value of the gradient, represents the variance of the gradient, ∈ is a smoothing term used to prevent zero division errors; λ is the weight decay coefficient, which controls the strength of regularization; η is the learning rate; The specific loss function L is calculated as follows: Where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value; Step 3.6, Weight Update: In each iteration, the AdamW optimizer updates the model parameters based on the calculated gradients and the regularization term.

6. The time series data intermittent learning method based on temporal attention as claimed in claim 5, characterized in that: The specific implementation of step 3.2 is as follows: For each time step t, Bi-GRU generates two hidden states: one from the forward GRU, i.e., GRU forward , and the other comes from the reverse GRU, i.e. GRU backward ; Among them, x t is the time series data at time step t, refers to the hidden state generated by the forward GRU, Refers to the hidden state generated by the reverse GRU; The output of Bi-GRU is the concatenation of these two hidden states, representing the information at each time step:

7. The time series data intermittent learning method based on temporal attention as claimed in claim 5, characterized in that: The specific implementation of step 3.3 is as follows: The input features x at each time step t and Bi-GRU output h t are concatenated and passed through a fully connected layer for nonlinear transformation: f t =W f [x t ;h t ]+b f Among them, W f and b f is the parameter to be learned, and the concatenated feature vector f t Contains the input data and the timing information extracted by Bi-GRU; Based on the fusion feature f t , the importance of each time step is calculated through the attention mechanism, and the attention weight α t Calculated by: Among them, score(f t ,W att ) is the fusion feature f t and learnable weights W att The linear transformation can be expressed as: score(f t ,W att )=f t W att +b att According to the attention weight α t , perform weighted summation on the fusion features of all time steps to obtain the final time series representation h att :

8. The time series data intermittent learning method based on temporal attention as claimed in claim 1, characterized in that: In step 4, the actions taken by the passive IoT edge device include sensing data, executing a layer defined by the model in step 3, and waiting for the next time step, where the sensed data refers to the actual task data that the passive IoT edge device senses and is supposed to read; the Bi-GRU model learning process is divided into a feedforward layer, a back propagation layer, a feature fusion layer based on time attention, and a prediction optimization layer. The feedforward layer is used to extract preliminary features from the input data and perform nonlinear transformations to generate hidden representations. The input of the back propagation layer is the predicted output of the model, the true label, and the hidden state generated by the feedforward layer. The output is the updated model parameters. The goal of the back propagation layer is to optimize the weights of the model based on the loss function. The input of the feature fusion layer is the transformed content vector of the hidden state and input feature data, and the output is the fused time series representation, which is used for prediction tasks. In the feature fusion layer, the model uses the temporal attention mechanism to perform a weighted combination of the hidden state and other features generated by the feedforward layer to generate the final global feature representation. Waiting for the next time step means that no action is performed in the current time step to accumulate energy for the coherent execution of subsequent actions.

9. The time series data intermittent learning method based on temporal attention as claimed in claim 1, characterized in that: The goal of PPO is to maximize the following objective function: in: r t (θ′) is the ratio of the new strategy to the old strategy: In the formula, is the advantage function, which is calculated by generalized advantage estimation and is used to estimate the advantage of the current action; ∈ is a small hyperparameter that controls the range of policy updates; E t is the expected symbol, which represents the expected operation for all time steps t. The clip function is used to limit the strategy ratio to an acceptable range. CLIP (θ′) is the clipping objective function of PPO, which is used to measure the difference between the current strategy and the old strategy; π θ′ (a t |s t ) is the probability distribution of the current strategy, indicating that in state s t Next, take action a t The probability of is the probability distribution of the old strategy.

10. A time series data intermittent learning system based on temporal attention, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a time series data intermittent learning method based on time attention as described in any one of claims 1 to 9.

Images