A time attention-based time series data intermittent learning method and system

Abstract

The present application relates to the fields of passive Internet of Things, artificial intelligence and the like, in particular to a time attention-based time series data intermittent learning method and system. For the time series data processing problem in the passive sensing monitoring environment, the method adopts the variational inference guided adaptive batch selection and dynamic sample re-labeling method to select the samples meeting the conditions, then combines the Bi-GRU algorithm, carries out feature fusion based on the time attention mechanism, considers the execution sequence of different modules in the machine learning task, divides the whole model learning process into the feedforward layer, the back propagation layer and the feature fusion layer for hierarchical learning, and proposes a time attention-based time series data intermittent learning prediction mechanism to ensure the accuracy of the time series data prediction and find the local optimal solution on a limited decision domain to solve the action state explosion problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of passive Internet of Things, artificial intelligence, etc., and in particular to a time attention-based time series data intermittent learning method. BACKGROUND

[0002] Passive Internet of Things edge devices continuously obtain energy from the surrounding environment in different ways to maintain their basic functions, thereby completely eliminating the dependence on batteries, but the energy will be interrupted. Therefore, the execution of machine learning is intermittent, and an optimal action (for example, reading sensors, learning and classification) is usually selected dynamically at system runtime to monitor passive sensors in order to achieve the overall goal. Battery-free intermittent computing is fundamentally different from traditional computing, because intermittent computing devices violate one of the most basic assumptions of stable power supply. In addition, systems that pause and resume execution based on captured energy have a higher risk of losing the events of interest that they want to detect or learn. Since the acquisition of energy and the occurrence of events of interest are intermittent and uncertain, the edge device intermittent learning problem is a key scientific problem faced by passive sensor monitoring.

[0003] In practical passive sensor monitoring applications, a large amount of time series data can usually be obtained, and related machine learning algorithms (such as LSTM) are used to learn models based on historical observations to accurately predict the future. However, the time patterns hidden in historical information play an important role in accurate prediction of long time series, and dynamic future information, such as calendar events, can have a strong impact on the prediction model. In addition, since energy needs to be obtained from the environment in a passive sensor monitoring environment, the power supply system will be interrupted, so machine learning in this environment is an intermittent learning that needs to dynamically select an optimal operation (for example, reading sensors, learning and classification) at runtime, so that its overall goal (i.e., the number and type of samples required to complete learning) is optimal. Therefore, how to ensure the accuracy of time series prediction model for intermittent learning in a passive sensor monitoring environment has become an important problem. SUMMARY

[0004] For time series data processing problems in a passive sensor monitoring environment, a variational inference guided adaptive batch selection and dynamic sample reweighting method is used to select samples that meet the conditions, and then combined with a Bi-GRU algorithm, a time attention mechanism is used for feature fusion, and the execution order of different modules in the machine learning task is considered. The entire model learning process is divided into a feedforward layer, a backpropagation layer, and a feature fusion layer for hierarchical learning. A time attention-based time series data intermittent learning prediction mechanism is proposed to ensure the accuracy of time series data prediction, and to find a local optimal solution on a limited decision domain to solve the action state explosion problem.

[0005] To achieve the above object, the technical scheme provided by the application is a time attention-based intermittent learning method for time series data, comprising the following steps:

[0006] Step 1, obtaining the energy number obtained from the outside world by each time step of the passive Internet of Things edge device, that is, obtaining the energy time series data;

[0007] Step 2, sample selection is performed on the energy time series data, and the most representative samples are automatically selected from the energy time series data by a variational inference guided adaptive batch selection and dynamic sample re-labeling method;

[0008] Step 3, an online multi-layer Bi-GRU model based on a fusion attention mechanism and an AdamW optimizer is constructed, the energy number time series data samples in step 2 are input, and the energy number that can be obtained at a future time step is output;

[0009] Step 4, the energy level is defined according to the obtained energy number, and the energy level required by the passive Internet of Things edge device for each action at each time step is modeled by a graph neural network, wherein the nodes in the graph structure represent the action states corresponding to the energy level, and the edges represent the transition relationship between the states, and all action sequences within a limited range are found by a final proximal policy optimization algorithm PPO, and the best action sequence is selected.

[0010] Further, the implementation of the variational inference guided adaptive batch selection method in step 2 is as follows:

[0011] For each new sample x t in each data stream, variational inference is performed based on historical data to estimate the contribution of the sample to the current learning task, and the goal of variational inference is to infer the selection probability of the sample by maximizing the lower bound of the evidence, and the specific inference process is as follows:

[0012]

[0013] wherein, is the selection probability of the sample x t under the given historical data , q(θ) is the variational distribution, θ is the parameter, and E represents the expectation, which is used to calculate the lower bound of the evidence in variational inference;

[0014] In actual operation, the variational inference calculates a selection probability for each sample x t , and decides whether the sample is selected according to the probability.

[0015] Further, the implementation of the dynamic sample re-labeling method in step 2 is as follows:

[0016] For each time step t, let be the prediction of the model on the sample x t , y t be the true label, and the prediction error of the sample be Based on the error, the weight w t of the sample is dynamically updated by the following formula:

[0017]

[0018] where λ is a hyperparameter that controls the weight update.

[0019] Further, the specific implementation of step 2 is as follows:

[0020] Step 2.1, input a series of time series sample streams {x1, x2, …, x t}, each sample x t carries certain time information and possible labels;

[0021] Step 2.2, variational inference: for each time step t, based on the historical data i.e. {x1, x2, …, x t-1}) and the current sample x t , calculate the selection probability t of the sample x t through variational inference; If the probability exceeds a certain preset threshold, it is considered that the sample has a greater contribution to the current learning task and should be selected for learning;

[0022] Step 2.3, dynamic sample reweighting: for each selected sample x t , first calculate its prediction error where is the prediction of the model on the sample x t , y t is the true label; then adjust the weight w t of the sample according to the error, i.e. through the formula:

[0023]

[0024] Step 2.4, sample selection: based on the selection probability calculated through variational inference and the dynamically adjusted weight, select the most representative sample at the current time to form the current learning batch and use it for subsequent incremental learning;

[0025] Step 2.5, update the hyperparameter λ through the incremental learning algorithm, and use the currently selected sample batch for training;

[0026] Step 2.6, loop execution: as new sample stream x t+1 arrives, the whole process continues, and the variational inference and sample reweighting mechanism dynamically adjusts according to the changes in the data stream, continuously optimizing sample selection and model training.

[0027] Further, in step 3, the specific processing process of the online multi-layer Bi-GRU model is as follows:

[0028] Step 3.1, input data preprocessing: standardize and normalize the selected time series data samples {x1, x2, …, x T T} to ensure that the data is within a reasonable range; where T represents the selected T time step samples;

[0029] Step 3.2, Bi-GRU encoding: input the processed time series data into the Bi-GRU module to obtain the forward and reverse hidden states of each time step, and concatenate the two hidden states as the output;

[0030] Step 3.3, fusion attention mechanism: fuse the input features and Bi-GRU output features, calculate the attention weights, and perform weighted summation on each time step to obtain the global time series representation;

[0031] Step 3.4, prediction: input the weighted time series representation into the fully connected prediction layer to output the final prediction result;

[0032] Step 3.5, optimization training: use the AdamW optimizer to update the optimization model parameters, and train the model by minimizing the MSE loss function;

[0033] The update formula of the AdamW optimizer is as follows:

[0034]

[0035] where: θ t represents the online Bi-GRU model parameters, represents the mean of the gradient, represents the variance of the gradient, ∈ is a smoothing term to prevent division by zero errors; λ is the weight decay coefficient, which controls the strength of regularization; η is the learning rate;

[0036] The specific loss function L is calculated as follows:

[0037]

[0038] where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value;

[0039] Step 3.6, weight update: In each iteration, the AdamW optimizer updates the model parameters according to the calculated gradient and regularization term.

[0040] Further, the specific implementation of step 3.2 is as follows:

[0041] For each time step t, the Bi-GRU generates two hidden states: one from the forward GRU, i.e., GRU forward , and the other from the reverse GRU, i.e., GRU backward .

[0042]

[0043] where x t is the time series data at time step t, is the hidden state generated by the forward GRU, is the hidden state generated by the reverse GRU;

[0044] The output of the Bi-GRU is the concatenation of these two hidden states, representing the information at each time step:

[0045]

[0046] Further, the specific implementation of step 3.3 is as follows:

[0047] The input features x t and Bi-GRU output h t at each time step are concatenated and nonlinearly transformed through a fully connected layer:

[0048] f t = W f [x t ; h t ] + b f

[0049] where W f and b f are parameters to be learned, and the concatenated feature vector f t contains both the input data and the time series information extracted by the Bi-GRU;

[0050] Based on the fusion features f t , the importance of each time step is calculated through the attention mechanism, and the attention weight a t is calculated as follows:

[0051]

[0052] where score(f t , W att ) is the score function of the fusion features ft and learnable weights W att , can be represented as:

[0053] score(f t ,W att )=f t W att +b att

[0054] According to the attention weight α t , the fused features of all time steps are weighted and summed to obtain the final time sequence representation h att :

[0055]

[0056] Further, in step 4, the actions taken by the passive Internet of Things edge device include sensing data, executing a certain layer defined by the model in step 3, and waiting for the next time step, wherein the sensing data refers to the passive Internet of Things edge device sensing the actual task data to be read; the Bi-GRU model learning process includes a feedforward layer, a backpropagation layer, a feature fusion layer based on time attention, and a prediction optimization layer, the feedforward layer is used to extract preliminary features from input data, perform nonlinear transformation, and generate hidden representations, the backpropagation layer inputs are the prediction output of the model, the real label and the hidden state generated by the feedforward layer, and the output is the updated model parameters, the goal of the backpropagation layer is to optimize the weights of the model based on the loss function; the feature fusion layer inputs are the hidden state and the content vector obtained by transforming the input feature data, and the output is the fused time sequence representation used for prediction, in the feature fusion layer, the model combines the hidden state generated by the feedforward layer and other features by time attention mechanism to generate the final global feature representation; waiting for the next time step means that the current time step does not perform any action to accumulate energy for the subsequent consecutive execution of actions.

[0057] Further, the goal of PPO is to maximize the following objective function:

[0058]

[0059] wherein:

[0060] r t (θ′) is the ratio of the new policy to the old policy:

[0061]

[0062] In the formula, is the advantage function, which is calculated by generalized advantage estimation, and is used to estimate the advantage of the current action; ∈ is a small hyperparameter that controls the range of policy update; Et is the expected symbol, representing the expected operation for all time steps t, and the clip function is used to limit the policy ratio within an acceptable range; L CLIP (θ') is the clipped objective function of PPO, used to measure the difference between the current policy and the old policy; π θ′ (a t |s t ) is the probability distribution of the current policy, representing the probability of taking action a t in state s t , is the probability distribution of the old policy.

[0063] The application also provides a time attention-based intermittent learning system for time series data, characterized by comprising a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute the time attention-based intermittent learning method for time series data according to the above technical solution.

[0064] Compared with the prior art, the application has the following advantages and beneficial effects:

[0065] The application considers the utility of perception data and the execution order of different modules in machine learning tasks, combines time attention, uses the Bi-GRU algorithm and the intermittent learning method, and proposes a time attention-based passive sensing data intermittent learning mechanism to ensure the accuracy of the learning model. The main innovation is that feature fusion is performed based on the time attention mechanism, the execution order of different modules in machine learning tasks is considered, the entire model learning process is divided into a feedforward layer, a backpropagation layer and a feature fusion layer for hierarchical learning, so as to ensure the performance of the time series data learning model in the passive sensing environment, and find a local optimal solution on a limited decision domain to solve the action state explosion problem. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 The flowchart of the embodiment of the application. DETAILED DESCRIPTION

[0067] The technical solutions of the application will be further described below in combination with the drawings and embodiments.

[0068] For the intermittent learning and prediction problem of time series data in passive sensing monitoring, the application will solve it from the following two aspects.

[0069] In one aspect, from the perspective of time series data prediction, the hidden pattern representation is explicitly learned by Bi-GRU, propagating future information in both forward and backward directions, considering dynamic future information such as calendar events. In addition, the potential patterns in historical data can be better captured by using time attention mechanism, focusing on different parts of history to predict the future, that is, through a multi-modal fusion mechanism for combining features of different parts of history to better represent the future. However, this method does not consider the intermittent learning problem in passive sensing monitoring to ensure effective model learning, and also does not consider the sample selection problem to reduce computational overhead.

[0070] On the other hand, from the perspective of intermittent learning, the goal of intermittent learning is to enable edge devices that acquire energy to effectively and efficiently perform certain types of machine learning tasks, enabling intermittent learners to learn about machine learning data and application semantic tasks, for which heuristic algorithms are designed to determine a series of actions to achieve the expected learning goal under energy constraints, helping intermittent learners to decide whether to learn or discard training instances at runtime, which can increase the energy efficiency of the system.

[0071] As shown in Figure 1 , the present application provides a time attention-based time series data intermittent learning method, comprising the following steps:

[0072] Step 1: automatically select the most representative samples from time series data, through variational inference guided adaptive batch selection and dynamic sample reweighting, the model can intelligently determine which samples are most critical to the learning task and select these samples for training.

[0073] The time series data here refers to the amount of energy that each time step can obtain from the outside world in a passive environment, and sample selection is performed on the energy time series data.

[0074] In online learning, pre-defined criteria (such as fixed thresholds or similarity measures) are often used to select samples. Traditional methods often ignore the potential structure and dynamic changes of data streams, and this scheme adaptively selects samples by introducing variational inference combined with historical information of data streams.

[0075] For each new sample x t in a data stream, we perform variational inference based on historical data to estimate the contribution of the sample to the current learning task. The goal of variational inference is to infer the selection probability of the sample by maximizing the evidence lower bound (ELBO). The specific inference process is as follows:

[0076]

[0077] where, For sample x t Given the selection probability of sample x under historical data, q(θ) is the variational distribution, θ is the parameter of the sample selection method, which refers to all the parameters that need to be optimized in the learning process, E represents the expectation, and is used to calculate the lower bound of evidence in variational inference, which is obtained by integrating the parameter θ with respect to the variational distribution q(θ).

[0078] In actual operation, the variational inference calculates a selection probability for each sample x t and decides whether the sample is selected according to the probability. This selection process depends on the importance of the sample in the data stream relative to the historical data, thereby achieving dynamic sample selection based on the characteristics of the data stream.

[0079] To improve the effectiveness of selecting samples, we introduce a dynamic sample reweighting mechanism to make the learning process more focused on important samples by adjusting the weight of each sample in real time. This mechanism dynamically weights each sample based on its prediction error, so that the model pays more attention to samples that have a greater impact on prediction.

[0080] For each time step t, let be the predicted value of the method for sample x t , y t be the true label, and the prediction error of the sample be Based on the error, the weight w t of the sample is dynamically updated by the following formula:

[0081]

[0082] where λ is a hyperparameter that controls the weight update, usually with a small value (e.g. 0.1 or 0.01), controlling the sensitivity of the error to the weight. The larger the error ∈ t , the less accurate the model's prediction on that sample, and therefore the more important the sample, the larger the weight w t . Conversely, if the prediction error is small, the weight will decrease, thereby reducing the impact of the sample on the method. For samples with larger weights (i.e. samples with larger errors), they will be given higher priority in the training process, and the model will pay more attention to these samples. For samples with smaller weights (i.e. samples with smaller errors), their influence is low and they may be temporarily ignored. This mechanism dynamically adjusts the sample weight, so that the method constantly focuses on the most challenging or most valuable samples in the current data stream during the training process, avoiding irrelevant or easily predictable samples from occupying too much computing resources. In particular, when the data stream changes suddenly or the prediction error of the method is large, the dynamic reweighting mechanism can effectively enhance the flexibility and adaptability of sample selection.

[0083] Overall process​

[0084] 1. Input data stream: a series of time-series sample streams {x1, x2, …, x t}, each sample x t carrying certain time information and possibly a label (in supervised learning case).

[0085] 2. Variational inference: for each time step t, based on historical data (i.e. {x1, x2, …, x t-1}) and current sample x t , compute the selection probability t of sample x If the probability exceeds a certain preset threshold, it is considered that the sample has a greater contribution to the current learning task and should be selected for learning.

[0086] 3. Dynamic sample reweighting: for each selected sample x t , first compute its prediction error through the model, then adjust the weight w t of the sample according to the error, i.e. through the formula:

[0087]

[0088] The greater the weight, the more important the sample is to model training.

[0089] 4. Sample selection: based on the selection probability calculated by variational inference and the dynamically adjusted weight, select the most representative sample at the current time to form the current learning batch and use it for subsequent incremental learning.

[0090] 5. Model update: the model updates its parameters through incremental learning algorithm and trains using the current selected sample batch 6. Loop execution: as new sample stream x t+1 arrives, the whole process continues, and the variational inference and sample reweighting mechanism dynamically adjust according to the changes in the data stream, continuously optimizing sample selection and model training.

[0091] By introducing variational inference and dynamic sample reweighting mechanism, this technical solution makes sample selection not only rely on simple distance measurement or threshold judgment, but also adaptively adjust according to the changes in data stream. Variational inference provides a probability framework to intelligently judge the selection priority of samples, while the dynamic reweighting mechanism dynamically adjusts the importance of samples according to their prediction errors, ensuring that the model learns those samples that are most critical to the current task first. This method can flexibly cope with changes in data stream in online learning and intermittent learning scenarios, reduce unnecessary computational overhead, and improve learning effectiveness and resource utilization efficiency.

[0092] Step 2: Constructing an online Bi-GRU model based on the fusion attention mechanism and AdamW optimizer.

[0093] The present technical solution proposes an innovative online Bi-GRU model that combines the fusion attention mechanism and AdamW optimizer, aiming to improve the accuracy of time series data prediction and the efficiency of training. The model integrates input features and hidden state information, uses the fusion attention mechanism to enhance the representation of key moments, and introduces the AdamW optimizer for regularization to avoid overfitting and accelerate model convergence.

[0094] The current model applies the energy number time series data selected in Step 1, and according to the built-in counter, it counts the energy number of the edge device at each time step when it is turned on and off. The current model predicts the energy number that can be obtained at the future time step.

[0095] First, the time series data {x1, x2, …, xT} selected in Step 1 is standardized and normalized (T represents a total of T time steps of samples), to ensure that the data is within a suitable range, which helps to improve the training efficiency and stability of the model. Assume x T is the input feature at the t-th time step, after processing, the data can be used as the input of the Bi-GRU model. t

[0096] Bi-GRU is the core part of the model, used to encode the input time series data. Bi-GRU can capture both past and future information in time series data through two forward and backward GRU units. For each time step t, Bi-GRU generates two hidden states: one from the forward GRU and the other from the backward GRU.

[0097]

[0098] where x t is the time series data at time step t, is the hidden state generated by the forward GRU, is the hidden state generated by the backward GRU.

[0099] The output of Bi-GRU is the concatenation (or weighting) result of these two hidden states, representing the information at each time step:

[0100]

[0101] On the basis of Bi-GRU output, a fusion attention mechanism is used to further process the time series data. This mechanism fuses the input feature x t ​and the corresponding Bi-GRU output h t Generate fusion feature f t And calculate the attention weight a of each time step t .

[0102] 1. Fusion operation: input feature x of each time step t And Bi-GRU output h t Be spliced and nonlinearly transformed by a fully connected layer:

[0103] f t = W f [x t ;h t ]+b f

[0104] Where W f And b f Are parameters to be learned, and the spliced feature vector f t Contains input data and Bi-GRU extracted time sequence information.

[0105] 2. Calculate attention weight: based on fusion feature f t , the importance of each time step is calculated through attention mechanism. Attention weight a t Is calculated by:

[0106]

[0107] Where score(f t ,W att ) is a linear transformation of fusion feature f t And learnable weight W att , which can be represented as:

[0108] score(f t ,W att ) = f t W att +b att

[0109] 3. Weighted sum: according to the attention weight a t , the weighted sum of fusion features of all time steps is obtained, and the final time sequence representation h att :

[0110]

[0111] The weighted time sequence representation h att Is input to the prediction layer for final prediction. The prediction layer is usually a fully connected layer, which outputs the prediction result

[0112]

[0113] where W out and b out are the weights and biases of the fully connected layer, is the final prediction.

[0114] To optimize the model parameters and avoid overfitting, this model uses the AdamW optimizer. AdamW optimizer is an improved version of Adam, which effectively prevents overfitting phenomenon in the training process by introducing a weight decay term (L2 regularization). The update formula of AdamW optimizer is as follows:

[0115]

[0116] where:

[0117] θ t represents the online Bi-GRU model parameters;

[0118] represents the mean of the gradient. It is the weighted average of the current gradient, which gradually approaches the true gradient mean as the training progresses. represents the variance of the gradient. It is the weighted average of the square of the gradient, which gradually approaches the true gradient variance as the training progresses.

[0119] ∈ is a smoothing term to prevent division by zero errors;

[0120] λ is the weight decay coefficient, which controls the strength of regularization;

[0121] η is the learning rate;

[0122] AdamW optimizer adjusts the learning rate of each parameter adaptively and introduces weight decay in the update process, which helps to accelerate model training and improve generalization ability.

[0123] In the model training process, the mean square error loss function is used to calculate the error between the predicted value and the true value y. The specific loss function calculation is as follows:

[0124]

[0125] where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value. The MSE loss function measures the difference between the predicted value and the true value, and the training process aims to minimize this loss.

[0126] The model follows the following steps:

[0127] 1. Input data preprocessing: Standardize and normalize the selected time series data samples to ensure that the data is within a reasonable range.

[0128] 2. Bi-GRU encoding: The processed time series data is input into the Bi-GRU module to obtain the forward and reverse hidden states of each time step.

[0129] 3. Fusion attention mechanism: By fusing the input features and Bi-GRU output features, calculating the attention weights, and performing weighted summation on each time step, a global temporal representation is obtained.

[0130] 4. Prediction: The weighted time series representation is input into the fully connected prediction layer and the final prediction result is output.

[0131] 5. Optimization training: The AdamW optimizer is used to optimize the model parameters and train the model by minimizing the MSE loss function. 6. Weight update: In each iteration, the AdamW optimizer updates the model parameters based on the calculated gradient and regularization term.

[0132] By introducing a fused attention mechanism, the model comprehensively considers the input features and the hidden state of the Bi-GRU at each time step, enabling the model to automatically focus on the most important moments for prediction during training. Furthermore, the use of the AdamW optimizer effectively reduces the risk of overfitting and accelerates model training.

[0133] Based on the characteristics of Bi-GRU, the entire model learning process is divided into a feedforward layer, a backpropagation layer, a feature fusion layer based on temporal attention, and a prediction optimization layer. When a learning action is decided to start, it is carried out layer by layer in an intermittent manner, completing a learning cycle.

[0134] Then, in the present invention, the entire learning process can be carried out according to the neural network layer {l1, l2, ..., l m}Execute forward in sequence (feedforward layer), then execute backward (back propagation layer), feature fusion layer, and prediction optimization layer to complete a learning cycle. The input of the feedforward layer is time series data, and the output is hidden state. The feedforward layer is mainly responsible for extracting preliminary features from the input data, performing nonlinear transformation, and thus generating hidden representations. The input of the back propagation layer is the predicted output of the model, the true label, and the hidden state generated by the feedforward layer. The output is the updated model parameters. The goal of the back propagation layer is to optimize the weights of the model based on the loss function. The input of the feature fusion layer is the transformed content vector of the hidden state and input feature data, and the output is the fused time series representation, which is used for prediction tasks. In the feature fusion layer, the model uses the temporal attention mechanism to perform a weighted combination of the hidden state and other features generated by the feedforward layer to generate the final global feature representation.

[0135] The energy required for all actions of the edge device is counted and the amount of energy required for each action is defined as an energy level and sorted. Only when the current energy level is higher than the energy requirement of the action, the entire passive sensing system will select the next executable action.

[0136] Step 3: Build a GNN and PPO-based limited decision domain model to solve the action state explosion problem.

[0137] The predicted future time series data is processed to count the energy level of the system at each time step. The actions that the system can take are divided into perceiving data, executing a certain layer defined by the model in step 2, and waiting for the next time step, etc. Perceiving data refers to the edge device perceiving the actual task data to be read, and waiting for the next time step refers to not performing any action at the current time step to accumulate energy for the subsequent coherent execution of actions. Define the energy level required for each action and the amount of energy required, model it through a graph neural network, find all action sequences within a limited range, and finally make decisions through the trained PPO.

[0138] First, the system defines a limited state space. Specifically, the state space consists of the following elements:

[0139] Energy level: Discretize the energy state of the system into multiple levels, such as low, medium, and high, corresponding to different action selection ranges.

[0140] Data perception point: Data perception point is defined as the set of time nodes that the system can perceive. For example, data perception can only be performed at key moments to reduce unnecessary calculations.

[0141] Action set: The action set consists of multiple selectable actions, such as perceiving task data, executing a certain layer defined by the model in step 2, and waiting for the next time step, etc.

[0142] In the limited state space, the state may still be high-dimensional. In order to further reduce the computational complexity, we use a graph neural network for state dimension reduction processing. The state space is modeled as a graph, with nodes representing states and edges representing the transition relationship between states. The graph neural network learns a low-dimensional representation of each state node through graph convolution operations.

[0143] The state representation processed by the graph neural network is input into the subsequent reinforcement learning model as the basis for decision-making.

[0144] In the reduced decision space, we use the PPO (Proximal Policy Optimization) algorithm for policy optimization. PPO is a reinforcement learning algorithm based on policy gradient methods, which ensures the stability of policy updates through a clipped policy update method. The goal of PPO is to maximize the following objective function:

[0145]

[0146] where:

[0147] r t (θ) is the ratio of the new policy to the old policy:

[0148]

[0149] is the advantage function, which is calculated by Generalized Advantage Estimation (GAE) to estimate the advantage of the current action.

[0150] The clip function in PPO limits the policy ratio within an acceptable range, preventing the policy from being updated too much or too little, ensuring the stability of training.

[0151] ∈ is a small hyperparameter that controls the range of policy updates.

[0152] E t is the expectation symbol, representing the expected operation for all time steps t.

[0153] L CLIP (θ) is the PPO Clipped Objective Function, which measures the difference between the current policy and the old policy and optimizes the difference. The goal is to prevent excessive policy updates at each policy update, ensuring the stability of policy updates through the Clipping policy update method.

[0154] π θ (a t |s t ) is the probability distribution of the current policy, representing the probability of taking action a t in state s t . This probability is calculated by constructing a Gaussian distribution with the mean and standard deviation output by the PPO policy network;

[0155] The significance of this objective function is that if the new policy is better than the old policy, the update will be larger; if it is not, the update will be limited by the clipping operation to avoid excessive policy changes.

[0156] Specific process

[0157] 1. Initialize state space: Initialize a finite state space according to energy levels, data perception points, and action sets, ensuring that the size of the state space is controllable.

[0158] 2. Build graph structure: Model the initialized state space as a graph structure, with nodes representing certain states corresponding to the set of energy levels, actions, etc., and edges representing the transition relationship between states.

[0159] 3. Use GNN for dimension reduction: Use graph neural networks to reduce the dimension of the state space and obtain low-dimensional representations of each state.

[0160] 4. PPO reinforcement learning training: Use the reduced state representation to optimize the policy using the PPO algorithm. During training, the policy algorithm and edge devices interact with the environment according to the current selected policy and update the policy to maximize cumulative rewards. The algorithm updates its policy according to the immediate reward r t Perform action a t , then return the immediate reward r t and the next state s t+1 , update its policy according to the immediate reward r t . Specifically, PPO maximizes the long-term cumulative reward by optimizing the objective function.

[0161] 5. Strategy application: After sufficient training, use the final policy to make decisions in the environment and select the optimal action path.

[0162] Step 4: Use the above time series data prediction model and decision domain to predict time series data and select the best action sequence.

[0163] On the other hand, the embodiment of the present application also provides a time attention-based time series data intermittent learning system, characterized in that: it comprises a processor and a memory, the memory is used for storing program instructions, and the processor is used for calling the storage instructions in the memory to execute the time attention-based time series data intermittent learning method as described in the above technical solution.

[0164] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, without deviating from the spirit of the present application or exceeding the scope defined by the appended claims.

Claims

1. A time-series data intermittent learning method based on temporal attention, characterized in that: The steps include: Step 1: Obtain the amount of energy obtained from the outside world at each time step by the passive IoT edge device, that is, obtain energy time series data; Step 2: Sample selection is performed on the energy time series data. The most representative samples are automatically selected from the energy time series data through adaptive batch selection guided by variational inference and dynamic sample recalibration. Step 3: Build an online multi-layer Bi-GRU model based on the fusion attention mechanism and AdamW optimizer, input the energy number time series data sample in step 2, and output the energy number that can be obtained in the future time step; The Bi-GRU model learning process is divided into a feedforward layer, a backpropagation layer, a feature fusion layer based on temporal attention, and a prediction optimization layer. The feedforward layer is used to extract preliminary features from the input data and perform nonlinear transformations to generate hidden representations. The backpropagation layer inputs the model's predicted output, the true label, and the hidden state generated by the feedforward layer. The output is the updated model parameters. The goal of the backpropagation layer is to optimize the model's weights based on the loss function. The input of the feature fusion layer is the transformed content vector of the hidden state and input feature data, and the output is the fused time series representation used for prediction tasks. In the feature fusion layer, the model uses the temporal attention mechanism to perform a weighted combination of the hidden state and other features generated by the feedforward layer to generate the final global feature representation; In step 4, the energy level is defined based on the amount of energy obtained, as well as the energy level required for each action of the passive IoT edge device at each time step. Graph neural networks are used for modeling, where the nodes in the graph structure represent the action states corresponding to the energy levels, and the edges represent the transition relationships between states. The final proximal policy optimization algorithm (PPO) is used to find all action sequences within a limited range and select the optimal action sequence.

2. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: The implementation of the variational inference guided adaptive batch selection method in step 2 is as follows; For each new sample x in the data stream t , based on historical data Perform variational inference to estimate the contribution of the sample to the current learning task. The goal of variational inference is to infer the selection probability of the sample by maximizing the lower bound of evidence. The specific inference process is as follows: in, For sample x t Given historical data The probability of selection under , q(θ) is the variational distribution, θ is the parameter, E represents the expectation, which is used to calculate the lower bound of evidence in variational inference; In practice, variational inference is performed for each sample x t Calculate a selection probability And decide whether the sample is selected based on the probability.

3. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: The implementation of the dynamic sample recalibration method in step 2 is as follows; For each time step t, let For the model to sample x t The predicted value, y t is the true label, and the prediction error of the sample is Based on the error, the weight w of the sample t Dynamically updated using the following formula: Here, λ is a hyperparameter that controls weight updates.

4. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: The specific implementation of step 2 is as follows: Step 2.1: Input a series of time series sample streams {x1, x2, ..., x t }, each sample x t They all carry certain time information and possible tags; Step 2.2, variational inference: For each time step t, based on historical data That is, {x1,x2,…,x t-1 } and the current sample x t , calculate the sample x through variational inference t The probability of selection If the probability exceeds a preset threshold, the sample is considered to have a greater contribution to the current learning task and should be selected for learning; Step 2.3, dynamic sample recalibration: For each selected sample x t , first calculate its prediction error in For the model to sample x t The predicted value, y t is the true label; then adjust the weight w of the sample according to the error t , that is, through the formula: Step 2.4, sample selection: Based on the selection probability calculated by variational inference and the dynamically adjusted weights, the most representative samples at the current moment are selected to form the current learning batch And use it for subsequent incremental learning; Step 2.5, update the hyperparameter λ through the incremental learning algorithm, using the currently selected sample batch Conduct training; Step 2.6, loop execution: With the new sample stream x t+1 With the arrival of the new feature, the whole process continues, and the variational inference and sample recalibration mechanisms are dynamically adjusted according to the changes in the data stream, continuously optimizing sample selection and model training.

5. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: In step 3, the specific processing process of the online multi-layer Bi-GRU model is as follows: Step 3.1, input data preprocessing: select the time series data samples {x1, x2, ..., x T Standardization and normalization are performed to ensure that the data is within a reasonable range; where T represents the selection of samples of T time steps; Step 3.2, Bi-GRU encoding: Input the processed time series data into the Bi-GRU module to obtain the forward and reverse hidden states of each time step, and concatenate the two hidden states as the output; Step 3.3, fusion attention mechanism: by fusing the input features and Bi-GRU output features, calculating the attention weights, and performing weighted summation for each time step to obtain a global temporal representation; Step 3.4, prediction: input the weighted time series representation into the fully connected prediction layer and output the final prediction result; Step 3.5, optimization training: Use the AdamW optimizer to update the optimization model parameters and train the model by minimizing the MSE loss function; The update formula of the AdamW optimizer is as follows: Where: θ t represents the online Bi-GRU model parameters, represents the mean of the gradient, represents the variance of the gradient, ∈ is a smoothing term used to prevent division by zero errors; λ is the weight decay coefficient that controls the strength of regularization; η is the learning rate; The specific loss function L is calculated as follows: Where N is the number of samples, is the predicted value of the i-th sample, y i is the corresponding true value; Step 3.6, Weight Update: In each iteration, the AdamW optimizer updates the model parameters based on the calculated gradients and the regularization term.

6. The method for intermittent learning of time series data based on temporal attention according to claim 5, characterized in that: The specific implementation of step 3.2 is as follows: For each time step t, Bi-GRU generates two hidden states: one from the forward GRU, i.e., GRU forward , and the other comes from the reverse GRU, i.e. GRU backward ; Among them, x t is the time series data at time step t, refers to the hidden state generated by the forward GRU, Refers to the hidden state generated by the reverse GRU; The output of Bi-GRU is the concatenation of these two hidden states, representing the information at each time step:

7. The method for intermittent learning of time series data based on temporal attention according to claim 5, characterized in that: The specific implementation of step 3.3 is as follows: Input features x at each time step t and Bi-GRU output h t are concatenated and passed through a fully connected layer for nonlinear transformation: f t =W f [x t ;h t ]+b f Among them, W f and b f is the parameter to be learned, the concatenated feature vector f t Contains the input data and the timing information extracted by Bi-GRU; Based on the fusion feature f t , calculate the importance of each time step through the attention mechanism, the attention weight α t Calculated as follows: Among them, score(f t ,W att ) is the fusion feature f t and learnable weights W att The linear transformation is expressed as: score(f t ,W att )=f t W att +b att According to the attention weight α t , perform weighted summation on the fusion features of all time steps to obtain the final temporal representation h att :

8. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: In step 4, the actions taken by the passive IoT edge device include sensing data, executing a layer defined by the model in step 3, and waiting for the next time step. The sensed data refers to the actual task data that the passive IoT edge device senses and is supposed to read. Waiting for the next time step means that no action is performed in the current time step to accumulate energy for the coherent execution of subsequent actions.

9. The method for intermittent learning of time series data based on temporal attention according to claim 1, characterized in that: The goal of PPO is to maximize the following objective function: in: r t (θ′) is the ratio of the new policy to the old policy: Where, is the advantage function, which is calculated by generalized advantage estimation and is used to estimate the advantage of the current action; ∈ is a small hyperparameter that controls the range of policy updates; E t is the expected symbol, which represents the expected operation for all time steps t. The clip function is used to limit the strategy ratio to an acceptable range; L CLIP (θ′) is the clipping objective function of PPO, which is used to measure the difference between the current policy and the old policy; π θ′ (a t |s t ) is the probability distribution of the current strategy, indicating that in state s t Next, take action a t The probability of is the probability distribution of the old strategy.

10. A time-series data intermittent learning system based on temporal attention, characterized by: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a time series data intermittent learning method based on time attention as described in any one of claims 1 to 9.

Images