An edge device-oriented time sequence prediction causal representation lightweight learning method

Abstract

The application provides an edge device-oriented time series prediction causal representation lightweight learning method, and belongs to the technical field of time series prediction. The method comprises the following steps: performing lightweight block preprocessing on multi-source heterogeneous time series data collected by an edge device to construct a time series causal node set; adopting a lightweight constraint causal discovery algorithm to identify causal dependence and construct a lightweight time series causal graph; generating a low-dimensional causal representation vector through causal decoupling representation coding; constructing a lightweight time series prediction head to obtain an initial prediction result; obtaining a lightweight prediction model through structured knowledge distillation and dynamic parameter pruning optimization; and dynamically adjusting parameters based on incremental data to output a time series prediction result. Through multi-link collaborative optimization, the application realizes efficient processing of multi-source data, accurate causal modeling and extreme lightweight of the model, and solves the problems of high complexity of causal discovery, large number of model parameters and difficulty in adapting to edge device constraints in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of time series prediction technology, specifically a lightweight learning method for causal representation of time series prediction for edge devices. Background Technology

[0002] With the deepening of digital transformation, various terminal devices are gradually extending to the edge, generating massive amounts of heterogeneous data at the device level, creating an urgent need for real-time data processing and trend prediction. Time-series forecasting, as a core technology supporting autonomous device decision-making, anomaly warning, and resource scheduling, directly determines the operating efficiency and reliability of edge systems.

[0003] Current mainstream time series forecasting solutions largely rely on cloud computing power for model training and inference, which not only suffers from high transmission latency and large bandwidth consumption but also faces the risk of data privacy leaks. Deploying cloud models directly to edge devices is limited by the limited computing, storage, and energy resources at the edge, making it difficult to guarantee the model's inference speed and operational stability. Furthermore, existing methods are mostly based on correlations between data points, making it difficult to capture the underlying patterns in the data. They also lack generalization ability in complex and dynamic environments and are susceptible to noise and interference, leading to prediction biases. In addition, existing models lack efficient adaptive update mechanisms, failing to adapt to the constantly changing data distribution characteristics in edge scenarios, resulting in significant performance degradation after long-term operation. Unified processing of multi-source heterogeneous data also faces numerous bottlenecks. Deep fusion of data from different sources and formats is difficult, further limiting the improvement of prediction accuracy and failing to meet the combined requirements of real-time performance, accuracy, and lightweight design at the edge. Summary of the Invention

[0004] This invention provides a lightweight learning method for temporal prediction causal representation for edge devices, in order to address the shortcomings of existing technologies.

[0005] This invention provides a lightweight learning method for temporal prediction causal representations for edge devices, comprising:

[0006] Lightweight block preprocessing is performed on multi-source heterogeneous time-series data collected from edge devices to generate standardized time-series data blocks and construct a set of time-series causal nodes.

[0007] A lightweight constrained causal discovery algorithm based on local conditional independence testing identifies direct causal dependencies between temporal causal nodes, eliminates spurious associations, and constructs a lightweight temporal causal graph.

[0008] A causal decoupling representation encoding is performed on the lightweight temporal causal graph to separate causal factors and confusion factors in the temporal data and generate a low-dimensional causal representation vector.

[0009] A lightweight temporal prediction head based on depthwise separable convolution is constructed, and the low-dimensional causal representation vector is input into the prediction head to obtain the initial temporal prediction result.

[0010] By combining structured knowledge distillation and dynamic parameter pruning, a lightweight prediction model is obtained by jointly optimizing the causal representation encoding network and the temporal prediction head.

[0011] Based on incremental time-series data from edge devices, an incremental causal update mechanism is used to dynamically adjust the parameters of the lightweight time-series causal graph and the lightweight prediction model, and output the final time-series prediction results.

[0012] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices includes a lightweight block preprocessing procedure for multi-source heterogeneous temporal data collected from edge devices, comprising:

[0013] The raw multi-source time-series data collected by the edge device is divided into multiple time-series data blocks of equal length according to a fixed time granularity. The length of each time-series data block is dynamically configured according to the memory capacity of the edge device.

[0014] Missing values ​​in each time series data block are filled using forward imputation, and outliers are removed using outlier removal criteria based on statistical distribution.

[0015] Z-score standardization is performed on the preprocessed time series data.

[0016] Each variable in the standardized time-series data block is mapped to a time-series causal node, and a set of time-series causal nodes containing node identifier, variable type and timestamp attributes is constructed.

[0017] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices is provided. The process of constructing a lightweight temporal causal graph includes:

[0018] Extract the temporally ordered sequence of nodes from the temporally ordered causal node set, sort the nodes according to the order of timestamps, and retain the variable values ​​and time interval attributes of the nodes.

[0019] A lightweight PC algorithm based on local conditional independence testing is adopted, which performs conditional independence testing only within a preset order neighborhood of each node, and the test statistic is the chi-square statistic.

[0020] When the local chi-square test statistic is less than the critical value corresponding to the preset significance level, it is determined that there is no direct causal dependency between the two nodes, and the corresponding spurious association edge is removed.

[0021] Calculate the causal strength of the retained causal dependency edges.

[0022] A lightweight temporal causal graph is constructed using temporal causal nodes as vertices and causal dependencies as directed edges, combined with causal strength attributes.

[0023] According to the present invention, a lightweight learning method for temporal prediction causal representations for edge devices is provided, the process of generating low-dimensional causal representation vectors includes:

[0024] Using the adjacency matrix of a lightweight temporal causal graph and standardized temporal data blocks as input, local causal structure features of nodes are extracted using at least one lightweight graph convolutional layer.

[0025] By introducing causal decoupling constraints, the extracted features are decomposed into causal factor vectors and confusion factor vectors. The causal factor vectors contain only information that has a causal impact on the prediction results, while the confusion factor vectors contain non-causal noise and interference information.

[0026] Construct a causal decoupling loss function to constrain the degree of separation between causal factors and confounding factors.

[0027] The parameters of the representation encoding network are optimized by minimizing the causal decoupling loss function, and a low-dimensional causal representation vector of a preset dimension is output.

[0028] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices is provided. The process of constructing a lightweight temporal prediction head based on depthwise separable convolution includes:

[0029] The lightweight temporal prediction head adopts a multi-layer structure of depthwise separable convolutions. Each depthwise separable convolution consists of a depthwise convolutional layer and a pointwise convolutional layer, using a pre-set kernel size and a pre-set stride, with the same padding method.

[0030] After each depthwise separable convolution, a batch normalization layer and a ReLU activation function are added, and after the last convolution, a global average pooling layer and a fully connected layer are added.

[0031] The low-dimensional causal representation vector is reshaped into a two-dimensional tensor of shape (time step, representation dimension), and input into a lightweight time series prediction head.

[0032] The lightweight timing prediction head outputs timing prediction values ​​for a preset number of time steps in the future, with the prediction step size dynamically configured according to the application requirements of the edge device.

[0033] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices is provided, and the process of optimizing the model using structured knowledge distillation includes:

[0034] A high-precision teacher model based on Transformer is pre-trained on a cloud server, with the number of parameters in the teacher model being a preset multiple of that in the student model.

[0035] By simultaneously inputting the same low-dimensional causal representation vector into both the teacher model and the student model, the teacher prediction results and the student prediction results are obtained, respectively.

[0036] Construct a knowledge distillation loss function.

[0037] By minimizing the knowledge distillation loss function, the causal knowledge learned by the teacher model is transferred to the student model, thereby improving the prediction accuracy of the student model.

[0038] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices is provided, and the process of optimizing the model using dynamic parameter pruning includes:

[0039] After knowledge distillation is completed, the L1 norm of each convolutional kernel in the model is calculated as the importance score of that convolutional kernel.

[0040] For each layer of the network in the model, calculate the dynamic pruning threshold for that layer.

[0041] Convolutional kernels and their connection parameters with importance scores lower than the corresponding pruning threshold are removed from the model while preserving the causal structure integrity of the model.

[0042] The pruned model is then adjusted using a fine-tuned learning rate lower than the initial training learning rate.

[0043] According to the lightweight learning method for temporal prediction causal representation of edge devices provided by the present invention, after dynamic parameter pruning, a model quantization and compression step is further included. The specific process is as follows:

[0044] The pruned model parameters are quantized using an integer quantization method with a preset bit width, converting the original bit-width floating-point weights into integer weights with a preset bit width.

[0045] Calculate the average error of quantization compression.

[0046] When the average quantization error is less than the preset quantization error threshold, the quantized lightweight prediction model is saved.

[0047] According to the present invention, a lightweight learning method for temporal prediction causal representation for edge devices is provided. The process of dynamically adjusting model parameters using an incremental causal update mechanism includes:

[0048] When the edge device collects new incremental time-series data, it extracts the time-series causal node sequence corresponding to the incremental data.

[0049] Only the causal dependency between the new node and the original node is tested for local conditional independence, without the need to rebuild the complete temporal causal graph.

[0050] The causal strength of causal edges is updated based on incremental data.

[0051] When the causal strength is less than the preset lower limit, the corresponding causal edge is deleted. When a new causal dependency is discovered and its causal strength is greater than the preset upper limit, the corresponding causal edge is added.

[0052] The parameters of the lightweight prediction model are updated using an incremental learning approach, updating only the network layer parameters related to newly added causal edges, while keeping the parameters of other layers unchanged.

[0053] According to the lightweight learning method for causal representation of temporal prediction for edge devices provided by the present invention, the process of outputting the final temporal prediction result includes:

[0054] The time-series data collected in real time by the edge device is input into the preprocessing module to generate standardized time-series data blocks.

[0055] The standardized time-series data blocks are input into the updated lightweight time-series causal graph and lightweight prediction model to obtain real-time low-dimensional causal representation vectors and time-series prediction values.

[0056] The time series prediction values ​​are denormalized to restore the original data dimensions, and the final time series prediction results are obtained and output to the designated output terminal of the edge device.

[0057] The final time series prediction result is compared with the subsequently collected real data to calculate the prediction error. When the prediction error exceeds a preset threshold for a preset number of consecutive time steps, the next round of incremental causal update process is triggered.

[0058] This invention provides a lightweight learning method for causal representation in time series prediction for edge devices. It achieves efficient processing of multi-source time series data through lightweight block preprocessing, constructs an accurate time series causal graph using a lightweight constrained causal discovery algorithm, extracts robust causal features through causal decoupling representation, and achieves extreme lightweighting of the model by combining structured knowledge distillation and dynamic parameter pruning. It also uses an incremental causal update mechanism to ensure the long-term adaptability of the model on edge devices. This invention solves the problems of high computational complexity of causal discovery, large number of model parameters, and inability to adapt to the computing power and storage constraints of edge devices in existing technologies. It can achieve high-precision, low-latency time series prediction on resource-constrained edge devices. Attached Figure Description

[0059] The invention will now be further described with reference to the accompanying drawings.

[0060] Figure 1 This is a flowchart illustrating a lightweight learning method for temporal prediction causal representation for edge devices in this invention.

[0061] Figure 2This is a schematic diagram of the process for constructing a lightweight temporal cause-effect graph in this invention;

[0062] Figure 3 This is a schematic diagram of the process for generating low-dimensional causal representation vectors in this invention. Detailed Implementation

[0063] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0064] like Figures 1 to 3 As shown in the figure, an embodiment of the present invention provides a lightweight learning method for temporal prediction causal representation for edge devices, comprising:

[0065] Lightweight block preprocessing is performed on multi-source heterogeneous time-series data collected from edge devices to generate standardized time-series data blocks and construct a set of time-series causal nodes.

[0066] The raw multi-source time-series data collected by the edge device is divided into multiple time-series data blocks of equal length according to a fixed time granularity. The length of each time-series data block is dynamically configured according to the memory capacity of the edge device. The selection of time granularity is matched with the sampling frequency of the edge device to ensure that each data block contains sufficient time-series information without exceeding the memory processing limit of the edge device.

[0067] Missing values ​​in each time series data block are filled using forward imputation. For data with no more than 5 consecutive missing time steps, the nearest non-missing value is used for forward imputation. For data with more than 5 consecutive missing time steps, it is marked as invalid data and removed from the data block to avoid interference from invalid data with subsequent processing results.

[0068] Outliers are removed using outlier removal criteria based on statistical distribution. Specifically, the mean and standard deviation of each variable within the current data block are calculated. Data that exceeds the mean plus or minus three times the standard deviation are identified as outliers. Outliers are replaced with the mean of adjacent normal data to ensure the continuity and stability of the data.

[0069] The preprocessed time series data is standardized using Z-score, calculated as follows:

[0070]

[0071] in, For the standardized time series data at the t-th time step, This represents the original time series data at time step t. This represents the mean of all data within the current time-series data block. This represents the standard deviation of all data within the current time-series data block. To prevent the preset minimum value where the denominator is zero.

[0072] The standardized continuous numerical variables are discretized using equal-frequency methods, dividing each variable into a preset number of discrete intervals. The number of intervals is the square root of the number of variable values, or the optimal number of intervals is calculated according to the Freedman-Diaconis rule. This rule determines the interval width based on the interquartile range of the data, which can effectively avoid the influence of outliers on the interval division, ensuring that each interval contains approximately an equal number of samples. This converts the continuous variable into a categorical variable to meet the applicable prerequisites for the subsequent chi-square test.

[0073] Each variable in the discretized time-series data block is mapped to a time-series causal node. A set of time-series causal nodes containing node identifiers, variable types, and timestamp attributes is constructed. Each node identifier is assigned a globally unique identifier. Variable types are divided into two categories: numeric and categorical. Timestamps adopt the UTC standard time format, accurate to the millisecond level, to ensure the accuracy of the time-series relationship between nodes.

[0074] A lightweight constrained causal discovery algorithm based on local conditional independence testing identifies direct causal dependencies between temporal causal nodes, eliminates spurious associations, and constructs a lightweight temporal causal graph.

[0075] Extract the temporally ordered sequence of nodes from the temporally ordered causal node set, sort the nodes according to the order of their timestamps, and sort the nodes according to the lexicographical order of their node identifiers for nodes that occur at the same time. Preserve the variable values ​​and time interval attributes of the nodes. The time interval attribute records the time difference between adjacent nodes for subsequent calculation of causal delay.

[0076] A lightweight PC algorithm based on local conditional independence testing is adopted, which performs conditional independence testing only within a preset order neighborhood of each node. The preset order is set to 2 to 5, which can be dynamically adjusted according to the computing power of the edge device and the number of data variables: 2 to 3 orders are used for low-power microcontroller edge devices to control the amount of computation, while 4 to 5 orders can be used for edge gateways and industrial edge boxes with certain computing power to improve the accuracy of causal discovery. The preset order neighborhood refers to the set of nodes extending forward and backward along the temporal direction by a preset number of time steps with the current node as the center. By limiting the testing range, the computational complexity is greatly reduced, which is suitable for the computing power constraints of edge devices.

[0077] The test statistic used is the partial chi-square statistic, calculated using the following formula:

[0078]

[0079] in, This is the local chi-square test statistic, with the subscript L indicating the test within a local neighborhood. Let the joint frequency of observations of variables a and b be denoted as . Let m be the expected joint frequency of variables a and b, m be the number of categories of variable a, and n be the number of categories of variable b.

[0080] When the local chi-square test statistic is less than the critical value corresponding to the preset significance level, it is determined that there is no direct causal dependency between the two nodes, and the corresponding spurious association edges are removed. The preset significance level is usually set to 0.05, corresponding to 95% confidence, and can be adjusted according to the fault tolerance requirements of the application scenario to balance the recall and precision of causal discovery.

[0081] The causal strength of the retained causal dependency edges is calculated using a backdoor adjustment method based on observed data to calculate the intervention probability. This method does not require prior knowledge of the complete causal graph structure. Spurious associations are eliminated by controlling the observed set of confounding variables. The calculation formula is as follows:

[0082]

[0083] in, Let represent the causal strength from node i to node j, with a value in the range [0,1], and Z be the set of observed confounding variables. Given variable i and confusion variable z, the conditional probability of node j occurring. To confound the prior probability of variable z Let be the prior probability of node j occurring.

[0084] A lightweight temporal causal graph is constructed using temporal causal nodes as vertices and causal dependencies as directed edges, combined with causal strength attributes. It is stored in an adjacency list format, where each node only stores information about its direct neighbors and the corresponding causal edge attributes. Compared with the adjacency matrix storage method, it significantly reduces memory usage and is suitable for deployment on edge devices.

[0085] A causal decoupling representation encoding is performed on the lightweight temporal causal graph to separate causal factors and confusion factors in the temporal data and generate a low-dimensional causal representation vector.

[0086] The adjacency matrix of the lightweight temporal causal graph and the standardized temporal data block are used as input. At least one lightweight graph convolutional layer is used to extract the local causal structure features of the nodes. The lightweight graph convolutional layer adopts a grouped convolutional structure, which divides the input feature channels into multiple independent groups and performs convolution operations separately to reduce the number of parameters and computation.

[0087] Introducing causal decoupling constraints, the extracted features are decomposed into causal factor vectors. With confusion factor vector The causal factor vector contains only information that has a causal impact on the prediction result, while the confusion factor vector contains non-causal noise and interference information. The two are mapped through independent fully connected layers to ensure the separation of the representation space.

[0088] A causal decoupling loss function with gradient penalty is constructed, and an alternating optimization approach is used to constrain the degree of separation between causal factors and confusion factors. The calculation formula is as follows:

[0089]

[0090] in, The total loss for causal decoupling. The preset weighting coefficients, The loss of causal consistency is achieved using the intervention invariance loss, and the specific calculation formula is as follows:

[0091]

[0092] In the formula, For mathematical expectation operators, For causal factor vectors, This is a pre-set, minimal intervention amount. For the mapping function of the lightweight timing prediction head, For predicting the head in causal factors The gradient at the point is used to ensure that the change in the prediction result is linearly related to the intensity of the intervention when a small intervention is made on the causal factor, thus ensuring the interpretability and stability of the causal factor.

[0093] To address the confusion factor separation loss, a mutual information lower bound estimation method is employed, which minimizes the mutual information between causal factors and confusion factors through adversarial training. This is a gradient penalty term used to stabilize the adversarial training process and prevent gradient vanishing or exploding. represents the weight coefficient of the gradient penalty term.

[0094] The training process employs an alternating optimization strategy. First, the parameters of the confusion factor encoder are fixed to optimize the causal factor encoder. Then, the parameters of the causal factor encoder are fixed to optimize the confusion factor encoder. This process is repeated until the loss converges, effectively resolving the training instability caused by the conflict between the two loss objectives.

[0095] The parameters of the representation encoding network are optimized by minimizing the causal decoupling loss function, and a low-dimensional causal representation vector with a preset dimension is output. The representation dimension is configured according to the computing power and prediction accuracy requirements of the edge device, and the dimension is reduced as much as possible while ensuring prediction accuracy.

[0096] A lightweight temporal prediction head based on depthwise separable convolution is constructed, and the low-dimensional causal representation vector is input into the prediction head to obtain the initial temporal prediction result.

[0097] The lightweight temporal prediction head adopts a multi-layer structure of depthwise separable convolutions. Each depthwise separable convolution consists of a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer performs convolution operations independently on each input channel to extract temporal features within a single channel. The pointwise convolutional layer uses a 1×1 convolutional kernel to fuse features from different channels. Compared with standard convolution, this can significantly reduce the number of parameters and computation.

[0098] A pre-defined kernel size and stride are used, with the same padding method to ensure consistent input and output feature map sizes. The pre-defined kernel size is set to 3×1 or 5×1 (covering 3 or 5 time steps in the temporal dimension and 1 in the feature dimension). A 3×1 kernel is preferred to balance the local temporal feature extraction capability with computational cost. The pre-defined stride is set to 1 by default, but can be set to 2 if downsampling is required. The kernel size is set according to the local correlation of the temporal data, and the stride is set to 1 to preserve the complete temporal resolution.

[0099] After each depthwise separable convolution, a batch normalization layer and a ReLU activation function are added. The batch normalization layer normalizes the features of each batch, accelerating model convergence and preventing overfitting. The ReLU activation function introduces a non-linear transformation, enhancing the model's expressive power.

[0100] After the last convolutional layer, a global average pooling layer and a fully connected layer are added. The global average pooling layer performs a global average operation on the feature map of each channel, converting the two-dimensional feature map into a one-dimensional feature vector. The fully connected layer maps the feature vector to the final temporal prediction value.

[0101] The low-dimensional causal representation vector is reshaped into a two-dimensional tensor with shape (time step, representation dimension), which is then input into a lightweight time series prediction head. The lightweight time series prediction head outputs the time series prediction values ​​for a preset number of future time steps. The prediction step size is dynamically configured according to the application requirements of the edge device, supporting both single-step and multi-step prediction modes.

[0102] By combining structured knowledge distillation and dynamic parameter pruning, a lightweight prediction model is obtained by jointly optimizing the causal representation encoding network and the temporal prediction head.

[0103] The model is optimized using structured knowledge distillation. A high-precision teacher model based on Transformer is pre-trained on a cloud server. The number of parameters in the teacher model is a preset multiple of the student model, which is set to 8 to 20 times. In scenarios with sufficient cloud computing power and high accuracy requirements, a multiple of 15 to 20 times can be used to maximize the knowledge transfer effect. In scenarios with limited cloud resources or higher priority of lightweighting, a multiple of 8 to 12 times is used. The teacher model adopts a complete causal discovery algorithm and a complex network structure, which has higher prediction accuracy and causal representation ability.

[0104] By simultaneously inputting the same low-dimensional causal representation vector into both the teacher model and the student model, teacher prediction results and student prediction results are obtained respectively. The teacher prediction results contain rich soft label information and can reflect the similarity between different prediction results.

[0105] Construct the knowledge distillation loss function, and calculate the formula as follows:

[0106]

[0107] in, For the total loss of knowledge distillation, The preset weighting coefficients, The mean squared error loss between the student's predicted results and the true labels is used to ensure the student model's ability to fit the true labels. For real labels, For the prediction results of the student model, KL divergence loss is used to measure the difference between the output distributions of the teacher and student models, enabling the transfer of causal knowledge. For the prediction results of the teacher model, The preset temperature coefficient is used to smooth the output distribution and retain more soft label information. The square of the temperature coefficient is used to offset the effect of temperature on the KL divergence gradient and ensure the stability of gradient updates.

[0108] By minimizing the knowledge distillation loss function, the causal knowledge learned by the teacher model is transferred to the student model, thereby improving its prediction accuracy without increasing the number of parameters in the student model.

[0109] Dynamic parameter pruning is used to optimize the model. After knowledge distillation, the L1 norm of each convolutional kernel in the model is calculated as the importance score of the convolutional kernel. The larger the L1 norm, the greater the contribution of the features extracted by the convolutional kernel to the prediction result.

[0110] For each layer of the model, the layer importance coefficient is determined based on the layer's position in the model and its gradient contribution. Shallow networks are responsible for extracting basic temporal features and have a smaller impact on the overall performance of the model, so their layer importance coefficients are set to smaller values. Deep networks are responsible for extracting advanced causal semantic features and have a larger impact on prediction accuracy, so their layer importance coefficients are set to larger values.

[0111] Shallow networks The initial value is 0.5, deep network The initial value is 1.5, and the gradient dynamic adjustment step size is 0.1. During the fine-tuning process, the layer importance coefficient is dynamically adjusted in each round according to the gradient mean of the layer parameters, so that the pruning process is more in line with the actual contribution distribution of the model.

[0112] The dynamic pruning threshold for this layer is calculated using the following formula:

[0113]

[0114] in, The pruning threshold for the k-th layer of the network. Let k be the layer importance coefficient. The total number of convolutional kernels in the k-th layer of the network. Let L1 be the L1 norm of the i-th convolutional kernel in the k-th layer.

[0115] Convolutional kernels and their connection parameters with importance scores lower than the corresponding pruning threshold are removed from the model, while preserving the integrity of the model's causal structure, ensuring that the pruned model can still correctly extract causal features.

[0116] The pruned model is fine-tuned to restore its prediction accuracy. The fine-tuning process uses a learning rate lower than the initial training learning rate and only updates the values ​​of the retained parameters to avoid destroying the learned causal knowledge.

[0117] After dynamic parameter pruning is completed, a model quantization and compression step is also included. The pruned model parameters are quantized using an integer quantization method with a preset bit width. The original bit width floating-point weights are converted into integer weights with a preset bit width. The quantization process uses a symmetric quantization method to map the floating-point weights to the integer value range while retaining the sign information of the weights.

[0118] The average error of quantization compression is calculated using the following formula:

[0119]

[0120] in, For the average quantization error, This represents the total number of parameters in the model. For the j-th original bit-width floating-point weight, The integer weight is the preset bit width of the j-th quantized element.

[0121] When the average quantization error is less than the preset quantization error threshold, the quantized lightweight prediction model is saved. The preset quantization error threshold is set to 1×10. -4 ~1×10 -3 High prediction accuracy requires a scenario with a resolution of 1×10. -4 For scenarios prioritizing inference speed over storage usage, a value of 1×10 is used. -3 The quantized model has a significantly smaller size than the original model, while the inference speed is greatly improved, making it suitable for the storage and computing power constraints of edge devices.

[0122] Based on incremental time-series data from edge devices, an incremental causal update mechanism is used to dynamically adjust the parameters of the lightweight time-series causal graph and the lightweight prediction model, and output the final time-series prediction results.

[0123] When the edge device collects new incremental time-series data, it extracts the time-series causal node sequence corresponding to the incremental data, performs the same preprocessing operation as the initial data on the incremental data, and generates standardized incremental data blocks.

[0124] Local conditional independence is tested for the causal dependencies between new nodes and existing nodes. At the same time, causal edges between existing nodes whose causal intensity changes exceed a preset threshold are retested. This preset threshold is usually set to 20% to 30% of the original causal intensity. When the change in causal intensity exceeds this proportion, a retest is triggered. The scope of the retest is limited to edges that have temporal overlap with the new nodes. There is no need to rebuild the complete temporal causal graph, which greatly reduces the computational cost of the update process and solves the problem of causal graph lag caused by concept drift.

[0125] The causal strength of the causal edge is updated based on incremental data, and the calculation formula is as follows:

[0126]

[0127] in, For the updated causal strength, The causal strength before the update. The preset incremental learning rate is used to control the magnitude of the causal strength update. The change in causal strength is calculated using the backdoor adjustment method based on incremental data; that is, the difference between the new causal strength calculated using incremental data and the original causal strength.

[0128] When the causal strength is less than 0.1, the corresponding causal edge is deleted. When a new causal dependency is found and the causal strength is greater than 0.3, the corresponding causal edge is added to ensure that the causal graph can reflect changes in data distribution in a timely manner.

[0129] The parameters of the lightweight prediction model are updated using an incremental learning approach. Only the parameters of the network layers related to newly added and changed causal edges are updated, while the parameters of the remaining layers remain unchanged. The incremental learning process uses the mini-batch gradient descent algorithm, and the learning rate is set to one-tenth of the initial training learning rate to avoid destroying the stable knowledge already learned by the model.

[0130] The process of outputting the final time series prediction result includes: inputting the time series data collected in real time by the edge device into the preprocessing module to generate standardized time series data blocks. The preprocessing module adopts a pipelined processing method, processing the previous data block while collecting new data to ensure the real-time performance of data processing.

[0131] Standardized time-series data blocks are input into the updated lightweight time-series causal graph and lightweight prediction model to obtain real-time low-dimensional causal representation vectors and time-series prediction values. The inference process is accelerated using an optimized inference engine to ensure that the inference latency meets the real-time requirements of edge devices.

[0132] The time series prediction values ​​are destandardized, and the mean and standard deviation saved in the preprocessing stage are used to restore the predicted values ​​to the original data dimensions. The final time series prediction results are obtained and output to the designated output terminal of the edge device. The output terminals include local displays, edge gateways and cloud servers, etc., and multiple output methods are supported.

[0133] The final time-series prediction results are compared with the subsequently collected real data to calculate the prediction error, which is measured using the root mean square error. When the prediction error exceeds a preset threshold for a number of consecutive time steps, the next round of incremental causal update process is triggered. The preset number of consecutive time steps is set to 3 to 5 steps, and the preset error threshold is set to 1.2 to 1.5 times the root mean square error of the initial validation set of the model. At the same time, a forced update mechanism with a fixed time interval is set to ensure the long-term effectiveness of the causal graph and the model.

[0134] In summary, this embodiment provides a lightweight learning method for causal representation in time series prediction for edge devices. It achieves efficient processing of multi-source time series data through lightweight block preprocessing, constructs an accurate time series causal graph using a lightweight constrained causal discovery algorithm, extracts robust causal features through causal decoupling representation, achieves extreme lightweighting of the model by combining structured knowledge distillation and dynamic parameter pruning, and ensures the long-term adaptability of the model on edge devices by using an incremental causal update mechanism. It solves the problems of high computational complexity of causal discovery, large number of model parameters, and inability to adapt to the computing power and storage constraints of edge devices in existing technologies, and can achieve high-precision, low-latency time series prediction on resource-constrained edge devices.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight learning method for temporal prediction causal representations for edge devices, characterized in that, include: Lightweight block preprocessing is performed on multi-source heterogeneous time-series data collected from edge devices to generate standardized time-series data blocks and construct a set of time-series causal nodes. A lightweight constrained causal discovery algorithm based on local conditional independence test identifies direct causal dependencies between temporal causal nodes, eliminates spurious associations, and constructs a lightweight temporal causal graph. A causal decoupling representation encoding is performed on a lightweight temporal causal graph to separate causal factors and confusion factors in the temporal data and generate a low-dimensional causal representation vector. A lightweight temporal prediction head based on depthwise separable convolution is constructed, and a low-dimensional causal representation vector is input into the prediction head to obtain the initial temporal prediction result; A lightweight prediction model is obtained by combining structured knowledge distillation and dynamic parameter pruning to jointly optimize the causal representation encoding network and the temporal prediction head. Based on incremental time-series data from edge devices, an incremental causal update mechanism is used to dynamically adjust the parameters of the lightweight time-series causal graph and the lightweight prediction model, and output the final time-series prediction results.

2. The lightweight learning method for temporal prediction causal representation of edge devices according to claim 1, characterized in that, The process of lightweight block preprocessing for multi-source heterogeneous time-series data collected from edge devices includes: The raw multi-source time-series data collected by the edge device is divided into multiple time-series data blocks of equal length according to a fixed time granularity. The length of each time-series data block is dynamically configured according to the memory capacity of the edge device. Missing values ​​in each time series data block are filled using forward imputation, and outliers are removed using outlier removal criteria based on statistical distribution. The preprocessed time series data is Z-score standardized. Each variable in the standardized time-series data block is mapped to a time-series causal node, and a set of time-series causal nodes containing node identifier, variable type and timestamp attributes is constructed.

3. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 1, characterized in that, The process of constructing a lightweight time-series cause-effect graph includes: Extract the temporally ordered sequence of nodes from the temporally ordered causal node set, sort the nodes according to the order of timestamps, and retain the variable values ​​and time interval attributes of the nodes. A lightweight PC algorithm based on local conditional independence test is adopted, which performs conditional independence test only in the neighborhood of a preset order for each node, and the test statistic is the chi-square statistic. When the local chi-square test statistic is less than the critical value corresponding to the preset significance level, it is determined that there is no direct causal dependency between the two nodes, and the corresponding spurious association edge is removed. Calculate the causal strength of the retained causal dependency edges; A lightweight temporal causal graph is constructed using temporal causal nodes as vertices and causal dependencies as directed edges, combined with causal strength attributes.

4. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 1, characterized in that, The process of generating low-dimensional causal representation vectors includes: The adjacency matrix of a lightweight temporal causal graph and a normalized temporal data block are used as input, and at least one lightweight graph convolutional layer is used to extract the local causal structure features of the nodes. By introducing causal decoupling constraints, the extracted features are decomposed into causal factor vectors and confusion factor vectors. The causal factor vectors contain only information that has a causal impact on the prediction results, while the confusion factor vectors contain non-causal noise and interference information. Construct a causal decoupling loss function to constrain the degree of separation between causal factors and confounding factors; The parameters of the representation encoding network are optimized by minimizing the causal decoupling loss function, and a low-dimensional causal representation vector of a preset dimension is output.

5. The lightweight learning method for temporal prediction causal representation of edge devices according to claim 1, characterized in that, The process of constructing a lightweight temporal prediction head based on depthwise separable convolutions includes: The lightweight temporal prediction head adopts a multi-layer depthwise separable convolution stack structure. Each depthwise separable convolution consists of a depthwise convolutional layer and a pointwise convolutional layer. It uses a convolutional kernel of a preset size and a preset stride, and the padding method is the same. After each depthwise separable convolution, a batch normalization layer and a ReLU activation function are added, and after the last convolution, a global average pooling layer and a fully connected layer are added. The low-dimensional causal representation vector is reshaped into a two-dimensional tensor of shape (time step, representation dimension), and input into a lightweight time series prediction head. The lightweight timing prediction head outputs timing prediction values ​​for a preset number of time steps in the future, with the prediction step size dynamically configured according to the application requirements of the edge device.

6. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 1, characterized in that, The process of optimizing the model using structured knowledge distillation includes: A high-precision teacher model based on Transformer is pre-trained on a cloud server, with the number of parameters in the teacher model being a preset multiple of that in the student model; By simultaneously inputting the same low-dimensional causal representation vector into both the teacher model and the student model, the teacher prediction results and the student prediction results are obtained respectively. Construct a knowledge distillation loss function; By minimizing the knowledge distillation loss function, the causal knowledge learned by the teacher model is transferred to the student model, thereby improving the prediction accuracy of the student model.

7. The lightweight learning method for temporal prediction causal representation of edge devices according to claim 1, characterized in that, The process of optimizing the model using dynamic parameter pruning includes: After knowledge distillation is completed, the L1 norm of each convolutional kernel in the model is calculated as the importance score of that convolutional kernel; For each layer of the network in the model, calculate the dynamic pruning threshold for that layer; Convolutional kernels and their connection parameters with importance scores lower than the corresponding pruning threshold are removed from the model while preserving the causal structure integrity of the model. The pruned model is then adjusted using a fine-tuned learning rate lower than the initial training learning rate.

8. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 7, characterized in that, After dynamic parameter pruning is completed, a model quantization and compression step is also included. The specific process is as follows: The pruned model parameters are quantized using an integer quantization method with a preset bit width, converting the original bit-width floating-point weights into integer weights with a preset bit width. Calculate the average error of quantization compression; When the average quantization error is less than the preset quantization error threshold, the quantized lightweight prediction model is saved.

9. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 1, characterized in that, The process of dynamically adjusting model parameters using an incremental causal update mechanism includes: When the edge device collects new incremental time-series data, it extracts the time-series causal node sequence corresponding to the incremental data. Only the causal dependency between the new node and the original node is tested for local conditional independence, without the need to rebuild the complete temporal causal graph; Update the causal strength of causal edges based on incremental data; When the causal strength is less than the preset lower limit of causal strength, the corresponding causal edge is deleted; when a new causal dependency is discovered and the causal strength is greater than the preset upper limit of causal strength, the corresponding causal edge is added. The parameters of the lightweight prediction model are updated using an incremental learning approach, updating only the network layer parameters related to newly added causal edges, while keeping the parameters of the other layers unchanged.

10. The lightweight learning method for temporal prediction causal representation for edge devices according to claim 1, characterized in that, The process of outputting the final time series prediction results includes: The time-series data collected in real time by the edge device is input into the preprocessing module to generate standardized time-series data blocks; The standardized time-series data blocks are input into the updated lightweight time-series causal graph and lightweight prediction model to obtain real-time low-dimensional causal representation vectors and time-series prediction values. The time series prediction values ​​are denormalized to restore the original data dimensions, and the final time series prediction results are obtained and output to the designated output terminal of the edge device. The final time series prediction result is compared with the subsequently collected real data to calculate the prediction error. When the prediction error exceeds a preset threshold for a preset number of consecutive time steps, the next round of incremental causal update process is triggered.

Images