Edge computing-oriented recurrent neural network model compression method and system

Abstract

The invention relates to an edge computing-oriented recurrent neural network model compression method and system, and the method comprises the steps: constructing a recurrent neural network model, and training the recurrent neural network model based on historical power time sequence data, thereby obtaining a teacher model; compressing the teacher model by using the compression configuration scheme, and obtaining a plurality of indexes; performing iterative optimization on the compression configuration scheme by using a multi-objective evolutionary algorithm and a multi-dimensional optimization objective function based on multiple indexes to obtain an optimal compression configuration scheme; repeating the compression process by using the optimal compression configuration scheme to obtain a student model; carrying out distillation training on the student model by utilizing the feature distribution output by the teacher model, and deploying the student model after distillation training at an edge node; and performing iterative optimization on the student model or the compression configuration scheme by utilizing a sensitivity analysis mechanism based on the real-time data obtained after deployment. Compared with the prior art, the system has the advantages of high integration level and good compression effect.

Description

Technical Field

[0001] This invention relates to the field of data processing and prediction technology, and in particular to a method and system for compressing recurrent neural network models for edge computing. Background Technology

[0002] With the development of smart grids and the Internet of Things (IoT) in the power system, power systems are gradually exhibiting characteristics of edge-distributed, data-driven, and intelligently autonomous operation. Deploying intelligent algorithms at edge nodes such as substations, distribution rooms, and terminal loads to achieve tasks such as fault detection, electricity consumption forecasting, energy consumption analysis, and equipment status identification has become an important development direction for power system intelligence. Time-series data, as the most important data form in the power system, is characterized by strong continuity, obvious periodicity, and sparse anomaly features. For modeling and analyzing this type of data, Recurrent Neural Networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), have been widely applied to power time-series data prediction and modeling tasks, demonstrating good modeling capabilities and prediction accuracy.

[0003] However, RNN-type models are typically large in terms of parameter count, computational complexity, and model storage, making them unsuitable for deployment in edge computing scenarios. Power edge computing nodes are usually limited by computing power, storage space, and energy consumption constraints, making it difficult to support the operation of original large-scale RNN models. Furthermore, the high latency and memory overhead of time-series models during inference also limit their real-time performance and response efficiency, especially in scenarios dealing with multi-source concurrent data.

[0004] Existing model compression methods, such as pruning, quantization, and knowledge distillation, while offering some optimization for model size and computational cost in general scenarios, largely fail to address the structural optimization specific to the characteristics of power time-series data and still present trade-offs in maintaining model prediction accuracy. For example, structural pruning methods often rely on extensive repetitive training, making them difficult to adapt to the needs of online model updates at edge computing environments; quantization methods are prone to model performance instability when processing highly dynamic power load data; and knowledge distillation methods require a teacher model for training, increasing training complexity and deployment costs.

[0005] Therefore, existing technologies still have shortcomings in addressing the deployment and operational efficiency issues of temporal RNN models in power edge computing scenarios. For example, the model structure is not customized and optimized for the characteristics of power time-series data, and compression easily results in the loss of temporal dependency capabilities. Most compression methods are not very resource-friendly for edge computing, making it difficult to achieve lightweight, low-latency deployment. Furthermore, the current lack of a unified collaborative strategy to comprehensively optimize model compression ratio, prediction performance, and energy efficiency leads to high training costs and weak generalization ability, making it difficult to adapt to the online learning needs in dynamic power scenarios. Therefore, there is an urgent need for a temporal recurrent neural network model compression method that is designed for power edge computing scenarios, balances model compression ratio and performance stability, and possesses high efficiency and deployability, in order to further promote the practicality and intelligence of power smart terminals.

[0006] The invention disclosed in CN112700057B is a method for predicting power load based on a modular recurrent neural network, comprising the following steps: 1. Constructing a recurrent neural network containing an input layer, hidden layers, and an output layer, and dividing the hidden layer into several modules; 2. Constructing different modular recurrent neural networks through different combinations of four hidden layer module update strategies and two recurrent connection pruning strategies; 3. Performing Z-score standardization on the acquired power load data and dividing it into training, validation, and test sets according to time sequence. The model is trained using the training set, tuned using the validation set, and finally evaluated on the test set. Compared with the widely used gated recurrent neural networks, the model under this framework can achieve accurate prediction of power load while effectively reducing the number of network training parameters. In particular, the modular recurrent neural network based on the adaptive update strategy performs the best. However, this scheme takes into account the insufficient power edge resources, and the performance of the pruned model is insufficient for deployment at the edge. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for compressing recurrent neural network models for edge computing.

[0008] The objective of this invention can be achieved through the following technical solutions: A compression method for recurrent neural network models for edge computing, comprising the following steps: Step 1: Construct a recurrent neural network model and train the recurrent neural network model based on historical power time series data to obtain the teacher model; Step 2: Perform a compression process on the teacher model using a pre-designed compression configuration scheme to obtain multiple indicators of the compressed teacher model; based on the multiple indicators, iteratively optimize the compression configuration scheme using a multi-objective evolutionary algorithm and a pre-constructed multi-dimensional optimization objective function to obtain the optimal compression configuration scheme; repeat the compression process using the optimal compression configuration scheme to obtain the student model; Step 3: Use the feature distribution output by the teacher model to perform distillation training on the student model, and deploy the distilled student model on the edge node; based on the real-time data obtained after deployment, use a sensitivity analysis mechanism to iteratively optimize the student model or compression configuration scheme.

[0009] Furthermore, the compression process specifically includes: The redundant units and low-contribution parameters in the teacher model are identified by a structure-aware channel importance assessment method, and then the redundant units and low-contribution parameters are removed. Finally, the parameters are quantized to compress the teacher model.

[0010] Furthermore, specific methods for assessing the importance of channels in a structure-aware manner include: Let the set of the number of hidden units in the loop layer be . Define the structural awareness importance index This is used to measure the contribution of each hidden unit to the output of the teacher model, and utilizes gradient integral, activation mean method, or... The norm is used to solve for the structure-aware importance index; According to the preset compression ratio , , keep before The most important hidden unit is selected, and the remaining hidden units are removed; the parameter quantization specifically includes: using K-Means clustering to quantize the floating-point weights into 8-bit integer form.

[0011] Furthermore, the training process specifically includes: The recurrent neural network model is an LSTM model; the training objective of the LSTM model is to minimize the prediction loss. in, Indicates based on parameter set Given the input sequence from time 1 to time t, the model outputs the predicted value at time t. Indicates the predicted value. For parameter set, This represents the historical power time series input data sequence from time 1 to time t. This means minimizing the model's prediction loss within the feasible region of the parameter set θ.

[0012] Furthermore, the training process also includes: While minimizing the prediction loss, a weight sparsity constraint is embedded, specifically: in, This represents the total loss of the model. This represents the mean squared error loss between the model's predicted values ​​and the actual values. For hyperparameters, This is the weight parameter matrix of the hidden layer of the model. The total number of channels to be filtered in the model. For channel gating variables.

[0013] Furthermore, the distillation training process specifically includes: Distillation training is performed by matching the feature distributions of the output and intermediate layers of the teacher and student models, with the loss function being: in, Let x be the feature representation of the l-th layer of the teacher model outputting the input data x. These are the weighting coefficients; The prediction loss is the difference between the student model's predicted values ​​and the actual values. The loss is the difference in feature distribution between the teacher model and the student model. This represents the true label value of the power time series data. This represents the final predicted value output by the student model for the input data x. This is the feature representation of the l-th layer of the student model for the input data x.

[0014] Furthermore, the multiple indicators include the prediction error, number of model parameters, and model inference latency of the compressed teacher model; the multi-dimensional optimization objective function is: Wherein, C represents the compression configuration scheme; The prediction error of the compressed teacher model, For the number of model parameters, For model inference delay, Optimize the objective function from multiple dimensions.

[0015] Furthermore, the multi-objective evolutionary algorithm is a genetic algorithm, and the process of iteratively optimizing the compressed configuration scheme specifically includes: The compressed configuration scheme is encoded as a chromosome vector in a genetic algorithm. A single-point crossover strategy and site mutation are used to continuously generate new compressed configuration schemes with different parameters. Finally, a roulette wheel algorithm is used to filter all compressed configuration schemes based on their fitness values. When the maximum number of generations or fitness convergence is reached, the selection process stops and the optimal compression configuration scheme is obtained.

[0016] Furthermore, the sensitivity analysis mechanism specifically includes: Obtain the variation range of key parameters in the student model, namely hidden layer size, quantization bit width, and pruning ratio; evaluate the impact of different compression configuration schemes on inference accuracy, energy consumption, and latency, and construct performance-resource curves; When system resource fluctuations are detected during operation, the system first determines whether the student model needs structural optimization or the compression configuration scheme needs adjustment based on pre-set online indicators. If the compression configuration scheme needs adjustment, it is adjusted according to the performance-resource curve, and a new student model is deployed on the edge node.

[0017] The present invention also provides a recurrent neural network model compression system for edge computing, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods described above.

[0018] Compared with the prior art, the present invention has the following advantages: (1) This invention combines structure-aware pruning, knowledge distillation and multi-objective optimization methods on the basis of traditional time series model compression methods for the first time. It is applicable to the power edge computing scenario and achieves a balance between model volume, inference latency and accuracy. By introducing a structural importance assessment mechanism: redundant units and low-contribution parameters in the recurrent neural network model are identified and removed through a structure-aware channel importance assessment method, and then the parameters are quantified. This method achieves the ability to more accurately preserve key neurons while compressing recurrent neural network models, thereby improving the performance of the compressed recurrent neural network models.

[0019] (2) This invention introduces knowledge distillation after compressing the original recurrent neural network model: By matching the feature distributions of the output and intermediate layers of the original recurrent neural network (RNN) model and the compressed RNN model, distillation training is performed. This transfers the performance of the original RNN model (the teacher model) to the compressed RNN model (the student model), thus compensating for the loss of information and model performance caused by compression. Knowledge distillation compensates for the information loss caused by compression, maintaining the predictive ability of the compressed model and enabling its deployment in resource-constrained power edge scenarios.

[0020] (3) This invention utilizes a multi-objective optimization search compression scheme, namely, the coordinated use of multi-objective optimization functions and genetic algorithms, to achieve accurate selection of the optimal compression scheme and the best deployment model; and combined with online sensitivity analysis, the model structure can be dynamically adjusted according to the resources of edge devices, thereby improving the adaptability and energy efficiency of the system, and has significant engineering practical value and economic benefits. Attached Figure Description

[0021] Figure 1 This is a flowchart of a recurrent neural network model compression method for edge computing provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for compressing recurrent neural network models for edge computing, as provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] Example 1 like Figure 1 As shown, this embodiment provides a method for compressing recurrent neural network models for edge computing, including: The specific process of establishing the recurrent neural network compression model is as follows: S1: Construct a recurrent neural network model and train the recurrent neural network model based on historical power time series data to obtain the teacher model; Specifically, Constructing the original time-series model: A recurrent neural network model (RNN / LSTM / GRU) is constructed based on historical power time-series data and trained to obtain a baseline model with high prediction accuracy. The recurrent neural network model compression object in this invention is the original trained time-series model. The model structure includes an input layer, one or more hidden recurrent layers (such as LSTM / GRU), and an output layer. The model compression goal is to maintain the modeling accuracy and time-series dependency capability of power time-series data while reducing the model parameter size, computational complexity, and inference latency.

[0026] Preferred, Let the original training data be a time series set. Each Indicates time t The power system state is input, and the corresponding predicted value is output. The original model is constructed using a standard LSTM network. The parameter set is The model training objective is to minimize the prediction loss: S2: Perform a compression process on the teacher model using a pre-designed compression configuration scheme and obtain multiple indicators of the compressed teacher model; based on the multiple indicators, use a multi-objective evolutionary algorithm and a pre-built multi-dimensional optimization objective function to iteratively optimize the compression configuration scheme to obtain the optimal compression configuration scheme; repeat the compression process using the optimal compression configuration scheme to obtain the student model. S201: Design a structure-aware compression strategy: Combining the periodicity and stability characteristics of power load data, identify redundant units and low-contribution parameters in the model through the channel importance assessment method of structure awareness, and form an initial model pruning strategy. Specifically, Let the set of the number of hidden units in the loop layer be . Define the structural awareness importance index This is used to measure the contribution of each hidden unit to the model output. Common methods include gradient integral, activation mean, or... Norm.

[0027] Based on the set compression ratio , keep before The hidden unit with the highest importance ranking is selected, and the remaining units are pruned. The pruned model is denoted as... ; Preferred, Building a teacher model and student model ,in For the original model, This is the pruned or compressed model. Distillation training is performed by matching the feature distributions of the output layer and intermediate layers, with the loss function being: in The model represents the first Layer feature output, These are the weighting coefficients; S202: Constructing a multi-objective optimization framework: Based on a multi-objective evolutionary algorithm, an optimization objective function is established by integrating three dimensions: model compression rate, prediction accuracy, and inference latency. An efficient compression scheme search algorithm suitable for edge computing scenarios is designed to generate the optimal compression model configuration under resource-constrained scenarios. Specifically, To achieve the optimal balance between model accuracy, model size, and inference speed, a multi-objective evolutionary algorithm (such as NSGA-II) is introduced, with the following objective function designed: Prediction error of the compressed model Model parameter count Model inference latency (or unit sample processing time) The optimization objective is: Where C represents the compression configuration scheme. The compressed configuration scheme is encoded as chromosome vectors in a genetic algorithm, where each gene represents the retention ratio or activation type of a certain layer. An iterative search is then performed using a genetic algorithm. Encoding method: Use binary encoding or real number encoding, and determine the chromosome length according to the number of model layers; Crossover operator: Employs a single-point crossover strategy; Mutation operator: uses site mutation, with a probability set to ; Selection strategy: Use the roulette wheel algorithm to select individuals to enter the next generation based on their fitness values; Termination condition: Reaching the maximum number of generations or fitness convergence.

[0028] S3: Use the feature distribution output by the teacher model to distill and train the student model, and deploy the distilled student model to the edge node; based on the real-time data obtained after deployment, use the sensitivity analysis mechanism to iteratively optimize the student model or compression configuration scheme.

[0029] S301: Introducing a knowledge-guided compression mechanism: Constructing a lightweight student model and using the feature distribution output by the teacher model for distillation training to improve the learning ability and generalization performance of the compressed model, thereby achieving effective transfer of knowledge from the original model. S302: Deployment and Dynamic Adjustment Mechanism: The compressed model is deployed on edge nodes, and inference performance and system resource usage are monitored in real time with data streams. The model structure is dynamically adjusted through a lightweight incremental update strategy to achieve long-term online optimization and adaptive compression.

[0030] Specifically, To improve the model's adaptability in edge environments, a lightweight sensitivity analysis mechanism is introduced: Obtain the range of variation for key parameters (such as hidden layer size, quantization bit width, and pruning ratio). Evaluate the impact of different compression configurations on inference accuracy, energy consumption, and latency, and construct performance-resource curves. ; The optimal compression strategy is selected using this curve and deployed on edge nodes. During operation, system resource fluctuations (such as CPU utilization and power consumption) are monitored, and online indicators are used to determine whether to trigger model structure switching or compression strategy adjustment, thereby achieving adaptive and efficient compression in edge scenarios.

[0031] Example 2 This embodiment provides a recurrent neural network model compression system for edge computing, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods in Embodiment 1.

[0032] like Figure 2 As shown, an efficient compression method for temporal recurrent neural network models for power edge computing scenarios is proposed. This method optimizes the compression of typical substation multi-channel voltage, current, and frequency data (such as substation monitoring data with a sampling rate of 10Hz for 30 consecutive days). The specific steps include: Step 1, Original Model Construction: A standard LSTM model is used as the baseline model structure, with input in the form of... The time-series power data is used, where T=864000 is the data length of one day, F=6 is the number of channels (such as three-phase voltage and three-phase current), and the output is the prediction result for the next time step.

[0033] Step 2, edge-aware sparse training: During the original model training process, weight sparsity constraints are embedded. The specific method is as follows: During each round of weight updates, the hidden layer weight matrix is ​​adjusted. Apply an L1 norm penalty term; Add a soft gate mechanism to filter key input features channel by channel; The sparsity control formula is: in For hyperparameters, For channel gating variables.

[0034] Step 3, structural pruning and parameter quantization, involves applying a structural pruning algorithm to the sparsely trained model. This includes: deleting connections with weights close to 0; removing redundant LSTM units; and quantizing floating-point weights into 8-bit integers using K-Means clustering. The pruning rate was set to 30%, and the quantization error was controlled within 5% of the original model's MSE. In the experiment, the model size was compressed from the original 15.6MB to 2.1MB, achieving a compression rate of 86.5%.

[0035] Step 4, Edge device deployment and testing: Deploy the compressed model to the edge computing terminal (such as Raspberry Pi 4B with 2GB of memory) in the power distribution room, and test it on the real-time power data stream. The latency should be controlled within 30ms, and the prediction accuracy should be kept within the error range of the original model (such as RMSE of 0.0193).

[0036] Step 5, interpretability sensitivity analysis, analyzes the importance of different channel features and evaluates their sensitivity to prediction errors. The results are shown in Table 1: Definitions: Incremental update strategy: This is an efficient model maintenance mechanism that refers to downloading, transmitting, or learning only the changed parts of the model (such as weight differences, new parameters, or gradient fine-tuning for new data) on edge devices, rather than replacing the entire model file. It achieves rapid iteration and adaptation of deployed models by minimizing network data transmission and local computational overhead, thereby balancing model performance, resource consumption, and update timeliness in dynamic edge environments.

[0037] Gradient integral is an interpretable method used to evaluate the importance of neurons or parameters in a neural network. It quantifies the contribution of a neuron to the final prediction by calculating the integral (or expectation) of the gradient of the model's output relative to a specific neuron (such as a hidden unit) over the input sample. Specifically, it accumulates gradients along the path from the input sample to the baseline, thus capturing the combined influence of neurons throughout the decision-making process. It is often used in pruning decisions during model compression to identify and retain key components that significantly affect the output.

[0038] Activation mean method: This is a structured evaluation method for measuring the importance of hidden units in a neural network. It involves running a trained model on a validation dataset, calculating the average activation value of each hidden unit's output, and using this average as a quantitative indicator of the unit's importance. Its core assumption is that units with higher average activation values ​​participate more actively in the model's inference process, contribute more to the network's output, and are therefore more important; conversely, low activation values ​​are considered redundant. This method is often used for initial screening before model pruning to identify and remove neurons or channels with low contribution, thereby achieving model compression.

[0039] Norm: Also known as the Manhattan norm, it is the sum of the absolute values ​​of all elements in a vector space. In machine learning, the L1 norm is often used for regularization (Lasso regularization) to produce sparse solutions. By penalizing the sum of the absolute values ​​of model weights, it drives unimportant weights to approach zero, thereby achieving feature selection or model compression. In neural network pruning, the size of the L1 norm of the weight matrix is ​​often used as an indicator of the importance of neurons or channels—the smaller the norm, the more likely the set of parameters is to be redundant and can be pruned.

[0040] NSGA-II (Non-dominated sorting genetic algorithm II) is a classic multi-objective evolutionary optimization algorithm used to find the optimal solution set (Pareto front) when there are multiple conflicting objectives. It stratifies the population according to quality through non-dominated sorting and introduces a crowding comparison operator to maintain the diversity and distribution of the solution set, efficiently generating a set of balanced optimal solutions in a single run. In model compression, NSGA-II is often used to automatically search for compression configurations that simultaneously optimize accuracy, size, and speed, achieving automatic trade-offs among multiple objectives.

[0041] Single-point crossover strategy: This is a fundamental operation in genetic algorithms that simulates gene recombination. This strategy randomly selects a pair of parent chromosomes, determines an identical split point (gene locus), and then exchanges all gene segments after that point between the two parents, thereby generating two new offspring individuals. This method preserves some structural features of the parents and explores new combinations of solutions through segment recombination, making it one of the core mechanisms for maintaining population diversity and driving evolution. In model compression configuration search, it is used to mix local features of different excellent configurations to generate potentially better new configurations.

[0042] Genetic algorithms are metaheuristic optimization algorithms that simulate the mechanisms of natural selection and heredity in biological evolution. They encode potential solutions to a problem as "chromosomes" and simulate biological evolutionary operations such as selection, crossover, and mutation. Through iterative processes, they continuously filter, recombine, and fine-tune the population, allowing individuals with higher fitness to be preserved and reproduce, thus gradually approaching the optimal solution to complex problems. This algorithm is particularly suitable for solving high-dimensional, nonlinear, and multi-objective optimization problems, such as automatically searching for the best hyperparameter configuration combination in model compression.

[0043] Locus mutation: This is a fundamental operation in genetic algorithms. It randomly changes the value of one or more gene loci on an individual's chromosome with a predetermined small probability (such as flipping a 0 in binary code to a 1, or applying a small random perturbation to a value in real number code). This mechanism increases the diversity of the population by introducing randomness, helping the algorithm escape local optima and explore new solution spaces. It is a key step in maintaining evolutionary vitality and discovering potentially better solutions.

[0044] Roulette wheel selection is a selection strategy within genetic algorithms. It assigns a selection probability based on the proportion of each individual's fitness value to the total fitness of the population (higher fitness results in a higher probability). The algorithm simulates the random process of a roulette wheel spinning, repeatedly selecting individuals as parents according to this probability distribution. This ensures that superior individuals have a greater chance of being retained and reproduced, while maintaining population diversity and guiding the population towards better evolution. It is a classic method for mathematically representing the principle of natural selection, "survival of the fittest."

[0045] K-Means clustering is a classic unsupervised machine learning algorithm used to divide a dataset into K mutually exclusive clusters. It iteratively optimizes the cluster assignment by calculating the distance of each data point to the centroid of each cluster and assigning it to the nearest cluster. The cluster centroids are then recalculated until the cluster assignments are stable. In model quantization, K-Means is used to cluster full-precision weights into K classes (e.g., 256 classes corresponding to 8-bit quantization), using the centroid of each cluster to represent all weights in that class. This achieves efficient compression by replacing the original values ​​with indices, significantly reducing model storage and computational overhead.

[0046] MSE: Mean Squared Error, is one of the most commonly used loss functions and performance evaluation metrics in regression tasks. It calculates the mean of the squared differences between the model's predicted values ​​and the actual values.

[0047] RMSE: Root Mean Square Error, is a commonly used performance evaluation metric in regression problems. It is the square root of MSE. RMSE retains the same dimensions as the original data, more intuitively reflecting the average magnitude of the prediction error, and is more sensitive to larger errors. It is often used to measure the accuracy of model predictions in scenarios such as time series forecasting and signal processing.

[0048] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for compressing recurrent neural network models for edge computing, characterized in that, include: Step 1: Construct a recurrent neural network model and train the recurrent neural network model based on historical power time series data to obtain the teacher model; Step 2: Perform a compression process on the teacher model using a pre-designed compression configuration scheme to obtain multiple indicators of the compressed teacher model; based on the multiple indicators, iteratively optimize the compression configuration scheme using a multi-objective evolutionary algorithm and a pre-constructed multi-dimensional optimization objective function to obtain the optimal compression configuration scheme; repeat the compression process using the optimal compression configuration scheme to obtain the student model; Step 3: Use the feature distribution output by the teacher model to perform distillation training on the student model, and deploy the distilled student model on the edge node; based on the real-time data obtained after deployment, use a sensitivity analysis mechanism to iteratively optimize the student model or compression configuration scheme.

2. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The compression process specifically includes: The redundant units and low-contribution parameters in the teacher model are identified by a structure-aware channel importance assessment method, and then the redundant units and low-contribution parameters are removed. Finally, the parameters are quantized to compress the teacher model.

3. The method for compressing a recurrent neural network model for edge computing according to claim 2, characterized in that, The structure-aware channel importance assessment method specifically includes: Let the set of the number of hidden units in the loop layer be . Define the structural awareness importance index This is used to measure the contribution of each hidden unit to the output of the teacher model, and utilizes gradient integral, activation mean method, or... The norm is used to solve for the structure-aware importance index; According to the preset compression ratio , , keep before The most important hidden unit is selected, and the remaining hidden units are removed; the parameter quantization specifically includes: using K-Means clustering to quantize the floating-point weights into 8-bit integer form.

4. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The training process specifically includes: The recurrent neural network model is an LSTM model; the training objective of the LSTM model is to minimize the prediction loss. in, Indicates based on parameter set Given the input sequence from time 1 to time t, the model outputs the predicted value at time t. Indicates the predicted value. For parameter set, This represents the historical power time series input data sequence from time 1 to time t. This means minimizing the model's prediction loss within the feasible region of the parameter set θ.

5. The method for compressing a recurrent neural network model for edge computing according to claim 4, characterized in that, The training process also includes: While minimizing the prediction loss, a weight sparsity constraint is embedded, specifically: in, This represents the total loss of the model. This represents the mean squared error loss between the model's predicted values ​​and the actual values. For hyperparameters, This is the weight parameter matrix of the hidden layer of the model. The total number of channels to be filtered in the model. For channel gating variables.

6. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The distillation training process specifically includes: Distillation training is performed by matching the feature distributions of the output and intermediate layers of the teacher and student models, with the loss function being: in, Let x be the feature representation of the l-th layer of the teacher model outputting the input data x. These are the weighting coefficients; The prediction loss is the difference between the student model's predicted values ​​and the actual values. The loss is the difference in feature distribution between the teacher model and the student model. This represents the true label value of the power time series data. This represents the final predicted value output by the student model for the input data x. This is the feature representation of the l-th layer of the student model for the input data x.

7. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The aforementioned multiple indicators are the prediction error, model parameter count, and model inference latency of the compressed teacher model; the optimization expression of the multi-dimensional optimization objective function is: Wherein, C represents the compression configuration scheme; The prediction error of the compressed teacher model, For the number of model parameters, For model inference delay, Optimize the objective function from multiple dimensions.

8. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The multi-objective evolutionary algorithm is a genetic algorithm, and the process of iteratively optimizing the compressed configuration scheme specifically includes: The compressed configuration scheme is encoded as a chromosome vector in a genetic algorithm. A single-point crossover strategy and site mutation are used to continuously generate new compressed configuration schemes with different parameters. Finally, a roulette wheel algorithm is used to filter all compressed configuration schemes based on their fitness values. When the maximum number of generations or fitness convergence is reached, the selection process stops and the optimal compression configuration scheme is obtained.

9. The method for compressing a recurrent neural network model for edge computing according to claim 1, characterized in that, The sensitivity analysis mechanism specifically includes: Obtain the variation range of key parameters in the student model, namely hidden layer size, quantization bit width, and pruning ratio; evaluate the impact of different compression configuration schemes on inference accuracy, energy consumption, and latency, and construct performance-resource curves; When system resource fluctuations are detected during operation, the system first determines whether the student model needs structural optimization or the compression configuration scheme needs adjustment based on pre-set online indicators. If the compression configuration scheme needs adjustment, it is adjusted according to the performance-resource curve, and a new student model is deployed on the edge node.

10. A compression system for recurrent neural network models for edge computing, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images