Lightweight model edge deployment optimization method based on incremental learning
By employing incremental parameter updates, structured pruning, and mixed precision quantization on edge devices, combined with dynamic resource control, the inference accuracy and stability issues of lightweight models in edge computing environments are resolved, enabling adaptive optimization and continuous stable operation of the model in resource-constrained environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANNING CHAOSEN TECHNOLOGY CO LTD
- Filing Date
- 2025-12-16
- Publication Date
- 2026-06-26
AI Technical Summary
In edge computing environments, existing lightweight models suffer from decreased inference accuracy and degraded feature representation when faced with changes in data distribution. They also lack resource adaptability and stability. Incremental learning methods are difficult to implement on edge devices, and data management strategies lack dynamic filtering mechanisms, resulting in unstable model operation.
By employing incremental parameter updates, structured pruning, hybrid precision quantization, and dynamic resource regulation techniques, and by constructing a data cache sequence, monitoring the resource status of edge devices, adaptively updating and optimizing the model, maintaining the stability of the backbone structure, dynamically adjusting the model structure and accuracy, and eliminating abnormal data fragments, this approach achieves the goal of maintaining the stability of the backbone structure, dynamically adjusting the model structure and accuracy, and eliminating abnormal data fragments.
This ensures that the model maintains its feature representation capability on edge devices under varying data distribution conditions, improves operational stability and resource utilization efficiency, and guarantees the training quality and long-term stability of the incremental learning process.
Smart Images
Figure CN121660113B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of edge deployment technology, and in particular to a lightweight model edge deployment optimization method based on incremental learning. Background Technology
[0002] In edge computing scenarios, model inference tasks typically need to run stably for extended periods on devices with limited computing power, memory, and energy consumption. Existing technologies generally perform inference by pre-training lightweight models offline and deploying them to the edge. However, because edge environment data has significant temporal variation characteristics, its distribution is prone to gradual shifts with changes in scene, sensing mode, or device status. Lightweight models with fixed structures often experience problems such as decreased inference accuracy, degraded feature representation, and model failure when facing continuously changing data environments.
[0003] Existing incremental learning techniques can improve a model's adaptability to new data distributions in server or cloud environments through parameter expansion or local updates. However, directly applying these methods to edge devices presents significant limitations. On one hand, edge devices struggle to handle the high-intensity gradient updates and network structure adjustments required in traditional incremental learning, easily leading to increased latency and resource consumption. On the other hand, existing incremental update processes lack constraints on the structural characteristics of lightweight models, making the model backbone susceptible to disturbances during updates, resulting in decreased structural stability and further impacting the reliability of inference results.
[0004] Furthermore, current data management methods generally employ static caching strategies, lacking dynamic filtering mechanisms for anomalous, drifting, or low-value data fragments generated during inference. This results in a chaotic dataset structure used for training during incremental updates, failing to effectively reflect the real-time data characteristics at the edge. Simultaneously, existing technologies lack real-time monitoring mechanisms for edge device resources, failing to adaptively adjust model quantization accuracy, pruning ratios, or inference paths based on processor utilization, memory usage, and inference latency. This leads to performance fluctuations or unstable operation of the model in resource-constrained environments.
[0005] Therefore, how to provide a lightweight model edge deployment optimization method based on incremental learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose a lightweight model edge deployment optimization method based on incremental learning. This invention fully utilizes incremental parameter updates, structured pruning, mixed precision quantization, and dynamic resource regulation techniques. It describes in detail a data caching iteration mechanism that enables adaptive model updates, lightweight optimization, and runtime-driven processing in resource-constrained edge environments. This method has the advantages of strong model stability, high resource adaptability, and consistently reliable inference performance.
[0007] The edge deployment optimization method for lightweight models based on incremental learning according to embodiments of the present invention includes the following steps:
[0008] S1. Collect edge-side data and preprocess it, construct a data cache sequence and generate a dataset to be processed according to the set format;
[0009] S2. Load the lightweight base model, freeze the main parameters of the model, set the expandable parameter section in the model structure, and complete the initial configuration of the model.
[0010] S3. Extract the feature distribution index of the dataset to be processed, measure the difference with the existing feature distribution of the model, label the feature change segments according to the measurement results, and generate trigger identifier sequences.
[0011] S4. Based on the trigger identifier sequence, incremental updates are performed on the expandable parameter segment. During the update process, the stable weight state of the backbone segment is maintained, and the feature replay method is used to calibrate the feature offset generated during the update process.
[0012] S5. Based on the importance of the weights, perform structured pruning on the updated model structure, and use a mixed precision quantization method to perform multi-level lightweight processing to generate a lightweight updated model.
[0013] S6. Monitor the resource status of edge devices and optimize the quantization accuracy, pruning ratio and inference configuration of the lightweight update model according to the resource status.
[0014] S7. Deploy the optimized and adjusted lightweight update model to the edge-side inference execution path, record the running status of the inference output, and iteratively update the data cache sequence.
[0015] Optionally, the construction of the dataset to be processed includes:
[0016] The collected edge data is parsed, missing values are filled, and values are normalized. The normalized data is segmented and sliced according to time order, and the sliced data fragments are written into the data cache sequence.
[0017] Feature extraction is performed on data fragments in the data cache sequence to generate corresponding feature records;
[0018] Data fragments and feature records are combined according to the set data structure rules and encoded according to a fixed field order to form a dataset to be processed.
[0019] Optionally, S2 specifically includes:
[0020] S21. Load the lightweight base model, perform structural analysis on each network layer unit of the model, and extract the input dimension, output dimension and weight parameter set of each network layer unit in hierarchical order.
[0021] S22. Set training prohibition flags for each item in the weight parameter set, close the gradient calculation channel of the corresponding network layer unit, and skip the update calculation of the marked weight parameters during the training phase of the model.
[0022] S23. Construct linear mapping units for scalable parameter segments and insert them between adjacent network layer units. Determine the shape of the mapping matrix based on the output dimension of the previous network layer unit and the input dimension of the next network layer unit, and add the linear mapping units to the forward propagation path of the model.
[0023] S24. Initialize the sparse weights of the mapping matrix of the expandable parameter segment, generate non-zero parameter positions according to the preset sparsity using the grouped sparse coding method, and write them into the mapping matrix of the expandable parameter segment.
[0024] S25. Set the mapping matrix of the scalable parameter segment to a trainable state, participate in gradient calculation and parameter update during the incremental update phase.
[0025] Optionally, S3 specifically includes:
[0026] S31. Extract the feature vectors of each data segment in the dataset to be processed, and use the kernel density estimation method to calculate the feature probability density function to obtain the corresponding feature density sequence.
[0027] S32. Estimate the kernel density of the existing feature distribution of the model in the same dimension, and perform normalization to generate a reference density sequence;
[0028] S33. The Wasserstein distance algorithm is used to calculate the distribution difference between the feature density sequence and the reference density sequence, and the distribution difference values are arranged in chronological order to form a difference measurement sequence.
[0029] S34. Dynamically update the threshold of the difference measure sequence, and perform segment scanning on the updated difference measure sequence to define continuous data intervals that cross the threshold as feature change intervals, specifically including:
[0030] The difference measure sequence is segmented by a sliding window, and the local mean, local standard deviation and local fractions are calculated in each window. An initial threshold sequence is then constructed based on the local mean and local fractions.
[0031] The initial threshold sequence is subjected to bidirectional exponential smoothing to generate a forward smoothed threshold sequence and a backward smoothed threshold sequence, respectively. The two sequences are then fused element-wise to obtain a dynamic threshold update sequence.
[0032] The difference measurement sequence and the dynamic threshold update sequence are differentially calculated item by item, and the sign of the difference results is determined to generate a cross-threshold label sequence.
[0033] For the cross-threshold labeled sequence, continuous segment detection is performed, and the start and end positions of the continuous labeled segments are recorded using the run-length encoding method;
[0034] The continuous marked segments of the records are filtered by segment length, segments that are lower than the set length threshold are removed, and the filtered continuous segments are defined as feature change segments.
[0035] S35. Number the characteristic change segments and encode the numbering results into a trigger identifier sequence.
[0036] Optionally, S33 specifically includes:
[0037] S331. Based on the feature density sequence and the reference density sequence, construct discrete probability vector groups respectively, use Euclidean distance to calculate the distance relationship between vectors, generate the corresponding cost matrix set, and perform dimension clipping and matrix alignment on the cost matrix set.
[0038] S332. Construct an initial transmission matrix for regularized optimal transmission based on the cost matrix set, and update it cyclically in the order of row normalization and column normalization using the Sinkhorn-Knopp method.
[0039] S333. Introduce a stabilization factor to correct the numerical stability of the row normalization and column normalization results of each iteration, and determine the convergence status based on the corrected iteration matrix.
[0040] S334. After the convergence condition is met, the final iteration matrix and the cost matrix set are multiplied element-wise and the product matrix is summed globally to obtain the Wasserstein distance sequence arranged in time order.
[0041] S335. Perform segment smoothing, sequence denoising, and fluctuation suppression on the Wasserstein distance sequence to generate a difference measure sequence.
[0042] Optionally, S4 specifically includes:
[0043] S41. Based on the trigger identifier sequence, extract the corresponding feature vector sequence in the dataset to be processed, and perform forward and backward gradient propagation in the scalable parameter segment to obtain the gradient vector of the scalable parameter segment.
[0044] S42. Calculate the orthogonal components of the gradient vector in the historical gradient space using the orthogonal gradient method, and use the orthogonal components as the update direction of the scalable parameter segment;
[0045] S43. Calculate the Fisher information estimates of each parameter in the scalable parameter segment based on the gradient vector, perform weighted shrinkage processing on the update direction, and generate a stable update vector.
[0046] S44. Update the parameters of the expandable parameter segment based on the stable update vector, keep the weight parameters of the backbone segment in a frozen state, and record the feature vectors before and after the update.
[0047] S45. Construct a feature playback matrix, map the feature vectors before and after the update into the corresponding positions in the feature playback matrix, perform feature alignment and calculate the offset vector, and perform projection correction and amplitude compensation on the offset vector, specifically including:
[0048] Write the feature vectors before and after the update into the corresponding positions of the feature playback matrix, perform L2 normalization on each feature vector, and construct the feature covariance matrix.
[0049] Principal component analysis is performed on the feature covariance matrix to extract the principal component feature subspace, and the feature vectors before and after the update are projected onto the feature subspace.
[0050] Normalized cross-correlation calculation is performed on the feature vectors in the feature subspace, and the vector positions in the feature playback matrix are calibrated and aligned based on the cross-correlation peak positions.
[0051] The offset vector is obtained by performing element-wise difference calculation on the aligned feature vector, and the offset vector is sparsely encoded to generate a sparse representation of the offset vector.
[0052] Affine transformation estimation is performed on the sparse representation, and the offset vector after projection correction is obtained by solving the affine transformation parameter matrix.
[0053] By calculating the correction coefficients of the vector at different amplitude scales in segments, the offset vector after projection correction is subjected to multi-scale amplitude compensation processing.
[0054] Optionally, S5 specifically includes:
[0055] S51. Based on the updated model parameters, divide the network into units, statistically analyze the weight parameters in each unit, calculate the mean weight and L2 norm of each channel, and generate a channel importance matrix.
[0056] S52. Based on the channel importance matrix, set an importance threshold, generate a structured pruning mask for channels with scores below the threshold, and use the pruning mask to block the corresponding network layer output channel unit.
[0057] S53. Perform statistical analysis on the weight parameter distribution of each network layer unit in the pruned model structure, specifically including:
[0058] Calculate the value fluctuation range and weight response change amplitude of each network layer unit, and sort the network layer units according to the change amplitude;
[0059] Network layer units with changes in the upper range of the ranking are classified as high-sensitivity layer groups, network layer units with changes in the middle range of the ranking are classified as medium-sensitivity layer groups, and network layer units with changes in the lower range of the ranking are classified as low-sensitivity layer groups.
[0060] Based on the impact of each layer on accuracy during the inference process, they are respectively labeled as high-precision requirement units, medium-precision requirement units, and low-precision requirement units.
[0061] Based on the accuracy requirements, the corresponding quantization bit width selection range is determined, and the quantization bit width, quantization step size and zero offset parameters are specified for each network layer unit to form a mixed precision quantization table.
[0062] S54. Load the three layers into their respective quantization paths and assign the corresponding quantization bit widths according to the mixed precision quantization table. Then perform quantization mapping, step size calibration, and zero-point offset calibration in sequence.
[0063] S55. Perform forward inference tests based on the calibrated model structure, record inference latency and memory usage, and determine the lightweight update model through preset resource constraints.
[0064] Optionally, S6 specifically includes:
[0065] S61. Periodically sample the processor utilization, memory usage and inference latency of edge devices, construct resource state vectors and perform time smoothing processing;
[0066] S62. Perform difference analysis on the resource status vector and the preset resource threshold, generate resource status tags for processor utilization, memory utilization and inference latency respectively, and perform dynamic threshold offset on the tags.
[0067] S63. Dynamically adjust the lightweight update model based on resource status markers, specifically including:
[0068] When the processor utilization status flag is in an over-limit state, the quantization precision of each network layer unit in the lightweight update model is replaced, and the corresponding quantization step size and zero offset parameters are updated after the replacement.
[0069] When the memory usage status flag is in an over-limit state, the pruning ratio of the lightweight update model is incrementally expanded, the validity of low-importance network layer units is re-evaluated, and newly added low-importance network layer units are hidden.
[0070] When the inference latency state flag is in an over-limit state, the inference configuration of the lightweight update model is restructured at the structural level by replacing high-complexity operators, merging adjacent network layer units and compressing intermediate features to adjust the inference path.
[0071] When all three state markers are within the normal range, the quantization accuracy, pruning ratio, and inference configuration of the lightweight update model remain unchanged, and the resource state vector is updated to enter the next sampling period.
[0072] Optionally, S7 specifically includes:
[0073] S71. Load the optimized and adjusted lightweight update model into the edge-side inference execution path, perform structural analysis on the operation nodes in the inference execution path, and establish a mapping relationship between the inference path and the model operation unit.
[0074] S72. During the inference process, the runtime latency, device occupancy and feature output stability of each computing node are sampled in real time, and fluctuation detection and anomaly localization are performed on the sampling results. The abnormal segments are associated with the corresponding data input segments to generate a running state sequence.
[0075] S73. Perform iterative updates on the data cache sequence based on the running status sequence, extract features from input segments marked as abnormal, and replace the corresponding positions in the data cache sequence.
[0076] The beneficial effects of this invention are:
[0077] First, by introducing scalable parameter segments and incremental update mechanisms into the lightweight model, this invention enables the model to maintain its feature representation ability even when the distribution of edge data continues to change, and prevents a significant decrease in inference accuracy due to disturbance of the main structure, thereby effectively solving the problems of easy model degradation and insufficient adaptability in the prior art.
[0078] Secondly, this invention enables the model to automatically perform adaptive optimization of accuracy and structure based on the processor utilization, memory usage and inference latency of edge devices through structured pruning, hybrid precision quantization and resource state-driven dynamic adjustment mechanism, which significantly improves the model's operational stability and resource utilization efficiency in computing-constrained environments.
[0079] Finally, this invention uses a data cache iteration update method driven by the running state sequence to automatically remove, reconstruct, and rearrange abnormal segments, drifting segments, and low-value segments, so that the data participating in incremental updates can continuously maintain temporal rationality and feature validity, thereby ensuring the training quality and long-term stability of the incremental learning process. Attached Figure Description
[0080] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0081] Figure 1 This is an overall flowchart of the lightweight model edge deployment optimization method based on incremental learning proposed in this invention;
[0082] Figure 2 This is a flowchart illustrating the scalable parameter segment configuration and incremental update process of the lightweight model edge deployment optimization method based on incremental learning proposed in this invention.
[0083] Figure 3 This is a flowchart illustrating the iterative update process of the data cache sequence for the lightweight model edge deployment optimization method based on incremental learning proposed in this invention. Detailed Implementation
[0084] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0085] refer to Figure 1-3 The lightweight model edge deployment optimization method based on incremental learning includes the following steps:
[0086] S1. Collect edge-side data and preprocess it, construct a data cache sequence and generate a dataset to be processed according to the set format;
[0087] S2. Load the lightweight base model, freeze the main parameters of the model, set the expandable parameter section in the model structure, and complete the initial configuration of the model.
[0088] S3. Extract the feature distribution index of the dataset to be processed, measure the difference with the existing feature distribution of the model, label the feature change segments according to the measurement results, and generate trigger identifier sequences.
[0089] S4. Based on the trigger identifier sequence, incremental updates are performed on the expandable parameter segment. During the update process, the stable weight state of the backbone segment is maintained, and the feature replay method is used to calibrate the feature offset generated during the update process.
[0090] S5. Based on the importance of the weights, perform structured pruning on the updated model structure, and use a mixed precision quantization method to perform multi-level lightweight processing to generate a lightweight updated model.
[0091] S6. Monitor the resource status of edge devices and optimize the quantization accuracy, pruning ratio and inference configuration of the lightweight update model according to the resource status.
[0092] S7. Deploy the optimized and adjusted lightweight update model to the edge-side inference execution path, record the running status of the inference output, and iteratively update the data cache sequence.
[0093] In this embodiment, the construction of the dataset to be processed includes:
[0094] The collected edge data is parsed, missing values are filled, and values are normalized. The normalized data is segmented and sliced according to time order, and the sliced data fragments are written into the data cache sequence.
[0095] Feature extraction is performed on data fragments in the data cache sequence to generate corresponding feature records;
[0096] Data fragments and feature records are combined according to the set data structure rules and encoded according to a fixed field order to form a dataset to be processed.
[0097] In this embodiment, S2 specifically includes:
[0098] S21. Load the lightweight base model, perform structural analysis on each network layer unit of the model, and extract the input dimension, output dimension and weight parameter set of each network layer unit in hierarchical order.
[0099] S22. Set training prohibition flags for each item in the weight parameter set, close the gradient calculation channel of the corresponding network layer unit, and skip the update calculation of the marked weight parameters during the training phase of the model.
[0100] S23. Construct linear mapping units for scalable parameter segments and insert them between adjacent network layer units. Determine the shape of the mapping matrix based on the output dimension of the previous network layer unit and the input dimension of the next network layer unit, and add the linear mapping units to the forward propagation path of the model.
[0101] S24. Initialize the sparse weights of the mapping matrix of the expandable parameter segment, generate non-zero parameter positions according to the preset sparsity using the grouped sparse coding method, and write them into the mapping matrix of the expandable parameter segment.
[0102] S25. Set the mapping matrix of the scalable parameter segment to a trainable state, participate in gradient calculation and parameter update during the incremental update phase.
[0103] In this embodiment, S3 specifically includes:
[0104] S31. Extract the feature vectors of each data segment in the dataset to be processed, and use the kernel density estimation method to calculate the feature probability density function to obtain the corresponding feature density sequence.
[0105] S32. Estimate the kernel density of the existing feature distribution of the model in the same dimension, and perform normalization to generate a reference density sequence;
[0106] S33. The Wasserstein distance algorithm is used to calculate the distribution difference between the feature density sequence and the reference density sequence, and the distribution difference values are arranged in chronological order to form a difference measurement sequence.
[0107] S34. Dynamically update the threshold of the difference measure sequence, and perform segment scanning on the updated difference measure sequence to define continuous data intervals that cross the threshold as feature change intervals, specifically including:
[0108] The difference measure sequence is segmented by a sliding window, and the local mean, local standard deviation and local fractions are calculated in each window. An initial threshold sequence is then constructed based on the local mean and local fractions.
[0109] The initial threshold sequence is subjected to bidirectional exponential smoothing to generate a forward smoothed threshold sequence and a backward smoothed threshold sequence, respectively. The two sequences are then fused element-wise to obtain a dynamic threshold update sequence.
[0110] The difference measurement sequence and the dynamic threshold update sequence are differentially calculated item by item, and the sign of the difference results is determined to generate a cross-threshold label sequence.
[0111] For the cross-threshold labeled sequence, continuous segment detection is performed, and the start and end positions of the continuous labeled segments are recorded using the run-length encoding method;
[0112] The continuous marked segments of the records are filtered by segment length, segments that are lower than the set length threshold are removed, and the filtered continuous segments are defined as feature change segments.
[0113] S35. Number the characteristic change segments and encode the numbering results into a trigger identifier sequence.
[0114] In this embodiment, S33 specifically includes:
[0115] S331. Based on the feature density sequence and the reference density sequence, construct discrete probability vector groups respectively, use Euclidean distance to calculate the distance relationship between vectors, generate the corresponding cost matrix set, and perform dimension clipping and matrix alignment on the cost matrix set.
[0116] S332. Construct an initial transmission matrix for regularized optimal transmission based on the cost matrix set, and update it cyclically in the order of row normalization and column normalization using the Sinkhorn-Knopp method.
[0117] S333. Introduce a stabilization factor to correct the numerical stability of the row and column normalization results for each iteration, and determine the convergence status based on the corrected iteration matrix. Specifically, this includes:
[0118] Calculate the element-level difference matrix between two adjacent iteration matrices, and perform matrix norm calculation on the difference matrix to obtain the corresponding convergence index sequence;
[0119] The convergence index sequence is compared with the set convergence threshold item by item, and the iteration positions below the threshold are marked as convergent.
[0120] The continuity of the iterative marker sequence is checked. When the number of iterations that continuously meet the convergence threshold reaches a set number, the iterative matrix sequence is determined to have converged.
[0121] If the convergence condition is not met after the number of iterations reaches the set upper limit, the iteration is terminated and the current iteration matrix is taken as the final result.
[0122] S334. After the convergence condition is met, the final iteration matrix and the cost matrix set are multiplied element-wise and the product matrix is summed globally to obtain the Wasserstein distance sequence arranged in time order.
[0123] S335. Perform segment smoothing, sequence denoising, and fluctuation suppression on the Wasserstein distance sequence to generate a difference measure sequence.
[0124] In this embodiment, S4 specifically includes:
[0125] S41. Based on the trigger identifier sequence, extract the corresponding feature vector sequence in the dataset to be processed, and perform forward and backward gradient propagation in the scalable parameter segment to obtain the gradient vector of the scalable parameter segment.
[0126] S42. Calculate the orthogonal components of the gradient vector in the historical gradient space using the orthogonal gradient method, and use the orthogonal components as the update direction of the scalable parameter segment;
[0127] S43. Calculate the Fisher information estimates of each parameter in the scalable parameter segment based on the gradient vector, perform weighted shrinkage processing on the update direction, and generate a stable update vector.
[0128] S44. Update the parameters of the expandable parameter segment based on the stable update vector, keep the weight parameters of the backbone segment in a frozen state, and record the feature vectors before and after the update.
[0129] S45. Construct a feature playback matrix, map the feature vectors before and after the update into the corresponding positions in the feature playback matrix, perform feature alignment and calculate the offset vector, and perform projection correction and amplitude compensation on the offset vector, specifically including:
[0130] Write the feature vectors before and after the update into the corresponding positions of the feature playback matrix, perform L2 normalization on each feature vector, and construct the feature covariance matrix.
[0131] Principal component analysis is performed on the feature covariance matrix to extract the principal component feature subspace, and the feature vectors before and after the update are projected onto the feature subspace.
[0132] Normalized cross-correlation calculation is performed on the feature vectors in the feature subspace, and the vector positions in the feature playback matrix are calibrated and aligned based on the cross-correlation peak positions.
[0133] The offset vector is obtained by performing element-wise difference calculation on the aligned feature vector, and the offset vector is sparsely encoded to generate a sparse representation of the offset vector.
[0134] Affine transformation estimation is performed on the sparse representation, and the offset vector after projection correction is obtained by solving the affine transformation parameter matrix.
[0135] By calculating the correction coefficients of the vector at different amplitude scales in segments, the offset vector after projection correction is subjected to multi-scale amplitude compensation processing.
[0136] In this embodiment, S5 specifically includes:
[0137] S51. Based on the updated model parameters, divide the network into units, statistically analyze the weight parameters in each unit, calculate the mean weight and L2 norm of each channel, and generate a channel importance matrix.
[0138] S52. Based on the channel importance matrix, set an importance threshold, generate a structured pruning mask for channels with scores below the threshold, and use the pruning mask to block the corresponding network layer output channel unit.
[0139] S53. Perform statistical analysis on the weight parameter distribution of each network layer unit in the pruned model structure, specifically including:
[0140] Calculate the value fluctuation range and weight response change amplitude of each network layer unit, and sort the network layer units according to the change amplitude;
[0141] Network layer units with changes in the upper range of the ranking are classified as high-sensitivity layer groups, network layer units with changes in the middle range of the ranking are classified as medium-sensitivity layer groups, and network layer units with changes in the lower range of the ranking are classified as low-sensitivity layer groups.
[0142] Based on the impact of each layer on accuracy during the inference process, they are respectively labeled as high-precision requirement units, medium-precision requirement units, and low-precision requirement units.
[0143] Based on the accuracy requirements, the corresponding quantization bit width selection range is determined, and the quantization bit width, quantization step size and zero offset parameters are specified for each network layer unit to form a mixed precision quantization table.
[0144] S54. Load the three layers into their respective quantization paths and assign the corresponding quantization bit widths according to the mixed precision quantization table. Then perform quantization mapping, step size calibration, and zero-point offset calibration in sequence.
[0145] S55. Perform forward inference tests based on the calibrated model structure, record inference latency and memory usage, and determine the lightweight update model through preset resource constraints.
[0146] In this embodiment, S6 specifically includes:
[0147] S61. Periodically sample the processor utilization, memory usage and inference latency of edge devices, construct resource state vectors and perform time smoothing processing;
[0148] S62. Perform difference analysis on the resource status vector and the preset resource threshold, generate resource status tags for processor utilization, memory utilization and inference latency respectively, and perform dynamic threshold offset on the tags.
[0149] S63. Dynamically adjust the lightweight update model based on resource status markers, specifically including:
[0150] When the processor utilization status flag is in an over-limit state, the quantization precision of each network layer unit in the lightweight update model is replaced, and the corresponding quantization step size and zero offset parameters are updated after the replacement.
[0151] When the memory usage status flag is in an over-limit state, the pruning ratio of the lightweight update model is incrementally expanded, the validity of low-importance network layer units is re-evaluated, and newly added low-importance network layer units are hidden.
[0152] When the inference latency state flag is in an over-limit state, the inference configuration of the lightweight update model is restructured at the structural level by replacing high-complexity operators, merging adjacent network layer units and compressing intermediate features to adjust the inference path.
[0153] When all three state markers are within the normal range, the quantization accuracy, pruning ratio, and inference configuration of the lightweight update model remain unchanged, and the resource state vector is updated to enter the next sampling period.
[0154] In this embodiment, S7 specifically includes:
[0155] S71. Load the optimized and adjusted lightweight update model into the edge-side inference execution path, perform structural analysis on the operation nodes in the inference execution path, and establish a mapping relationship between the inference path and the model operation unit.
[0156] S72. During the inference process, the runtime latency, device occupancy and feature output stability of each computing node are sampled in real time, and fluctuation detection and anomaly localization are performed on the sampling results. The abnormal segments are associated with the corresponding data input segments to generate a running state sequence.
[0157] S73. Perform iterative updates on the data cache sequence based on the running state sequence, extract features from input segments marked as abnormal, and replace the corresponding positions in the data cache sequence, specifically including:
[0158] The data cache sequence is divided into segments, and the segment priority is calculated based on timestamp, access frequency and feature sparsity.
[0159] For segments with low priority, remove the corresponding data fragments from the data cache sequence;
[0160] Perform feature reconstruction on the data segments associated with the inference state marker, perform feature consistency verification on the reconstructed data segments, and replace the corresponding segments in the cache;
[0161] The insertion position of the new data segment is calculated based on the timestamp and feature distribution. The new data segment is then inserted into the corresponding segment and the sorting structure of the adjacent segments is adjusted.
[0162] The data cache sequence after elimination, replacement and insertion operations is globally rearranged to verify the sequence temporal continuity, feature distribution continuity and cache capacity and generate a new data cache structure.
[0163] Perform an order consistency check on the updated data cache sequence. If the check passes, complete the update of the data cache sequence for this round.
[0164] Example 1:
[0165] To verify the feasibility of this invention in practice, it was applied to a type of intelligent terminal system that requires real-time processing of multi-source sensor data. This system is deployed long-term on edge devices with limited computing power, requiring continuous reception of high-frequency sensor feature inputs and the provision of real-time prediction and status assessment services. However, due to unstable environmental conditions, fluctuating sensor noise levels, and constantly changing operating loads, traditional fixed-structure lightweight models experience problems such as accumulated prediction bias, feature representation degradation, and inference latency fluctuations after a period of continuous operation, severely impacting system stability. This invention addresses these problems through incremental learning, structured pruning, mixed-precision quantization, and a state-driven data cache iteration update mechanism.
[0166] In real-world scenarios, edge devices' processors are consistently under high load. Due to significant shifts in input data distribution across different time periods, the feature layer output of fixed-structure models gradually becomes unstable. Applying this invention's method first constructs a data cache sequence and uses kernel density estimation and Wasserstein distance to monitor data distribution changes. When significant differences occur, scalable parameter segment updates are triggered, enabling the model to adapt to new data features promptly. During periods of high device resource fluctuation, this invention dynamically adjusts the model's quantization accuracy and pruning ratio by real-time sampling of device occupancy and inference latency, ensuring the model structure maintains stable inference speed under varying load conditions. Since abnormal and noisy segments exist in the input data, this invention uses runtime status evaluation to re-extract, reconstruct, and replace features from corresponding segments in the data cache sequence, ensuring the data participating in incremental updates maintains stable quality.
[0167] During continuous testing, the method of this invention effectively reduced the model parameter size while maintaining prediction accuracy, resulting in more stable inference speed. Comparative tests showed that the system without this method experienced an inference latency that increased to 1.85 times its original level after continuous operation, with a prediction error increase of approximately 34%. However, with this method, the average inference latency decreased by 27.4%, the maximum latency fluctuation decreased to 7.3%, and the prediction error decreased to 0.89 times its original level. The model parameter size decreased from 2.8M to 1.96M during operation, while maintaining model accuracy above 95%. The proportion of invalid segments in the data cache sequence decreased from approximately 21% to less than 5%, and the feature distribution offset significantly converged. The peak device resource usage decreased by approximately 19%, and the number of system restarts during long-running tasks approached zero.
[0168] As can be seen, this invention can effectively achieve continuous and stable operation of the model in edge devices. Through dynamic structural adjustment, incremental learning, and data cache reconstruction mechanisms, it maintains the model's adaptability in changing scenarios, enabling the model to maintain high accuracy, low latency, and high stability even under limited computing power. The runtime-driven edge deployment optimization system provided by this invention has significant engineering value and practical significance.
[0169] Table 1. Performance Comparison Data of Edge-Side Models
[0170]
[0171] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A lightweight model edge deployment optimization method based on incremental learning, characterized in that, Includes the following steps: S1. Collect edge-side data and preprocess it, construct a data cache sequence and generate a dataset to be processed according to the set format; S2. Load the lightweight base model, freeze the main parameters of the model, set the expandable parameter section in the model structure, and complete the initial configuration of the model. S3. Extract the feature distribution index of the dataset to be processed, measure the difference with the existing feature distribution of the model, label the feature change segments according to the measurement results, and generate trigger identifier sequences. S4. Based on the trigger identifier sequence, incremental updates are performed on the expandable parameter segment. During the update process, the stable weight state of the backbone segment is maintained, and the feature replay method is used to calibrate the feature offset generated during the update process. S4 specifically includes: S41. Based on the trigger identifier sequence, extract the corresponding feature vector sequence in the dataset to be processed, and perform forward and backward gradient propagation in the scalable parameter segment to obtain the gradient vector of the scalable parameter segment. S42. Calculate the orthogonal components of the gradient vector in the historical gradient space using the orthogonal gradient method, and use the orthogonal components as the update direction of the scalable parameter segment; S43. Calculate the Fisher information estimates of each parameter in the scalable parameter segment based on the gradient vector, perform weighted shrinkage processing on the update direction, and generate a stable update vector. S44. Update the parameters of the expandable parameter segment based on the stable update vector, keep the weight parameters of the backbone segment in a frozen state, and record the feature vectors before and after the update. S45. Construct a feature playback matrix, map the feature vectors before and after the update into the corresponding positions in the feature playback matrix, perform feature alignment and calculate the offset vector, and perform projection correction and amplitude compensation on the offset vector, specifically including: Write the feature vectors before and after the update into the corresponding positions of the feature playback matrix, perform L2 normalization on each feature vector, and construct the feature covariance matrix. Principal component analysis is performed on the feature covariance matrix to extract the principal component feature subspace, and the feature vectors before and after the update are projected onto the feature subspace. Normalized cross-correlation calculation is performed on the feature vectors in the feature subspace, and the vector positions in the feature playback matrix are calibrated and aligned based on the cross-correlation peak positions. The offset vector is obtained by performing element-wise difference calculation on the aligned feature vector, and the offset vector is sparsely encoded to generate a sparse representation of the offset vector. Affine transformation estimation is performed on the sparse representation, and the offset vector after projection correction is obtained by solving the affine transformation parameter matrix. By calculating the correction coefficients of the vector at different amplitude scales in segments, the offset vector after projection correction is subjected to multi-scale amplitude compensation processing. S5. Based on the importance of the weights, perform structured pruning on the updated model structure, and use a mixed precision quantization method to perform multi-level lightweight processing to generate a lightweight updated model. S6. Monitor the resource status of edge devices and optimize the quantization accuracy, pruning ratio and inference configuration of the lightweight update model according to the resource status. S7. Deploy the optimized and adjusted lightweight update model to the edge-side inference execution path, record the running status of the inference output, and iteratively update the data cache sequence.
2. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, The construction of the dataset to be processed includes: The collected edge data is parsed, missing values are filled, and values are normalized. The normalized data is segmented and sliced according to time order, and the sliced data fragments are written into the data cache sequence. Feature extraction is performed on data fragments in the data cache sequence to generate corresponding feature records; Data fragments and feature records are combined according to the set data structure rules and encoded according to a fixed field order to form a dataset to be processed.
3. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, S2 specifically includes: S21. Load the lightweight base model, perform structural analysis on each network layer unit of the model, and extract the input dimension, output dimension and weight parameter set of each network layer unit in hierarchical order. S22. Set training prohibition flags for each item in the weight parameter set, close the gradient calculation channel of the corresponding network layer unit, and skip the update calculation of the marked weight parameters during the training phase of the model. S23. Construct linear mapping units for scalable parameter segments and insert them between adjacent network layer units. Determine the shape of the mapping matrix based on the output dimension of the previous network layer unit and the input dimension of the next network layer unit, and add the linear mapping units to the forward propagation path of the model. S24. Initialize the sparse weights of the mapping matrix of the expandable parameter segment, generate non-zero parameter positions according to the preset sparsity using the grouped sparse coding method, and write them into the mapping matrix of the expandable parameter segment. S25. Set the mapping matrix of the scalable parameter segment to a trainable state, participate in gradient calculation and parameter update during the incremental update phase.
4. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, S3 specifically includes: S31. Extract the feature vectors of each data segment in the dataset to be processed, and use the kernel density estimation method to calculate the feature probability density function to obtain the corresponding feature density sequence. S32. Estimate the kernel density of the existing feature distribution of the model in the same dimension, and perform normalization to generate a reference density sequence; S33. The Wasserstein distance algorithm is used to calculate the distribution difference between the feature density sequence and the reference density sequence, and the distribution difference values are arranged in chronological order to form a difference measurement sequence. S34. Dynamically update the threshold of the difference measure sequence, and perform segment scanning on the updated difference measure sequence to define continuous data intervals that cross the threshold as feature change intervals, specifically including: The difference measure sequence is segmented by a sliding window, and the local mean, local standard deviation and local fractions are calculated in each window. An initial threshold sequence is then constructed based on the local mean and local fractions. The initial threshold sequence is subjected to bidirectional exponential smoothing to generate a forward smoothed threshold sequence and a backward smoothed threshold sequence, respectively. The two sequences are then fused element-wise to obtain a dynamic threshold update sequence. The difference measurement sequence and the dynamic threshold update sequence are differentially calculated item by item, and the sign of the difference results is determined to generate a cross-threshold label sequence. For the cross-threshold labeled sequence, continuous segment detection is performed, and the start and end positions of the continuous labeled segments are recorded using the run-length encoding method; The continuous marked segments of the records are filtered by segment length, segments that are lower than the set length threshold are removed, and the filtered continuous segments are defined as feature change segments. S35. Number the characteristic change segments and encode the numbering results into a trigger identifier sequence.
5. The lightweight model edge deployment optimization method based on incremental learning according to claim 4, characterized in that, Specifically, S33 includes: S331. Based on the feature density sequence and the reference density sequence, construct discrete probability vector groups respectively, use Euclidean distance to calculate the distance relationship between vectors, generate the corresponding cost matrix set, and perform dimension clipping and matrix alignment on the cost matrix set. S332. Construct an initial transmission matrix for regularized optimal transmission based on the cost matrix set, and update it cyclically in the order of row normalization and column normalization using the Sinkhorn-Knopp method. S333. Introduce a stabilization factor to correct the numerical stability of the row normalization and column normalization results of each iteration, and determine the convergence status based on the corrected iteration matrix. S334. After the convergence condition is met, the final iteration matrix and the cost matrix set are multiplied element-wise and the product matrix is summed globally to obtain the Wasserstein distance sequence arranged in time order. S335. Perform segment smoothing, sequence denoising, and fluctuation suppression on the Wasserstein distance sequence to generate a difference measure sequence.
6. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, S5 specifically includes: S51. Based on the updated model parameters, divide the network into units, statistically analyze the weight parameters in each unit, calculate the mean weight and L2 norm of each channel, and generate a channel importance matrix. S52. Based on the channel importance matrix, set an importance threshold, generate a structured pruning mask for channels with scores below the threshold, and use the pruning mask to block the corresponding network layer output channel unit. S53. Perform statistical analysis on the weight parameter distribution of each network layer unit in the pruned model structure, specifically including: Calculate the value fluctuation range and weight response change amplitude of each network layer unit, and sort the network layer units according to the change amplitude; Network layer units with changes in the upper range of the ranking are classified as high-sensitivity layer groups, network layer units with changes in the middle range of the ranking are classified as medium-sensitivity layer groups, and network layer units with changes in the lower range of the ranking are classified as low-sensitivity layer groups. Based on the impact of each layer on accuracy during the inference process, they are respectively labeled as high-precision requirement units, medium-precision requirement units, and low-precision requirement units. Based on the accuracy requirements, the corresponding quantization bit width selection range is determined, and the quantization bit width, quantization step size and zero offset parameters are specified for each network layer unit to form a mixed precision quantization table. S54. Load the three layers into their respective quantization paths and assign the corresponding quantization bit widths according to the mixed precision quantization table. Then perform quantization mapping, step size calibration, and zero-point offset calibration in sequence. S55. Perform forward inference tests based on the calibrated model structure, record inference latency and memory usage, and determine the lightweight update model through preset resource constraints.
7. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, S6 specifically includes: S61. Periodically sample the processor utilization, memory usage and inference latency of edge devices, construct resource state vectors and perform time smoothing processing; S62. Perform difference analysis on the resource status vector and the preset resource threshold, generate resource status tags for processor utilization, memory utilization and inference latency respectively, and perform dynamic threshold offset on the tags. S63. Dynamically adjust the lightweight update model based on resource status markers, specifically including: When the processor utilization status flag is in an over-limit state, the quantization precision of each network layer unit in the lightweight update model is replaced, and the corresponding quantization step size and zero offset parameters are updated after the replacement. When the memory usage status flag is in an over-limit state, the pruning ratio of the lightweight update model is incrementally expanded, the validity of low-importance network layer units is re-evaluated, and newly added low-importance network layer units are hidden. When the inference latency state flag is in an over-limit state, the inference configuration of the lightweight update model is restructured at the structural level by replacing high-complexity operators, merging adjacent network layer units and compressing intermediate features to adjust the inference path. When all three state markers are within the normal range, the quantization accuracy, pruning ratio, and inference configuration of the lightweight update model remain unchanged, and the resource state vector is updated to enter the next sampling period.
8. The lightweight model edge deployment optimization method based on incremental learning according to claim 1, characterized in that, Specifically, S7 includes: S71. Load the optimized and adjusted lightweight update model into the edge-side inference execution path, perform structural analysis on the operation nodes in the inference execution path, and establish a mapping relationship between the inference path and the model operation unit. S72. During the inference process, the runtime latency, device occupancy and feature output stability of each computing node are sampled in real time, and fluctuation detection and anomaly localization are performed on the sampling results. The abnormal segments are associated with the corresponding data input segments to generate a running state sequence. S73. Perform iterative updates on the data cache sequence based on the running status sequence, extract features from input segments marked as abnormal, and replace the corresponding positions in the data cache sequence.