Model structured pruning method and device based on edge computing architecture

Abstract

The present invention proposes a model structured pruning method and device based on edge computing architecture, which includes: grouping edge devices according to the hardware resource information of each edge device in the cluster; for edge devices in the same group, taking the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware as resource constraints, pruning the neural network model deployed on the edge device; and calibrating the pruned neural network model. This method improves the hardware processing speed of the computing device, reduces memory consumption, and thus improves the internal performance of the computer system.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to a model structured pruning method and device based on an edge computing architecture. Background Art

[0002] In recent years, with the continuous development of deep learning technology and the popularization of edge computing devices, deep learning models are increasingly being used in people's daily lives, bringing convenience to people's lives. For edge architectures, the hardware environments of different edge devices are different, and the computing, memory and other resources of some edge devices are often limited. Therefore, it is necessary to deploy high-performance deep learning models on them for compression, reduce the computational complexity and memory consumption during model reasoning, and fully optimize resource allocation. Among them, model pruning technology is widely used in neural network compression and acceleration. By removing redundant model parameters, it can effectively reduce the computational complexity and spatial complexity of deep neural networks.

[0003] However, pruning a neural network model usually results in a decrease in accuracy, so it is necessary to fine-tune the pruned network training to restore the accuracy of the original model as much as possible. Most pruning techniques require fine-tuning the pruned model with the original training data to restore the model accuracy. Although such pruning methods are effective, in practice, due to privacy considerations, the original training data used to develop the model may not be accessible, so the pruned model cannot be further fine-tuned to achieve practical accuracy. Secondly, training the model on a large amount of original training data will incur a large computational overhead, increasing the time and computational costs of the deployment pipeline. If the model is not trained again, the entire deployment process is usually completed in a few minutes to more than ten minutes. However, retraining with the full amount of data after pruning takes hours or even days depending on the amount of data, and occupies a large amount of system computing resources, reducing the processing speed and efficiency of the entire system. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a model structured pruning method and device based on edge computing architecture. This method has small data requirements, reduces model fine-tuning time, improves the hardware processing speed of the computing device, reduces memory consumption, and thereby improves the internal performance of the computer system.

[0005] In order to achieve the above objectives, the present invention provides a model structured pruning method based on edge computing architecture, comprising:

[0006] Group edge devices according to the hardware resource information of each edge device in the cluster;

[0007] For edge devices in the same group,

[0008] The actual running time and memory resource consumption of the neural network accelerator on the edge device hardware are used as resource constraints to prune the neural network model deployed on the edge device;

[0009] Calibrate the pruned neural network model.

[0010] In some embodiments, a method for pruning a neural network model deployed on the edge device includes:

[0011] The neural network model deployed on the edge device is grouped according to the dependencies between model layers into multiple groups of substructures;

[0012] According to the weight importance score of each group of substructures, the substructures whose importance is below a certain threshold are marked;

[0013] The weights of the marked substructures are pruned, and the layers of the same group of substructures are pruned simultaneously to achieve the best pruning rate for each layer.

[0014] The optimal pruning rate of each layer is determined according to the resource constraint condition.

[0015] In some embodiments, the neural network model deployed on the edge device is grouped according to dependencies between model layers into a plurality of substructures, comprising:

[0016] The neural network model deployed on the edge device is represented in the form of a graph structure, where the nodes of the graph structure represent the layers of the model, and the edges in the graph represent the dependencies between the layers;

[0017] Consider any node as the initial node, and find an adjacent node adjacent to the initial node through the dependency relationship;

[0018] The adjacent node continues to search for the next adjacent node until the last adjacent node does not detect a new adjacent node;

[0019] All the queried adjacent nodes and the initial node are divided into the same group of substructures.

[0020] In some embodiments, the method of determining the weight importance score of each group of substructures comprises:

[0021] Use the L2 norm indicator to estimate the weight importance of different dimensions in the same layer and determine the weight importance score of each layer;

[0022] The weight importance scores of each layer in the same group of substructures are averaged to determine the weight importance score of the group of substructures.

[0023] In some embodiments, a method for determining an optimal pruning rate for each layer comprises:

[0024] Prune the weights of a single layer of the model to the initial pruning rate of the layer and run each layer separately to determine the running time and memory resource consumption of each layer on the hardware;

[0025] Use the minimum mean square error of each layer output to estimate the error of pruning the single layer weights, and calculate the sum of the errors of each layer to determine the overall error of the model;

[0026] The resource constraint condition is constructed by the sum of the running time and memory resource consumption of all layers on the hardware, the overall error of the model is minimized, and the optimal pruning rate of each layer is determined.

[0027] In some embodiments, calibrating the pruned neural network model comprises:

[0028] Apply a sparsity penalty to the labeled substructure, compressing the representational power of the substructure to the unpruned part of the model to reduce the information loss caused by pruning;

[0029] The marked substructures are deleted, and the pruned neural network model is calibrated on the calibration dataset to reduce the difference in the output of each layer of the neural network model before and after pruning, thereby restoring the model accuracy.

[0030] In some embodiments, a norm square penalty regression method is used to impose a sparsity penalty on the labeled substructure, and the target loss function is constructed as:

[0031]

[0032] Among them, L re represents the calibration loss, which is the output error term during model calibration, and is used to ensure that the output of the sparse model does not deviate from the original model during the sparsification process; W l,[i,:,:] represents the weight of the i-th output dimension in the l-th layer; λ l,i is the corresponding sparsification penalty coefficient, for the dimension planned to be pruned and deleted, λ l,i > 0, to force the corresponding weights to approach 0; for other unpruned dimensions, λ l,i =0 to maintain the original feature extraction capability.

[0033] In some embodiments, the calibration loss is determined based on the weights and input features of each layer of the neural network model before pruning and the corresponding weights and input features in the neural network model after pruning;

[0034] The calibration loss is used to calibrate the pruned neural network model to reduce the difference in output of each layer of the neural network model before pruning.

[0035] In some embodiments, the calibration loss is expressed as:

[0036]

[0037] Among them, L represents the number of layers of the model, s i is the index of the remaining dimension of the i-th layer, f i represents the parameterized layer, W i and X i are the weights and input features of the i-th layer of the neural network model before pruning, and The corresponding weights and input features in the pruned neural network model respectively.

[0038] In some embodiments,

[0039] Scale the learning rates of each layer of the model to optimize this calibration loss;

[0040] Among them, the learning rate of the i-th layer is scaled as:

[0041]

[0042] Among them, base_lr represents the preset standard learning rate.

[0043] On the other hand, the present invention also provides a model structured pruning device based on edge computing architecture, which adopts the above-mentioned model structured pruning method, and the device at least includes:

[0044] A cluster grouping module is used to group edge devices according to the hardware resource information of each edge device in the cluster;

[0045] For edge devices in the same group,

[0046] A pruning module, used to prune the neural network model deployed on the edge device by taking the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware as resource constraints;

[0047] The calibration module is used to calibrate the pruned neural network model.

[0048] In some embodiments, the pruning module further comprises:

[0049] A structure grouping module is used to group the neural network model deployed on the edge device into multiple groups of substructures according to the dependencies between model layers;

[0050] An importance assessment module is used to mark the substructures whose importance is below a certain threshold according to the weighted importance scores of each group of substructures;

[0051] The pruning execution module is used to prune the weights of the marked substructures, and the layers of the same group of substructures are pruned simultaneously to achieve the best pruning rate for each layer.

[0052] The optimal pruning rate of each layer is determined according to the resource constraint condition.

[0053] In addition, another aspect of the present invention provides a storage medium for storing a computer program for executing the above-mentioned model structured pruning method based on edge computing architecture.

[0054] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned model structured pruning method based on edge computing architecture when executing the computer program.

[0055] It can be seen from the above scheme that the advantages of the present invention are:

[0056] The model structured pruning method based on the edge computing architecture provided by the present invention can make full use of the different hardware computing capabilities and memory storage consumption of each group of edge devices, fully improve the hardware processing speed of the corresponding edge computing devices, reduce memory consumption, and reduce the time cost and computing cost of deploying pipelines, thereby improving the overall data processing capability and operating efficiency of the edge computing system and improving the internal performance of the computer system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the overall process of a model structured pruning method based on an edge computing architecture provided in one embodiment of the present invention;

[0058] Figure 2 for Figure 1 Detailed flow diagram of step S2;

[0059] Figure 3 It is the residual block structure diagram;

[0060] Figure 4 for Figure 1 Detailed flow diagram of step S3 in FIG.

[0061] Figure 5 It is a framework diagram of the model structured pruning device based on the edge computing architecture of the present invention;

[0062] in:

[0063] 40-Model structured pruning device based on edge computing architecture

[0064] 41-Cluster Grouping Module

[0065] 42-Pruning module

[0066] 421-Structural Grouping Module

[0067] 422-Importance Assessment Module

[0068] 423-Pruning execution module

[0069] 43-Calibration module

[0070] 431-Weight Sparsification Module

[0071] 432-Output Calibration Module

[0072] S1-S3, S21-S23, S31-S32-steps. DETAILED DESCRIPTION

[0073] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0074] See also Figure 1 As shown in Figure 1 The figure shows an overall flow chart of a model structured pruning method based on an edge computing architecture provided by an embodiment of the present invention.

[0075] A model structured pruning method based on edge computing architecture includes at least the following steps:

[0076] S1. Group edge devices according to the hardware resource information of each edge device in the cluster;

[0077] S2. For edge devices in the same group,

[0078] The actual running time and memory resource consumption of the neural network accelerator on the edge device hardware are used as resource constraints to prune the neural network model deployed on the edge device;

[0079] S3. Calibrate the pruned neural network model.

[0080] In this embodiment, the method integrates the memory space and computing power of multiple edge devices based on the edge computing distributed framework, and groups the edge devices according to the hardware resource information of each edge device in the cluster. The hardware resource information of edge devices in different groups is different, and the hardware computing power and memory of edge devices in the same group are roughly the same. By performing the same structured pruning scheme on the neural network models deployed in the same group of edge devices, the pruning process of the group of neural network models is constrained by the hardware computing power and memory of the group of edge devices. The hardware resource information of edge devices in different groups is different, and the resource constraints of neural network model pruning are also different, so that in the process of pruning the neural network models of each group of edge devices, the different hardware computing power and memory storage consumption of each group of edge devices can be fully utilized, which can fully improve the hardware processing speed of the corresponding edge computing devices, reduce memory consumption, and thus improve the overall data processing capability and operation efficiency of the edge computing system, and improve the internal performance of the computer system.

[0081] Generally speaking, for the model pruning process in step S2 and the calibration process in step S3, the fine-tuning process of the structured pruning scheme in the prior art does not consider the original model weight information, and only uses the original training data to restore the accuracy of the pruned model. Although this method is effective, in general, due to privacy considerations, the original training data used to develop the model may not be accessible, so the pruned model cannot be further fine-tuned to achieve practical accuracy. Secondly, training the model on a large amount of data will incur a large computational overhead and take a long time.

[0082] In this regard, a post-training structured pruning framework is proposed in this embodiment, and the calibration of the pruned model is achieved by aligning the output activation values ​​of each layer of the original model and the pruned model. The framework takes a pre-trained model and a predefined pruning rate as input, first groups the neural network weights according to layer dependencies, and measures the importance to determine the redundant structure, and then performs a two-stage calibration process to restore the model accuracy. In the first stage, a sparse penalty is imposed on the pruned channels to reduce the information loss caused by pruning, and in the second stage, the pruned model is calibrated to restore accuracy. The model framework avoids fine-tuning in a retraining manner, uses only a small amount of data, and can generate a pruned model in minutes. Compared with the existing structured pruning technology, it only requires about 1% of the data, reduces the amount of data storage, and reduces the model fine-tuning time by 2 orders of magnitude, thereby reducing the time cost and computing cost of the deployment pipeline, and improving the processing efficiency of the convolutional neural network accelerator, so that the hardware processing speed of the corresponding computing device can be fully improved, memory consumption is reduced, and the internal performance of the computer system is improved. The pruning process of step S2 and the calibration process of step S3 are described in detail below.

[0083] See also Figure 2As shown in Figure 2 A detailed flow chart of step S2 is shown.

[0084] In step S2, the coupled structures are grouped according to the model layer dependencies, and the importance of the structures is comprehensively measured. According to the preset pruning rate, the substructures with low importance are marked to prune the marked substructures. Specifically:

[0085] First, the neural network model deployed on the edge device is grouped according to the dependencies between the model layers and divided into multiple groups of substructures (S21). Specifically in step S21, the neural network model deployed on the edge device is represented in the form of a graph structure. The nodes of the graph structure represent the layers of the model, and the edges in the graph represent the dependencies between the layers. According to the graph structure, any node is regarded as the initial node, and an adjacent node adjacent to the initial node is found through the dependency relationship; the adjacent node continues to look for the next adjacent node adjacent to it until the last adjacent node does not detect a new adjacent node; finally, all the queried adjacent nodes and the initial node are divided into the same group of substructures. For example, the neural network model F is formalized as F = {f1, f2, ..., F l}, where f i represents a parameterized layer or a non-parametric operation. i The input and output are represented as f i- and f i+ , used to indicate that the weights of the same layer have different pruning schemes in the input dimension and output dimension. The dependencies of the network structure are divided into inter-layer dependencies and intra-layer dependencies. Inter-layer dependencies are determined by the connectivity of adjacent layers, and intra-layer dependencies are determined by specific modules whose inputs and outputs share the same pruning scheme, such as batch normalization layers. In this embodiment, the neural network model structure is abstracted into a graph structure. The above components f i are nodes in the graph, and dependencies are edges in the graph. If two nodes are connected by a path, they are grouped into the same group. By traversing all nodes in the graph, the model structure can be automatically analyzed and the structures in the model can be grouped. These substructure layers grouped into the same group need to be pruned at the same time and share the same pruning scheme. Figure 3 A residual block (Residulablock) shown, f 1- and f 4- The input of is the same intermediate feature, so it is grouped in the same group and has the same pruning scheme to align the dimensions and maintain network correctness. The model structure analysis step will eventually Figure 3 The structures shown are divided into the following groups: {f 1- ,f 4-},{f 1+ ,f 2-},{f 2+ ,f3-},{f 3+ ,f 4+ ,f 5- ,f 5+}.

[0086] Further, after the neural network model structure is grouped by step S21, the substructures below an importance threshold are marked according to the weight importance scores of each group of substructures (S22). The existing importance evaluation method only calculates the weights of a single layer independently, without considering the structural dependencies across layers. Since the layers in the same group need to be pruned at the same time, this framework evaluates the importance of the group level. Specifically in step S22, the weight importance of different dimensions of the same layer is estimated using the L2 norm indicator to determine the weight importance score of the layer; then, the weight importance scores of each layer in the same group of substructures are averaged, the importance scores of each layer are summarized within the group, and the weight importance score of the group of substructures is determined to evaluate the importance of the group level. In some embodiments, the weight importance score of each group can be further normalized, and the importance scores of different groups can be compared, such as sorting in order of size, and according to the set weight importance threshold, for example, the threshold is set to the last two groups of substructures in the sorting, at least one group with a low sorting weight importance score is screened out to prune the at least one group of substructures. In some embodiments, the weight importance threshold is set to a specific score, and the weight importance score determined for each group can be directly compared with the weight importance threshold. If it is lower than the weight importance threshold, the substructure of the group is screened out for pruning.

[0087] Finally, the weights of the marked substructures are pruned, and the layers of the same group of substructures are pruned simultaneously to achieve the optimal pruning rate of each layer (S23). The optimal pruning rate of each layer is determined based on resource constraints, that is, based on the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware.

[0088] Since the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware are related to the hardware structure. Retaining a specific number of channels in a single layer can make full use of hardware resources such as parallel cores. If the hardware resources are not fully utilized, pruning more channels at certain pruning rates does not translate into actual acceleration or reduction in memory consumption. Therefore, it is necessary to explicitly determine resource consumption for different sparsity levels. Assuming that the overall execution time of the model is approximately the sum of the running times of each layer, this embodiment simplifies the overall problem of the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware to determining the pruning rate of each layer. Assuming that the model has L layers, the specific steps for solving the pruning rate are: pruning the weights of a single layer of the model to the initial pruning rate of the layer and running the layer alone, determining the running time and memory resource consumption of the layer on the hardware. where pl is the layer pruning rate, 1≤l≤L. Then, the minimum mean square error of the output of this layer is used to estimate the error of pruning the weights of a single layer, and the sum of the errors of each layer is counted to determine the overall error of the model Finally, construct the pruning rate-resource consumption-error lookup table P, and construct the resource constraint condition based on the sum of the hardware running time and memory resource consumption of all layers Minimize the overall model error, that is The optimal pruning rate of each layer is determined by solving the integer linear programming problem. Since structured pruning prunes the entire channel at a time, the pruning rates in the table are discrete and can be efficiently solved using a dynamic programming algorithm.

[0089] In this embodiment, after the above steps S21-S23 are performed to group the coupled structures according to the model layer dependencies, comprehensively measure the structural importance, and mark the substructures with low importance according to the pruning rate to prune the marked substructures, it is necessary to calibrate the pruned neural network model to restore the model accuracy. Figure 4 As shown in Figure 4 A specific flow chart of step S3 is shown.

[0090] This step S3 includes two parts. First, a sparsity penalty is imposed on the marked substructure to compress the representation ability of the substructure to the unpruned part of the model to reduce the information loss caused by pruning (S31). By applying a regularization penalty to the pruned modules in the model, the weight information can be compressed to the unpruned part while maintaining the accuracy, reducing the model accuracy loss caused by the model pruning operation. Then, a global model calibration method using the layered activation error as a proxy target loss is used, that is, the marked substructure is deleted, and the pruned neural network model is calibrated on the calibration data set to reduce the difference in the output of each layer of the neural network model before and after pruning, and the model accuracy is restored (S32). This process only uses a small amount of data to calibrate the output activation values ​​of each layer of the pruned model, making full use of the information in the original model weights to quickly restore the model accuracy.

[0091] In some embodiments, for the weight sparsification process of step S31, a norm square penalty regression method is used to impose a sparsity penalty on the marked substructure, and the target loss function is constructed as:

[0092]

[0093] Among them, L re Represents the calibration loss, which is the output error term during model calibration, and is used to ensure that the output of the sparse model does not deviate from the original model during the sparsification process; is the penalty term imposed, W l,[i,:,:] represents the weight of the i-th output dimension in the l-th layer; λl,i is the corresponding sparsification penalty coefficient, for the dimension planned to be pruned and deleted, λ l,i > 0, to force the corresponding weights to approach 0; for other unpruned dimensions, λ l,i = 0 to maintain the original feature extraction capability. A larger penalty coefficient may make it difficult for the coefficient model to regress to the output of the original model, so this process can start with a smaller coefficient and gradually increase the penalty strength to mitigate the side effects of sparse penalties: λ = λ + δ, where δ is a predefined constant. This process sets the initial penalty coefficient to 0.02 and optimizes 10 rounds on the calibration data, with each round increasing the penalty coefficient by 0.02.

[0094] In structured pruning, the number of input and output channels of most layers is reduced, thereby limiting the intermediate features that can be used for calibration. In order to model the cumulative impact of errors in each layer, in some embodiments, for the output calibration process of step S32, a composite reconstruction loss is used to jointly optimize each layer of the model based on the eigenvalues ​​of the remaining dimensions, and the calibration loss is determined based on the weights and input features of each layer of the neural network model before pruning and the corresponding weights and input features in the pruned neural network model. The calibration loss is minimized and the pruned neural network model is calibrated using the calibration loss to reduce the difference in the output of each layer of the neural network model before pruning. The calibration loss is specifically expressed as:

[0095]

[0096] Among them, L represents the number of layers of the model, s i is the index of the remaining dimension of the i-th layer, f i represents the parameterized layer, W i and X i are the weights and input features of the i-th layer of the neural network model before pruning, and The corresponding weights and input features in the pruned neural network model are respectively pruned. The calibration loss extracts the output corresponding to the unpruned feature dimension in the original model to calculate the error. Directly optimizing the calibration loss function will cause the early layers to be over-optimized, resulting in unbalanced model parameter updates. For example, the first layer is optimized by the calibration error of itself and all subsequent layers, but the last layer is optimized only by its own calibration error, resulting in different optimization degrees for different layers. To alleviate this imbalance, in some embodiments, the learning rate of the i-th layer is scaled to:

[0097]

[0098] Among them, base_lr represents the preset standard learning rate. It is very important to ensure that the optimization target has an equal impact on all layers through learning rate scaling. During the output calibration process, the model is optimized on the calibration data for a total of 10 rounds.

[0099] In this embodiment, by applying a sparse penalty to the pruned channels in the above steps S31-S32 to reduce information loss caused by pruning, only a small amount of data is used to calibrate the output activation values ​​of each layer of the pruned model, so that the model accuracy can be quickly restored.

[0100] The model structuring method provided in the above embodiment of the present invention can be applied to edge terminal devices, and the terminal can support operating systems such as Windows, Android, IOS, Windows Phone, etc.

[0101] Reference Figure 5 , Figure 5 A model structured pruning device 40 based on edge computing architecture is shown, which can be implemented by Figure 1-Figure 4 The model structuring method based on the edge computing architecture shown, the model structuring pruning device provided in the embodiment of the present application can implement each process implemented by the above-mentioned model structuring pruning method.

[0102] A model structured pruning device 40 based on edge computing architecture, the device at least comprising:

[0103] The cluster grouping module 41 is used to group the edge devices according to the hardware resource information of each edge device in the cluster;

[0104] A pruning module 42 is used for pruning the neural network model deployed on the edge device of the same group by taking the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware as resource constraints;

[0105] The calibration module 43 is used to calibrate the pruned neural network model.

[0106] In some embodiments, the pruning module 42 further includes:

[0107] A structure grouping module 421 is used to group the neural network model deployed on the edge device into multiple groups of substructures according to dependencies between model layers;

[0108] The importance evaluation module 422 is used to mark the substructures whose importance scores are below a certain importance threshold according to the weighted importance scores of each group of substructures;

[0109] The pruning execution module 423 is used to prune the weights of the marked substructures, and prune the layers of the same group of substructures at the same time to achieve the best pruning rate of each layer.

[0110] Among them, the optimal pruning rate of each layer is determined according to resource constraints.

[0111] In some embodiments, the calibration module 43 further comprises:

[0112] Weight sparse module 431, used for

[0113] Apply a sparsity penalty to the labeled substructure, compressing the representational power of the substructure to the unpruned part of the model to reduce the information loss caused by pruning;

[0114] The output calibration module 432 is used to delete the marked substructures, calibrate the pruned neural network model on the calibration data set, reduce the difference between the outputs of each layer of the neural network model before and after pruning, and restore the model accuracy.

[0115] It should be understood that the descriptions of the above-mentioned model structured pruning method are also applicable to the model structured pruning device according to the embodiment of the present application. To avoid repetition, they are not described in detail again.

[0116] In addition, it should be understood that in the model structured pruning device 40 according to the embodiment of the present application, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the model structured pruning device can be divided into functional modules different from the modules illustrated above to complete all or part of the functions described above.

[0117] In addition, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the above-mentioned model structured pruning method are implemented and the same technical effect can be achieved.

[0118] It should be noted that the electronic devices in the embodiments of the present application may include mobile electronic devices and non-mobile electronic devices.

[0119] In addition, an embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above-mentioned model structured pruning method are implemented and the same technical effect can be achieved.

[0120] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0121] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be applied, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0122] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model structured pruning method based on edge computing architecture, characterized in that: include: Group edge devices according to the hardware resource information of each edge device in the cluster; For edge devices in the same group, The actual running time and memory resource consumption of the neural network accelerator on the edge device hardware are used as resource constraints to prune the neural network model deployed on the edge device, including: The neural network model deployed on the edge device is grouped according to the dependencies between model layers into multiple groups of substructures; According to the weight importance score of each group of substructures, the substructures whose importance is below a certain threshold are marked; Pruning weights of the marked substructures, and pruning layers of the same group of substructures simultaneously to achieve an optimal pruning rate for each layer, wherein the optimal pruning rate for each layer is determined according to the resource constraint condition; Calibrate the pruned neural network model, including: Apply a sparsity penalty to the labeled substructure, compressing the representational power of the substructure to the unpruned part of the model to reduce the information loss caused by pruning; The marked substructures are deleted, and the pruned neural network model is calibrated on the calibration dataset to reduce the difference in the output of each layer of the neural network model before and after pruning, thereby restoring the model accuracy.

2. The method according to claim 1, characterized in that The neural network model deployed on the edge device is grouped according to the dependencies between model layers into a plurality of substructures, including: The neural network model deployed on the edge device is represented in the form of a graph structure, where the nodes of the graph structure represent the layers of the model, and the edges in the graph represent the dependencies between the layers; Consider any node as the initial node, and find an adjacent node adjacent to the initial node through the dependency relationship; The adjacent node continues to search for the next adjacent node until the last adjacent node does not detect a new adjacent node; All the queried adjacent nodes and the initial node are divided into the same group of substructures.

3. The method according to claim 1 or 2, characterized in that: The method for determining the weight importance score of each group of substructures includes: Use the L2 norm indicator to estimate the weight importance of different dimensions in the same layer and determine the weight importance score of each layer; The weight importance scores of each layer in the same group of substructures are averaged to determine the weight importance score of the group of substructures.

4. The method according to claim 1, characterized in that: The method for determining the optimal pruning rate for each layer includes: Prune the model's single-layer weights to the initial pruning rate for each layer and run each layer separately to determine the running time and memory resource consumption of each layer on the hardware; Use the minimum mean square error of each layer output to estimate the error of pruning the single layer weights, and calculate the sum of the errors of each layer to determine the overall error of the model; The resource constraint condition is constructed by the sum of the running time and memory resource consumption of all layers on the hardware, the overall error of the model is minimized, and the optimal pruning rate of each layer is determined.

5. The method according to claim 1, characterized in that The norm square penalty regression method is used to impose sparsity penalty on the labeled substructure, and the target loss function is constructed as: Among them, L re represents the calibration loss, which is the output error term during model calibration, and is used to ensure that the output of the sparse model does not deviate from the original model during the sparsification process; W l,[i,:,:] represents the weight of the i-th output dimension in the l-th layer; λ l,i is the corresponding sparsification penalty coefficient, for the dimension planned to be pruned and deleted, λ l,i > 0, to force the corresponding weights to approach 0; for other unpruned dimensions, λ l,i =0 to maintain the original feature extraction capability.

6. The method according to claim 5, characterized in that The calibration loss is determined based on the weights and input features of each layer of the neural network model before pruning and the corresponding weights and input features in the neural network model after pruning; The calibration loss is used to calibrate the pruned neural network model to reduce the difference in output of each layer of the neural network model before pruning.

7. The method according to claim 6, characterized in that The calibration loss is expressed as: Among them, L represents the number of layers of the model, s i is the index of the remaining dimension of the i-th layer, f i represents the parameterized layer, W i and X i are the weights and input features of the i-th layer of the neural network model before pruning, and The corresponding weights and input features in the pruned neural network model respectively.

8. The method according to claim 6, characterized in that Scale the learning rates of each layer of the model to optimize this calibration loss; Among them, the learning rate of the i-th layer is scaled as: Among them, base_lr represents the preset standard learning rate.

9. A model structured pruning device based on edge computing architecture, characterized in that: The method according to any one of claims 1 to 8 is adopted, wherein the device at least comprises: A cluster grouping module is used to group edge devices according to the hardware resource information of each edge device in the cluster; A pruning module is used for pruning the neural network model deployed on the edge device of the same group by taking the actual running time and memory resource consumption of the neural network accelerator on the edge device hardware as resource constraints, and the pruning module further includes: A structure grouping module is used to group the neural network model deployed on the edge device into multiple groups of substructures according to the dependencies between model layers; An importance assessment module is used to mark the substructures whose importance is below a certain threshold according to the weighted importance scores of each group of substructures; The pruning execution module is used to prune the weights of the marked substructures, and prune the layers of the same group of substructures at the same time to achieve the best pruning rate of each layer, wherein the best pruning rate of each layer is determined according to the resource constraint condition: A calibration module is used to calibrate the pruned neural network model, and the calibration module further comprises: The weight sparsification module is used to impose a sparsity penalty on the labeled substructure and compress the representation capacity of the substructure to the unpruned part of the model to reduce the information loss caused by pruning; The output calibration module is used to delete the marked substructure, calibrate the pruned neural network model on the calibration dataset, reduce the difference between the outputs of each layer of the neural network model before and after pruning, and restore the model accuracy.

Images