Model pruning method, apparatus, device, storage medium and program product

Abstract

The application provides a model pruning method, device, equipment, storage medium and program product. The model pruning method comprises the following steps: constructing an early exit branch for each processing layer of a to-be-pruned model, the early exit branch corresponding to the processing layer is used for early exit of a channel of the to-be-pruned model, and data calculated by the processing layer is used for early exit; inputting a preset data set into the to-be-pruned model with the constructed early exit branch to obtain a model output result; determining an importance parameter value of each processing layer according to the model output result, the importance parameter value corresponding to the processing layer is used for representing a participation degree of the processing layer in calculation in a data processing process of the to-be-pruned model; and pruning each processing layer based on the importance parameter value of each processing layer and a target pruning proportion corresponding to the to-be-pruned model. The application can reduce the cost of the model while maintaining high model accuracy.

Description

Technical Field

[0001] This application belongs to the field of computer technology, specifically relating to a model pruning method, apparatus, device, storage medium, and program product. Background Technology

[0002] As deep learning technology continues to develop, the computational results of related models are becoming increasingly accurate. However, at the same time, the size of these models is also constantly increasing, consuming more and more memory, and their runtime latency and computational complexity are also rising, leading to a decrease in computational speed. Therefore, reducing the size of the models and the computational cost has become a necessary requirement for many large-scale models.

[0003] Model pruning techniques can effectively reduce model size, save memory, and make model computation faster. Pruning removes redundant operations and connections in large-scale models and resets weights with minimal impact on performance to zero, thus compressing large-scale models. However, current pruning methods often rely on fixed strategies or predefined methods, potentially applying the same pruning strategy to different models. This can easily lead to the removal of important operations or connections from the model being pruned, resulting in a decrease in computational accuracy. Summary of the Invention

[0004] This application proposes a model pruning method, apparatus, device, storage medium, and program product that can reduce the cost of the model while maintaining high model accuracy.

[0005] The first aspect of this application proposes a model pruning method, including:

[0006] Early termination branches are constructed for each processing layer of the model to be pruned. The early termination branches corresponding to the processing layers are used as channels for the data calculated by the processing layers to exit the model to be pruned in advance.

[0007] Input the preset dataset into the pruning model to be built to construct the early termination branch, and obtain the model output results;

[0008] Based on the output of the model, the importance parameter values ​​of each processing layer are determined. The importance parameter values ​​of the processing layer are used to characterize the degree of participation of the processing layer in the data processing of the model to be pruned.

[0009] Based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned, each processing layer is pruned.

[0010] In some embodiments of this application, the construction of early regression branches for each processing layer of the model to be pruned includes:

[0011] The early termination conditions for each processing layer in the pruning model are determined respectively. The early termination conditions corresponding to the processing layer are used to determine the conditions that the calculation results of the processing layer need to be satisfied to exit the pruning model in advance.

[0012] Early termination branches are constructed for each processing channel of each processing layer, and corresponding early termination conditions are configured for each early termination branch.

[0013] In some embodiments of this application, after constructing early regression branches for each processing layer of the model to be pruned, the method further includes:

[0014] Obtain the target dataset corresponding to the target use case;

[0015] Using the target dataset, the pruning model for constructing early-retreat branches is fine-tuned and trained.

[0016] In some embodiments of this application, determining the importance parameter values ​​of each processing layer based on the model output includes:

[0017] Based on the output of the model, the utilization rate of each processing layer is determined. The utilization rate of the processing layer is used to characterize the probability that the processing layer participates in the calculation during the data calculation process of the model to be pruned.

[0018] The utilization rate of each processing layer is normalized to obtain the importance parameter value of each processing layer.

[0019] In some embodiments of this application, the model output includes the final output and the output of at least one early termination branch; determining the utilization rate of each processing layer based on the model output includes:

[0020] The processing layers that participate in calculating the final output result and the processing layers that participate in calculating the output result of the early termination branch are determined respectively.

[0021] The number of times each processing layer in the model to be pruned participates in the calculation of the output result, and the total number of times the model outputs the result;

[0022] The utilization rate of each processing layer is calculated based on the target number of times corresponding to each processing layer and the total number of times.

[0023] In some embodiments of this application, pruning each processing layer based on the importance parameter value of each processing layer and the target pruning ratio corresponding to the model to be pruned includes:

[0024] The total pruning amount of the model to be pruned is determined based on the total number of parameters of the model to be pruned and the corresponding target pruning ratio.

[0025] Based on the total pruning amount of the model to be pruned and the importance parameter values ​​of each processing layer, the pruning task amount corresponding to each processing layer is determined;

[0026] Pruning is performed on each processing layer according to the pruning task corresponding to each processing layer.

[0027] In some embodiments of this application, determining the pruning task volume corresponding to each processing layer based on the total pruning amount of the model to be pruned and the importance parameter values ​​of each processing layer includes:

[0028] The pruning coefficient for each processing layer is determined based on the importance parameter values ​​of each processing layer. The pruning coefficient represents the proportion of the amount of parameters that need to be pruned from each processing layer to the total amount of pruning.

[0029] Based on the total pruning amount of the model to be pruned and the pruning coefficient corresponding to each processing layer, the pruning task amount corresponding to each processing layer is determined.

[0030] In some embodiments of this application, after pruning each of the processing layers, the method further includes:

[0031] Obtain the target dataset corresponding to the target use case;

[0032] The pruned model to be pruned is fine-tuned using the target dataset.

[0033] A second aspect of this application provides a model pruning device, comprising:

[0034] The branch construction module is used to construct early termination branches for each processing layer of the model to be pruned. The early termination branches corresponding to the processing layers are used as channels for the data calculated by the processing layers to exit the model to be pruned in advance.

[0035] The result acquisition module is used to input a preset dataset into the pruning model that constructs the early termination branch and obtain the model output result.

[0036] The importance determination module is used to determine the importance parameter value of each processing layer according to the output result of the model. The importance parameter value of the processing layer is used to characterize the degree of participation of the processing layer in the data processing of the model to be pruned.

[0037] The model pruning module is used to prune each of the processing layers based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned.

[0038] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0039] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0040] An embodiment of the fifth aspect of this application provides a computer program product including a computer program that is executed by a processor to implement the method described in the first aspect.

[0041] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0042] In this embodiment, the model pruning method first constructs early termination branches for each processing layer of the model to be pruned, then determines the importance parameter values ​​of each processing layer based on model early termination technology, and then prunes each processing layer of the model to be pruned based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned. Thus, based on model early termination technology, the importance of each processing layer of the model to be pruned can be accurately determined, and the importance parameter values ​​of each processing layer can be obtained. Then, when pruning the model, pruning can be performed based on the importance of each processing layer, reducing more weights or redundant connections and operations for less important processing layers, reducing fewer weights or redundant connections and operations for more important processing layers, and not pruning even for processing layers with higher importance. This allows for adaptive pruning of the model to be pruned based on the importance of each processing layer, effectively reducing model cost while maintaining high model accuracy.

[0043] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0045] In the attached diagram:

[0046] Figure 1 A comparative diagram of structured pruning and unstructured pruning is shown;

[0047] Figure 2 A schematic diagram illustrating the pruning principle of the model used in the prior art is shown;

[0048] Figure 3 A schematic flowchart of a model pruning method provided in an embodiment of this application is shown;

[0049] Figure 4 This illustration shows a schematic diagram of the pruning model for constructing early-retreat branches in an embodiment of this application;

[0050] Figure 5 A schematic diagram of the specific process in step S3 of this application embodiment is shown;

[0051] Figure 6 A schematic diagram of the specific process in step S31 of this application embodiment is shown;

[0052] Figure 7 A schematic diagram of the model pruning principle applied in the embodiments of this application is shown;

[0053] Figure 8 A schematic diagram of the specific process in step S4 of this application embodiment is shown;

[0054] Figure 9 This illustration shows a schematic diagram of the architecture of a model pruning device provided in one embodiment of this application;

[0055] Figure 10 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0056] Figure 11 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0057] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0058] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0059] Pruning is a model optimization technique that eliminates redundant operations and connections in large-scale models and zeroes out weights that have little impact on model performance, resulting in a smaller model. The accuracy of a pruned model is very close to that of an unpruned large model. Based on the granularity of pruning, model pruning can be divided into structured pruning and unstructured pruning. For example... Figure 1 As shown, structured pruning can be understood as filter-level pruning, primarily removing redundant channels, groups of convolutional kernels, and entire network layers; unstructured pruning includes pruning individual weights, weight vectors (rows / columns), and individual convolutional kernels. In related technologies, pruning is typically performed as follows... Figure 2 As shown, the same pruning strategy is applied at each layer, such as, but not limited to, removing the same weights or connections. However, such pruning can easily remove important operations or connections from the model being pruned, leading to a decrease in computational accuracy.

[0060] Based on this, the inventors of this application propose a model pruning method. This method determines the importance parameter values ​​of each processing layer of the model to be pruned based on model early termination technology. Then, based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned, pruning is performed on each processing layer of the model to be pruned. In this way, the importance of each processing layer of the model to be pruned can be accurately determined based on the model early termination technology. Then, when pruning the model, pruning can be performed based on the importance of each processing layer. More weights or redundant connections and operations are removed from less important processing layers, while fewer weights or redundant connections and operations are removed from more important processing layers. Processing layers with even higher importance can be left unpruned. Thus, the model to be pruned can be adaptively pruned according to the importance of each processing layer of the model to be pruned, effectively reducing model costs while maintaining high model accuracy.

[0061] Early termination refers to the ability of a model to terminate its processing and exit the model prematurely if the intermediate values ​​calculated by the model meet certain early termination conditions. In this embodiment, the early termination conditions are used to determine the conditions that must be met for the computational results (intermediate values) of the processing layer to exit the pruned model early. Early termination conditions may include, for example, the numerical value of the computational result reaching a preset threshold, or the loss value of the computational result reaching a convergence requirement. This embodiment does not specify the exact content of the early termination conditions; it is sufficient to determine whether the computational results of the processing layer can exit the pruned model early.

[0062] The model to be pruned can be any model that needs pruning, such as, but not limited to, large-scale language models like BERT (Bidirectional Encoder Representations from Transformer) and GPT (Generative Pre-training Transformer). The processing layer of the model to be pruned can be any layer structure, such as the transformation coding layers or connection layers of a Transformer network. The early-retreat branches corresponding to the processing layers serve as channels for data computed by the processing layers to exit the model to be pruned prematurely.

[0063] It is understood that this embodiment does not specifically limit the type of model to be pruned. It can be a neural network model or other models that require pruning. For example, decision tree models can improve their generalization ability and efficiency by reducing the depth and width of the decision tree through pruning techniques. Support Vector Machine (SVM) models can simplify the model and improve training and testing speed by reducing the number of support vectors and parameters through pruning techniques. Bayesian models can improve their efficiency and generalization ability by removing some unnecessary variables and parameters through pruning techniques. Ensemble learning models can be optimized by removing some weak classifiers or decision trees through pruning techniques. Some image processing models, such as SIFT (Scale Invariant Feature Transform) models and HOG (Histogram of Oriented Gradients) models, can also be optimized through pruning techniques.

[0064] The execution subject of this model pruning method can be the computer, server, server cluster, cloud platform, etc., on which the model to be pruned is applied, or it can be the specific execution device on these devices. Specifically, it can be applied to parallel computing architectures to reduce model size and thus improve parallel computing capabilities.

[0065] The model pruning method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0066] Please see Figure 3 This is a flowchart illustrating the model pruning method provided in the embodiments of this application, as shown below. Figure 3 As shown, the model pruning method may include the following steps:

[0067] Step S1: Construct early retirement branches for each processing layer of the model to be pruned.

[0068] In this embodiment, the model to be pruned can be a pre-trained model. That is, before executing the pruning method, the basic model can be pre-trained to obtain a large-scale model applicable to specific scenarios, i.e., the model to be pruned. Taking a large-scale language model as an example, the purpose of Natural Language Processing (NLP) is to enable computers to possess human abilities such as listening, speaking, reading, writing, translating, questioning, answering, searching, summarizing, conversing, and chatting, and to use knowledge and common sense for reasoning and decision-making, supporting scenarios such as customer service, diagnosis, law, and education. The purpose of pre-training is to move NLP from the original stage of manual parameter tuning to a stage where it can be applied on a large scale and replicated in large-scale industrial applications. Specifically, this embodiment can be based on initializing a large-scale language model (e.g., BERT model, GPT model) and completing the pre-training task on open-source large-scale language samples (which can be understood as training datasets). Then, for the pre-trained model to be pruned, early termination branches corresponding to each processing layer are constructed, and the importance of each processing layer of the model to be pruned can be determined through early termination techniques.

[0069] In some embodiments, step S1 may include the following processes: determining the early termination conditions for each processing layer in the model to be pruned; constructing early termination branches for each processing channel of each processing layer, and configuring corresponding early termination conditions for each early termination branch. The early termination conditions corresponding to the processing layer are used to determine the conditions that must be met for the computational results of the processing layer to exit the model to be pruned prematurely.

[0070] like Figure 4 The diagram shows a layer structure of a pruning model. Small squares 301 represent data blocks, and straight lines 303 connecting a series of data blocks through each processing layer 302 can be understood as abstract data channels. When constructing early termination branches, early termination branches 304 can be constructed corresponding to each processing channel of the processing layer. Each early termination branch can be evenly distributed... Figure 4 The exit layer 305 shown is connected. Specifically, it can be set so that as long as the calculation result corresponding to an early exit branch 304 of the same processing layer 302 can meet the early exit condition, it can be determined that the calculation result of the processing layer 302 can exit the pruning model early. Alternatively, it can be set so that the calculation results corresponding to all early exit branches of the same processing layer can meet the early exit condition before it can be determined that the calculation result of the processing layer can exit the pruning model early. This embodiment does not specifically limit this.

[0071] In other embodiments, after constructing early termination branches for each processing layer of the model to be pruned, the following processing may also be included: obtaining the target dataset corresponding to the target use scenario; and using the target dataset to fine-tune the training of the model to be pruned with the early termination branches constructed.

[0072] The target use case can be any scenario where the model to be pruned can be applied, such as, but not limited to, speech recognition, text prediction, and image processing. The target dataset corresponding to the target use case can be understood as the data applied in the target use case, such as the language to be recognized and its corresponding machine language, the text to be predicted and its corresponding text, and the image to be processed and its corresponding processing result.

[0073] In practical applications, after constructing the early termination branch, the target dataset mentioned above can be used to train the pruning model with the early termination branch, so that the constructed early termination branch can accurately judge the early termination of data in the target use scenario, thereby making the importance parameter values ​​of each processing layer determined based on the early termination technology more accurate.

[0074] Step S2: Input the preset dataset into the pruning model to be constructed for early termination branches, and obtain the model output results.

[0075] The preset dataset can be, but is not limited to, the target dataset mentioned above, or other open-source sample sets, as long as the importance of each processing layer of the model to be pruned can be accurately determined based on the model output results.

[0076] It is understandable that the model output may include the final output and the output of at least one early termination branch. The final output can be understood as the result of calculations involving all processing layers of the model to be pruned.

[0077] Step S3: Determine the importance parameter values ​​for each processing layer based on the model output results.

[0078] The importance parameter value corresponding to the processing layer characterizes the degree of participation of the processing layer in the data processing of the model to be pruned, which can also be understood as the degree of influence of the processing layer on the accuracy of the output result. The earlier the model output result exits, the fewer processing layers participate in the calculation of the model output result, and the importance parameter value of the processing layer that does not participate in the calculation is lower than the importance parameter value of the processing layer that does participate in the calculation. For example, the importance parameter value can be regarded as a proportional value, then the higher the degree of influence on the accuracy of the output result, the larger the importance parameter value of the processing layer.

[0079] It is understandable that the processing layers to be pruned in the model to be pruned can include all processing layers of the model to be pruned, or only processing layers that are far from the data input layer. For example, the first processing layer connected to the input layer will basically participate in the calculation of all model output results, so when pruning the model to be pruned, the first processing layer may not be pruned.

[0080] When determining the importance parameter values ​​for each processing layer, after inputting a preset dataset into the pruning model to construct the early regression branch, a model output result can be obtained for each set of input data in the preset dataset. This embodiment can determine the importance parameter values ​​for each processing layer based on the output results corresponding to each set of input data in the preset dataset.

[0081] Specifically, such as Figure 5 As shown, step S3 may include the following processing: step S31, determine the utilization rate of each processing layer according to the model output results; step S32, normalize the utilization rate of each processing layer to obtain the importance parameter value of each processing layer.

[0082] The utilization rate of the processing layer is used to characterize the probability that the processing layer participates in the calculation process during the data calculation of the model to be pruned. It can be understood as the probability that the calculation process of the model output results uses each processing layer when the above-mentioned preset dataset or other datasets that can be processed are processed and multiple model output results are obtained by applying the early termination technique. That is, the utilization rate of the processing layer is equal to the ratio of the number of times the processing layer is used to calculate the results in multiple model output results to the total number of model output results.

[0083] Based on the utilization rate of each processing layer, the importance parameter value of each processing layer is calculated in this embodiment. This can accurately reflect the importance of each processing layer in the model to be pruned. As a result, unimportant weights and structures can be subtracted during pruning to improve the accuracy of the pruned model.

[0084] Furthermore, such as Figure 6 As shown, step S31 above may include the following processes: step S311, determining the processing layers that participate in calculating the final output result and the processing layers that participate in calculating the output result of the early termination branch; step S312, counting the target number of times each processing layer in the model to be pruned participates in the calculation of the output result, and the total number of times the model output result is output; step S313, calculating the utilization rate of each processing layer based on the target number and the total number of times corresponding to each processing layer.

[0085] In this embodiment, when determining the utilization rate of each processing layer of the model to be pruned using the early termination technique, a preset dataset is input into the model to be pruned, which constructs the early termination branch. The preset dataset includes multiple sets of input data. Each set of input data is input from the input layer, passes through each processing layer of the model to be pruned sequentially, and is then output from the output layer. Each set of input data corresponds to one output result. Therefore, after the preset dataset is input into the model to be pruned, multiple results can be output. The total number of outputs of the model is the number of model output results, which is also the number of times each processing layer should participate in the calculation of the model output results without early termination of all intermediate data. The target number can be understood as the number of times each processing layer actually participates in the calculation of the model output results when intermediate data is terminated early. Thus, by combining the target number corresponding to each processing layer with the total number mentioned above, the utilization rate of each processing layer during the data processing of the preset dataset can be accurately calculated.

[0086] Specifically, the utilization rate of each processing layer can be determined by the ratio of the target number of times to the total number of times. Alternatively, a preset dataset can be input multiple times, and the utilization rate of each processing layer can be calculated multiple times, with the average of these calculations used as the final calculated utilization rate.

[0087] For example, the model to be pruned has 10 processing layers, arranged from input to output as the first processing layer, the second processing layer, and so on, with the tenth processing layer preceding the output layer. A preset dataset is input into the model to be pruned to construct early termination branches, resulting in 10,000 model outputs. After the first processing layer, there are 50 early termination model outputs; after the second processing layer, 100 early termination model outputs; after the third processing layer, 300 early termination model outputs; after the fourth processing layer, 600 early termination model outputs; after the fifth processing layer, 950 early termination model outputs; after the sixth processing layer, 1300 early termination model outputs; after the seventh processing layer, 1800 early termination model outputs; after the eighth processing layer, 2300 early termination model outputs; after the ninth processing layer, 2550 early termination model outputs; and the remaining 50 non-early termination models are the final output after the tenth processing layer. The target number of calculations for the first processing layer is 10,000, the target number of calculations for the second processing layer is 9,950, and the target number of calculations for the third to tenth processing layers are 9,850, 9,550, 8,950, 8,000, 6,700, 4,900, 2,600, and 50, respectively.

[0088] The utilization rate of each processing layer is calculated as follows:

[0089] The utilization rate of the first processing layer, S1 = 10000 / 10000 = 100%;

[0090] The utilization rate of the second processing layer, S2 = (10000-50) / 10000 = 99.5%;

[0091] The utilization rate of the third processing layer, S3 = (10000 - 50 - 100) / 10000 = 98.5%;

[0092] According to the above calculation method, the utilization rates of the fourth to tenth processing layers are 95.5%, 89.5%, 80.0%, 67.0%, 49.0%, 26.0%, and 0.5%, respectively.

[0093] After calculating the utilization rates, the utilization rates of each processing layer can be converted into importance parameter values ​​for each processing layer using a normalization method. These importance parameter values ​​can be denoted as Mi, where Mi is greater than or equal to 0 and less than or equal to 1. Specifically, processing layers with utilization rates greater than or equal to a first threshold have an importance parameter value of Mi = 1, and these layers can be left unpruned; processing layers with utilization rates less than or equal to a second threshold have an importance parameter value of Mi = 0, and these layers can be pruned entirely. For processing layers with utilization rates between the first and second thresholds, the normalization formula Mi = (Si - Smin) / (Smax - Smin) can be used for specific calculation. Here, Smin represents the minimum utilization rate between the first and second thresholds, and Smax represents the maximum utilization rate between the first and second thresholds. Furthermore, this embodiment does not specifically limit the values ​​of the first and second thresholds; those skilled in the art can set them according to actual conditions. For example, the first threshold can be greater than or equal to 95%, and the second threshold can be less than or equal to 1%.

[0094] It should be noted that the above calculation process for calculating the utilization rate and importance parameter values ​​of each processing layer is only an example. In actual applications, the utilization rate of each processing layer can be any possible situation, and this embodiment does not make any specific limitation on this.

[0095] Step S4: Based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned, prune each processing layer respectively.

[0096] In practical applications, when the model to be pruned is large, a target pruning ratio can be set based on the desired processing speed of the parallel architecture and the processing performance of the computer. The target pruning ratio can be understood as the ratio of the number of parameters to be pruned to the total number of parameters in the model to be pruned. When pruning the model, one can... Figure 7As shown, based on the importance parameter values ​​of each processing layer and the target pruning ratio, adaptive pruning is performed on each processing layer of the model to be pruned. Processing layers with higher importance parameter values ​​have fewer parameters pruned, while processing layers with lower importance parameter values ​​have more parameters pruned. This effectively reduces the size and cost of the model to be pruned, while also ensuring that the pruned model has high accuracy.

[0097] It is understood that this embodiment does not specify the exact value of the target pruning ratio, for example, it can be 10%-20%.

[0098] In some embodiments, such as Figure 8 As shown, step S4 may include the following steps: Step S41, determine the total pruning amount of the model to be pruned based on the total number of parameters of the model to be pruned and the corresponding target pruning ratio; Step S42, determine the pruning task amount corresponding to each processing layer based on the total pruning amount of the model to be pruned and the importance parameter value of each processing layer; Step S43, prune each processing layer according to the pruning task amount corresponding to each processing layer.

[0099] In this context, the total number of parameters in the model to be pruned can be understood as the total number of various parameters in the model to be pruned, denoted as Param_total. Therefore, the total pruning amount of the model to be pruned = Param_total * target pruning ratio. The pruning task amount corresponding to each processing layer can be understood as the number of parameters that need to be pruned in each processing layer, denoted as Param_i. Based on the relationship between the utilization rate and importance parameter values ​​of each processing layer, it can be seen that the pruning task amount Param_i of each processing layer is negatively correlated with its importance parameter value. It is understood that this embodiment does not specifically limit the specific relationship between the pruning task amount of each processing layer and its importance parameter value. Those skilled in the art can make specific settings according to the specific structure of the model to be pruned, as long as the two are negatively correlated and the pruning of the model to be pruned can be achieved.

[0100] In this embodiment, after determining the importance parameter values ​​corresponding to each processing layer, the total pruning amount of the model to be pruned can be calculated first, based on the total number of parameters of the model to be pruned and the corresponding target pruning ratio, according to the formula for calculating the total pruning amount mentioned above. Then, based on the total pruning amount of the model to be pruned and the importance parameter values ​​of each processing layer, the pruning task amount corresponding to each processing layer can be calculated according to the formula for calculating the pruning task amount mentioned above. Afterwards, each processing layer can be pruned according to the calculated pruning task amount of each processing layer.

[0101] Specifically, step S42 may include the following processing: determining the pruning coefficient corresponding to each processing layer based on the importance parameter value of each processing layer; and determining the pruning task corresponding to each processing layer based on the total pruning amount of the model to be pruned and the pruning coefficient corresponding to each processing layer.

[0102] Among them, the pruning coefficient of the processing layer can be denoted as Qi, which represents the proportional coefficient of the number of parameters to be pruned in each processing layer during pruning to the total pruning amount. Then the pruning task amount Param_i corresponding to each processing layer = total pruning amount * the pruning coefficient Qi corresponding to this processing layer. For example, it is preset to subtract 10% of the parameters of the model to be pruned, that is, the total pruning amount is 10% of the total number of parameters. Then the number of parameters to be pruned in each processing layer is respectively the total pruning amount * Qi. The sum of the pruning coefficients of all the processing layers to be pruned is equal to 1, that is, the sum of the pruning coefficients of the processing layers to be pruned is equal to 1, that is, Q1 + Q2 + …… + Q10 = 1. According to the relationship between the above usage rate and the importance parameter value, it can be obtained that M1 - M10 and S1 - S10 have the same distribution, and Q1 - Q10 and S1 - S10 have the opposite distribution. Specifically, according to the characteristics of the model to be pruned, the mapping function of the importance parameter value Mi and the pruning coefficient Qi can be set according to this distribution correlation, and then combined with the calculated usage rate of each processing layer, the importance parameter value Mi and the pruning coefficient Qi of each processing layer can be obtained. Given that the number of parameters in each layer of the model to be pruned is limited, and in order to minimize the impact on the accuracy of the model after pruning, the pruning amount of each processing layer can be controlled, that is, the product of the pruning coefficient Qi and the total pruning amount has an upper limit value. It should be noted that in this embodiment, the mapping function of the importance parameter value Mi and the pruning coefficient Qi is not specifically limited, and those skilled in the art can set it according to the specific characteristics of the model to be pruned. For example, when Mi = 1, Qi can be equal to 0. When Mi = 0, the product of Qi and the total pruning amount can be equal to the above upper limit value. When 0 < Mi < 1, based on the negative correlation between Qi and Mi, and Q1 + Q2 + …… + Q10 = 1, the specific value of Mi can be determined, and the specific value of Mi is usually not unique and can be specifically selected based on the specific structure of the model to be pruned.

[0103] After calculating the pruning task amount of each processing layer, the pruning ratio of each processing layer can also be calculated, that is, the ratio of the pruning task amount of the processing layer to the total number of parameters of this processing layer. Then the pruning ratio of the processing layer is equal to Param_total * target pruning ratio * Qi / Param_i.

[0104] It can be understood that after determining Mi and Qi, when calculating the pruning task amount corresponding to each processing layer, the corresponding Mi and Qi of each processing layer are not fixed. When actual pruning is performed, if the pruning task amount corresponding to a processing layer is greater than the upper limit value, adaptive adjustment can be carried out. That is, when the pruning coefficient Qi of a certain processing layer is large, it may not be possible to reach the pruning task amount that needs to be pruned for this processing layer even if the whole layer is subtracted. Then, adaptively, the pruning task amount that cannot be achieved for this processing layer can be分摊到相邻的处理层。(这里“分摊到相邻的处理层”翻译为“分摊 to adjacent processing layers”,感觉原文可能表述不太完整,你可以检查下原文是否准确)

[0105] In other embodiments, after pruning each processing layer as described above, the model pruning method may further include the following processing: obtaining a target dataset corresponding to the target use scenario; and using the target dataset to fine-tune the pruned model. The target dataset can be referred to in the above description of the target dataset, and will not be repeated here.

[0106] In practical applications, after pruning the model to be pruned, the target dataset can be used to train the pruned model to fine-tune the various parameters of the pruned model and improve its accuracy.

[0107] In another embodiment of this application, taking a large-scale language model as an example, the main process of the dynamic pruning method based on the early termination mechanism is as follows:

[0108] 1) Initialize a large-scale language model (e.g., BERT model, GPT model, etc.), complete the pre-training task of the model on a large-scale open-source language sample, and obtain a converged model to be pruned.

[0109] 2) Construct early retirement branches for each processing layer of the model to be pruned, forming a large early retirement model framework with multiple branches. Then, use the target dataset to fine-tune the large early retirement model framework with multiple branches.

[0110] 3) By using the early termination mechanism, the utilization rate of each processing layer of the model to be pruned is determined, and then the utilization rate of each processing layer is converted into the importance parameter value Mi by the normalization method.

[0111] 4) Based on the target pruning ratio required by the user (e.g., 10%, 20%, etc.), and referring to the importance parameter values ​​obtained in step 3), allocate pruning tasks to determine the pruning task volume of each processing layer, that is, the number of parameters that need to be pruned in each processing layer. Then, the number of parameters that need to be pruned in each processing layer is equal to Param_total * pruning ratio * Qi.

[0112] 5) Then, based on the pruning workload of each processing layer, prune each processing layer of the model to be pruned.

[0113] 6) Using the target dataset mentioned above, fine-tune the pruned model to improve its accuracy and minimize the degradation of its representational ability.

[0114] The model pruning method provided in this embodiment first constructs early regression branches for each processing layer of the model to be pruned. Then, it determines the importance parameter values ​​of each processing layer based on model early regression technology. Next, based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned, pruning is performed on each processing layer of the model to be pruned. In this way, the importance of each processing layer of the model to be pruned can be accurately determined based on model early regression technology, and the importance parameter values ​​of each processing layer can be obtained. Then, when pruning the model, pruning can be performed based on the importance of each processing layer, reducing more weights or redundant connections and operations for less important processing layers, reducing fewer weights or redundant connections and operations for more important processing layers, and not pruning even for the most important processing layers. Thus, the model to be pruned can be adaptively pruned according to the importance of each processing layer, effectively reducing model cost while maintaining high model accuracy.

[0115] Some embodiments of this application also provide a model pruning apparatus for performing the model pruning method provided in any of the above embodiments. Figure 9 A schematic diagram of the pruning device for this model is shown, such as... Figure 9 As shown, the pruning device for this model includes:

[0116] The branch construction module is used to build early retreat branches for each processing layer of the model to be pruned. The early retreat branches corresponding to the processing layers are used as channels for the data calculated by the processing layers to exit the model to be pruned in advance.

[0117] The result acquisition module is used to input a preset dataset into the model to be pruned after constructing the early termination branch, and obtain the model output results.

[0118] The importance determination module is used to determine the importance parameter values ​​of each processing layer based on the model output results. The importance parameter values ​​of the processing layer are used to characterize the degree of participation of the processing layer in the data processing of the model to be pruned.

[0119] The model pruning module is used to prune each processing layer based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned.

[0120] It is understood that the model pruning device provided in this embodiment and the model pruning method provided in this application embodiment are based on the same inventive concept and can at least achieve the same beneficial effects as the model pruning method. Moreover, the various implementation methods of the model pruning method embodiment are also applicable to the embodiment of this model pruning device, and will not be described again here.

[0121] It should be noted that the data involved in this application (including but not limited to data used for model training, stored data, and displayed data) are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0122] This application also provides an electronic device for performing the above-described model pruning method. Please refer to... Figure 10 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 10 As shown, the electronic device 4 includes: a processor 400, a memory 401, a bus 402, and a communication interface 403. The processor 400, the communication interface 403, and the memory 401 are connected via the bus 402. The memory 401 stores a computer program that can run on the processor 400. When the processor 400 runs the computer program, it executes the model pruning method provided in any of the foregoing embodiments of this application.

[0123] The memory 401 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 403 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0124] Bus 402 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 401 is used to store programs. After receiving an execution instruction, the processor 400 executes the program. The model pruning method disclosed in any of the foregoing embodiments of this application can be applied to the processor 400, or implemented by the processor 400.

[0125] The processor 400 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 400 or by instructions in software form. The processor 400 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 401. The processor 400 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the above method.

[0126] The electronic device provided in this application embodiment and the model pruning method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0127] This application also provides a computer-readable storage medium corresponding to the model pruning method provided in the foregoing embodiments. Please refer to... Figure 11 The computer-readable storage medium shown is an optical disc 50, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the model pruning method provided in any of the foregoing embodiments.

[0128] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0129] This application also provides a computer program product, including a computer program that is executed by a processor to implement the model pruning method described in any of the above embodiments.

[0130] The computer-readable storage medium and computer program product provided in the above embodiments of this application are based on the same inventive concept as the model pruning method provided in the embodiments of this application, and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0131] It should be noted that:

[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0133] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0134] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0135] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model pruning method, characterized in that, include: Early termination branches are constructed for each processing layer of the model to be pruned. The early termination branches corresponding to the processing layers are used as channels for the data calculated by the processing layers to exit the model to be pruned in advance. The preset dataset is input into the pruning model to construct the early termination branch. For each set of input data in the preset dataset, a model output result is obtained respectively. The preset dataset is the data applied in the target use scenario. The target use scenario includes any one of language recognition, text prediction, and image processing. The preset dataset includes any combination of the following: the language to be recognized and its corresponding machine language, the text to be predicted and its corresponding text, and the image to be processed and its corresponding processing result. Based on the output results of each model, the importance parameter value of each processing layer is determined. The importance parameter value of the processing layer is used to characterize the degree of participation of the processing layer in the data processing of the model to be pruned. The importance parameter value is greater than or equal to 0 and less than or equal to 1. Based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned, each processing layer is pruned.

2. The method according to claim 1, characterized in that, The process involves constructing early regression branches for each processing layer of the model to be pruned, including: The early termination conditions for each processing layer in the pruning model are determined respectively. The early termination conditions corresponding to the processing layer are used to determine the conditions that the calculation results of the processing layer need to be satisfied to exit the pruning model in advance. Early termination branches are constructed for each processing channel of each processing layer, and corresponding early termination conditions are configured for each early termination branch.

3. The method according to claim 1 or 2, characterized in that, After constructing early regression branches for each processing layer of the model to be pruned, the method further includes: Obtain the target dataset corresponding to the target use case; Using the target dataset, the pruning model for constructing early-retreat branches is fine-tuned and trained.

4. The method according to claim 1, characterized in that, The step of determining the importance parameter values ​​of each processing layer based on the obtained model output results includes: Based on the output results of each model, the utilization rate of each processing layer is determined. The utilization rate of the processing layer is used to characterize the probability that the processing layer participates in the calculation during the data calculation process of the model to be pruned. The utilization rate of each processing layer is normalized to obtain the importance parameter value of each processing layer.

5. The method according to claim 4, characterized in that, The model output includes the final output and the output of at least one early termination branch; The step of determining the utilization rate of each processing layer based on the obtained output results of each model includes: The processing layers that participate in calculating the final output result and the processing layers that participate in calculating the output result of the early termination branch are determined respectively. The number of times each processing layer in the model to be pruned participates in the calculation of the output result, and the total number of times the model outputs the result; The utilization rate of each processing layer is calculated based on the target number of times corresponding to each processing layer and the total number of times.

6. The method according to any one of claims 1, 2, 4-5, characterized in that, The step of pruning each processing layer based on its importance parameter value and the target pruning ratio corresponding to the model to be pruned includes: The total pruning amount of the model to be pruned is determined based on the total number of parameters of the model to be pruned and the corresponding target pruning ratio. Based on the total pruning amount of the model to be pruned and the importance parameter values ​​of each processing layer, the pruning task amount corresponding to each processing layer is determined; Pruning is performed on each processing layer according to the pruning task corresponding to each processing layer.

7. The method according to claim 6, characterized in that, The step of determining the pruning task volume corresponding to each processing layer based on the total pruning amount of the model to be pruned and the importance parameter values ​​of each processing layer includes: The pruning coefficient for each processing layer is determined based on the importance parameter values ​​of each processing layer. The pruning coefficient represents the proportion of the amount of parameters that need to be pruned from each processing layer to the total amount of pruning. Based on the total pruning amount of the model to be pruned and the pruning coefficient corresponding to each processing layer, the pruning task amount corresponding to each processing layer is determined.

8. The method according to any one of claims 1, 2, 4-5, characterized in that, After pruning each of the processing layers, the process further includes: Obtain the target dataset corresponding to the target use case; The pruned model to be pruned is fine-tuned using the target dataset.

9. A model pruning device, characterized in that, include: The branch construction module is used to construct early termination branches for each processing layer of the model to be pruned. The early termination branches corresponding to the processing layers are used as channels for the data calculated by the processing layers to exit the model to be pruned in advance. The result acquisition module is used to input a preset dataset into the pruning model to construct the early termination branch, and to obtain a model output result for each set of input data in the preset dataset; wherein, the preset dataset is data applied in the target use scenario, and the target use scenario includes any one of language recognition, text prediction, and image processing; the preset dataset includes any combination of the following: the language to be recognized and its corresponding machine language, the text to be predicted and its corresponding text, and the image to be processed and its corresponding processing result. The importance determination module is used to determine the importance parameter value of each processing layer based on the obtained output results of each model. The importance parameter value of each processing layer is used to characterize the degree of influence of the processing layer on the accuracy of the output results during the data processing of the model to be pruned. The importance parameter value is greater than or equal to 0 and less than or equal to 1. The model pruning module is used to prune each of the processing layers based on the importance parameter values ​​of each processing layer and the target pruning ratio corresponding to the model to be pruned.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method as described in any one of claims 1-8.

12. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 1-8.

Images