Hybrid pruning method and device of neural network model, storage medium and program product

Abstract

Embodiments of the present application provide a hybrid pruning method and device for a neural network model, a storage medium and a program product. The method can be applied in the field of artificial intelligence. The method comprises: obtaining first weights of a neural network model, the first weights being weights obtained by pre-pruning; and performing online pruning on the first weights according to statistical characteristics of current online activation data of the neural network model to obtain second weights of the neural network model. The method improves the sparsity of the neural network model, thereby reducing the computational complexity of the neural network model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a hybrid pruning method, apparatus, storage medium, and program product for a neural network model. Background Technology

[0002] With the development of IoT technology, edge computing devices such as wearable devices and smart home terminals are increasingly demanding real-time data processing. However, these devices have limited hardware resources, making it difficult to run neural network models with a large number of parameters.

[0003] Currently, even after compression, traditional neural network models still contain redundant computations, resulting in low sparsity.

[0004] Therefore, improving the sparsity of neural network models and reducing their computational load has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a hybrid pruning method, apparatus, storage medium, and program product for neural network models, which can improve the sparsity of neural network models and thereby reduce the computational load of neural network models.

[0006] In a first aspect, embodiments of this application provide a hybrid pruning method for a neural network model, comprising: obtaining a first weight of the neural network model, wherein the first weight is a weight obtained in advance through pruning;

[0007] Based on the statistical characteristics of the current online activation data of the neural network model, the first weights are pruned online to obtain the second weights of the neural network model.

[0008] In one possible implementation, the step of pruning the first weights online based on the statistical characteristics of the current online activation data of the neural network model to obtain the second weights of the neural network model includes:

[0009] Determine the statistical characteristics of the online activation data;

[0010] Based on the statistical characteristics and the first weight, determine the first importance score of each weight in the first weight;

[0011] The first weight is pruned online based on the first importance score to obtain the second weight.

[0012] In one possible implementation, determining the first importance score of each weight in the first weight based on the statistical characteristics and the first weight includes:

[0013] The product or sum of the statistical feature and each weight in the first weight is determined as the first importance score of each weight in the first weight.

[0014] In one possible implementation, the step of pruning the first weight online based on the first importance score to obtain the second weight includes:

[0015] The weights whose first importance score is greater than the first threshold in the first weights are determined as the second weights.

[0016] In one possible implementation, the statistical feature is any one of the following or the result of operations on any one or more of the following: mean, extreme value, or norm.

[0017] In one possible implementation, obtaining the first weights of the neural network model includes:

[0018] Based on the non-zero weights in the initial weights of the neural network model, weight pruning is performed on the initial weights to obtain the third weights;

[0019] Based on the offline activation data of the neural network model, the third weight is pruned offline to obtain the first weight.

[0020] In one possible implementation, the step of offline pruning of the third weight based on the offline activation data of the neural network model to obtain the first weight includes:

[0021] Based on the offline activation data and the third weight, determine the second importance score of each weight in the third weight;

[0022] The third weight is pruned offline based on the second importance score to obtain the fourth weight;

[0023] The neural network model using the fourth weight is trained, and the fourth weight is adjusted during the training process to obtain the first weight.

[0024] In one possible implementation, determining the second importance score of each weight in the third weight based on the offline activation data and the third weight includes:

[0025] The product or sum of the offline activation data and the corresponding weights in the third weight is determined as the second importance score of each weight in the third weight.

[0026] In one possible implementation, the offline pruning of the third weight based on the second importance score to obtain the fourth weight includes:

[0027] The weights in the third weights whose second importance score is greater than the second threshold are determined as the fourth weights.

[0028] Secondly, embodiments of this application provide a voice processing method, including:

[0029] Acquire voice data;

[0030] The speech data is input into a neural network model to obtain the speech processing result output by the neural network model, wherein the neural network model is a model pruned using a hybrid pruning method of the neural network model described in the first aspect and / or various possible embodiments of the first aspect.

[0031] Thirdly, embodiments of this application provide a chip, the chip including a processor and a data interface, the processor reading instructions stored in a memory through the data interface to execute a hybrid pruning method for a neural network model as described in the first aspect and / or various possible implementations of the first aspect.

[0032] Fourthly, embodiments of this application provide a hybrid pruning device for a neural network model, comprising: an acquisition module and a determination module, wherein:

[0033] The acquisition module is used to acquire the first weight of the neural network model, wherein the first weight is a weight obtained in advance through pruning;

[0034] The determining module is used to perform online pruning on the first weights based on the statistical characteristics of the current online activation data of the neural network model, so as to obtain the second weights of the neural network model.

[0035] In one possible implementation, the determining module is further configured to:

[0036] Determine the statistical characteristics of the online activation data;

[0037] Based on the statistical characteristics and the first weight, determine the first importance score of each weight in the first weight;

[0038] The first weight is pruned online based on the first importance score to obtain the second weight.

[0039] In one possible implementation, the determining module is further configured to:

[0040] The product or sum of the statistical feature and each weight in the first weight is determined as the first importance score of each weight in the first weight.

[0041] In one possible implementation, the determining module is further configured to:

[0042] The weights whose first importance score is greater than the first threshold in the first weights are determined as the second weights.

[0043] In one possible implementation, the statistical feature is any one of the following or the result of operations on any one or more of the following: mean, extreme value, or norm.

[0044] In one possible implementation, the acquisition module is further configured to:

[0045] Based on the non-zero weights in the initial weights of the neural network model, weight pruning is performed on the initial weights to obtain the third weights;

[0046] Based on the offline activation data of the neural network model, the third weight is pruned offline to obtain the first weight.

[0047] In one possible implementation, the acquisition module is further configured to:

[0048] Based on the offline activation data and the third weight, determine the second importance score of each weight in the third weight;

[0049] The third weight is pruned offline based on the second importance score to obtain the fourth weight;

[0050] The neural network model using the fourth weight is trained, and the fourth weight is adjusted during the training process to obtain the first weight.

[0051] In one possible implementation, the acquisition module is further configured to:

[0052] The product or sum of the offline activation data and the corresponding weights in the third weight is determined as the second importance score of each weight in the third weight.

[0053] In one possible implementation, the acquisition module is further configured to:

[0054] The weights in the third weights whose second importance score is greater than the second threshold are determined as the fourth weights.

[0055] Fifthly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0056] The memory stores computer-executed instructions;

[0057] The processor executes computer execution instructions stored in the memory, causing the processor to perform the hybrid pruning method of the neural network model as described in the first aspect and / or various possible embodiments of the first aspect.

[0058] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the hybrid pruning method of the neural network model described in the first aspect and / or various possible embodiments of the first aspect.

[0059] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the hybrid pruning method for a neural network model as described in the first aspect and / or various possible implementations of the first aspect.

[0060] The hybrid pruning method, apparatus, storage medium, and program product for neural network models provided in this application prune the neural network model in an offline stage to obtain a first weight; in an online stage, the first weight of the neural network model is obtained, and the first weight is pruned online according to the statistical characteristics of the current online activation data of the neural network model to obtain a second weight of the neural network model. The second weight guides the neural network model to skip invalid operations, thereby improving the sparsity of the neural network model and reducing the computational load of the neural network model. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0062] Figure 1 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 1 ;

[0063] Figure 2 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 2 ;

[0064] Figure 3 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 3 ;

[0065] Figure 4 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 4 ;

[0066] Figure 5 A schematic diagram of a hybrid pruning method for a neural network model provided in this application embodiment. Figure 1 ;

[0067] Figure 6A schematic diagram of a hybrid pruning method for a neural network model provided in this application embodiment. Figure 2 ;

[0068] Figure 7 This is a schematic diagram of the structure of a hybrid pruning device for a neural network model provided in an embodiment of this application;

[0069] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0070] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0072] In the embodiments of this application, the use of terms such as "first" and "second" is to distinguish between identical or similar items that have essentially the same function and effect. For example, "first electronic device" and "second electronic device" are merely used to distinguish different electronic devices and do not limit their order of execution. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0073] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0074] In this application, it should be understood that the terms used have the following meanings:

[0075] Offline: refers to the offline training phase of a neural network model, in which a large amount of historical data is used to complete the construction, training, verification, optimization, and solidification of the neural network model.

[0076] Online: refers to a neural network model trained offline that is deployed in a processing device to process input data in real time.

[0077] In some edge computing devices with limited hardware resources, such as wearable devices, smart home terminals, or industrial sensors, real-time data processing tasks, such as voice wake-up, environmental monitoring, or anomaly detection, need to be completed under low-power, low-cost hardware architectures. Due to the limitations of power consumption and hardware complexity in edge computing devices, neural network models running in edge computing devices need to reduce computational complexity through pruning techniques.

[0078] Currently, the main pruning methods for neural network models include weight pruning and weight-activation joint pruning.

[0079] Weight pruning reduces the complexity of neural network models by calculating weight importance scores, resetting weights with low importance scores to zero, and storing indices of non-zero weights to achieve sparse computation. While simple to implement, requiring only a single-path router to process the weight indices, weight pruning is limited by the low weight redundancy in small-parameter neural network models, resulting in a limited number of weights that can be pruned and making it difficult to significantly reduce the computational load of the neural network model.

[0080] Weight-activation joint pruning skips corresponding computational steps based on zero-value activations in the online activation data, building upon weight pruning, thereby achieving higher sparsity in neural network models. However, this method requires hardware support for dual-path data routing. This complex hardware architecture significantly increases chip area and power consumption, doubling the cost of small edge computing devices and even negating the computational advantages of sparse computing.

[0081] In view of this, embodiments of this application provide a hybrid pruning method for neural network models. This method prunes the neural network model offline to obtain a first weight, and then performs online pruning based on the statistical characteristics of the current online activation data and the first weight to obtain a second weight for the neural network model. The second weight guides the neural network model to skip invalid operations, thereby improving the sparsity of the neural network model and reducing the computational load of the neural network model.

[0082] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0083] Figure 1 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 1The execution subject of this method can be an electronic device or chip with corresponding data storage and computing capabilities, such as a computer, server, or server cluster, or a chip within a computer, server, or server cluster. Figure 1 As shown, the method includes:

[0084] S101, Obtain the first weight of the neural network model.

[0085] The first weight is the weight obtained after pre-pruning.

[0086] Optionally, the weight includes a weight value and a weight value index. The weight value represents the magnitude of the weight, and the weight value index indicates the coordinate position of the weight. For example, if the weight is "1,3,6,3", then the correspondence between the weight value and the weight value index is "1-1", "3-2", "6-3", and "3-4". That is, the weight index corresponding to the weight value 1 is 1, the weight index corresponding to the weight value 3 is 2, the weight index corresponding to the weight value 6 is 3, and the weight index corresponding to the weight value 4 is 4.

[0087] Neural network models are pre-trained models deployed in edge computing devices.

[0088] The first weight is the weight parameter obtained after pruning the neural network model during offline training.

[0089] The first weight is obtained by pruning the neural network model, which first achieves a part of static sparsity compared to the original weight.

[0090] S102. Based on the statistical characteristics of the current online activation data of the neural network model, the first weight is pruned online to obtain the second weight of the neural network model.

[0091] Online activation data is intermediate data temporarily generated by each layer of a neural network model when performing online computation tasks.

[0092] The statistical characteristics of online activation data are calculated by performing statistical calculations on the activation data, which can be used to measure the activity level of each data point in the online activation data.

[0093] Online pruning is a process of filtering the weights in the current computation stage of a neural network model. The weights obtained through online pruning are only effective in the current computation step and are not retained in the next stage of computation.

[0094] The second weight is the effective weight that each layer of the neural network model actually participates in the calculation.

[0095] Based on the statistical characteristics of the current online activation data of the neural network model, the activity level of the current online activation data can be determined. Online pruning of the first weight based on these statistical characteristics involves selecting weights from the perspective of online activation data. This allows the neural network pruning method to dynamically adjust weights according to the online activation data, improving the applicability of the pruning method in different application scenarios.

[0096] The hybrid pruning method for neural network models provided in this application prunes the neural network model in the offline stage to obtain a first weight, thereby giving the neural network model a low sparsity. In the online stage, the first weight of the neural network model is obtained, and the first weight is pruned online according to the statistical characteristics of the current online activation data of the neural network model to obtain a second weight of the neural network model. The second weight guides the neural network model to skip invalid operations, thereby further improving the sparsity of the neural network model and reducing the computational load of the neural network model.

[0097] Figure 2 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, the method for obtaining the first weight of the neural network model is described in detail. The hybrid pruning method for the neural network model provided in this embodiment includes:

[0098] S201. Based on the non-zero weights in the initial weights of the neural network model, perform weight pruning on the initial weights to obtain the third weights.

[0099] The initial weights are all the weight parameters of the neural network model before it has been trained and pruned.

[0100] Non-zero weights are parameters in the initial weights that have a non-zero value and need to be included in the calculation.

[0101] Weight pruning is the process of resetting some weights to zero and removing them from a neural network model based on their importance. Weight importance can be represented by weight values; for example, a weight importance threshold can be set, and weights with values ​​less than the threshold can be removed to improve the sparsity of the neural network model.

[0102] The third weight is the weight value retained after weight pruning the initial weights of the neural network model.

[0103] By performing weight pruning on the initial weights of a neural network model, redundant parameters in the model can be initially eliminated, reducing the number of basic parameters in the model.

[0104] S202. Based on the offline activation data of the neural network model, perform offline pruning on the third weight to obtain the first weight.

[0105] Offline activation data is intermediate data generated by each layer of a neural network model during the training phase when the training data is processed.

[0106] Offline pruning is a pruning process performed during the offline training phase of a neural network model. After pruning, the network structure and weights of the neural network model will be fixed.

[0107] Offline pruning of the third weight based on offline activation data of the neural network model involves adjusting the weights of the neural network model according to the input data. This offline pruning, performed from the perspective of input data, can break through the upper limit of weight pruning and further improve the sparsity of the neural network model.

[0108] Figure 3 A flowchart illustrating a hybrid pruning method for a neural network model provided in this application embodiment. Figure 3 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, this embodiment provides a detailed explanation of how to prune the third weight offline using the offline activation data of the neural network model to obtain the first weight. The hybrid pruning method for the neural network model provided in this embodiment includes:

[0109] S301. Based on the offline activation data and the third weight, determine the second importance score of each weight in the third weight.

[0110] The second importance score is used in the offline phase to determine whether each weight in the third weight is retained.

[0111] In one possible implementation, determining the second importance score of each weight in the third weight based on offline activation data and the third weight includes: determining the product or sum of the offline activation data and the corresponding weight in the third weight as the second importance score of each weight in the third weight.

[0112] For example, the product of the offline activation data and the corresponding weights in the third weight is used as the second importance score. For instance, if the offline activation data for a certain time slice is "4, 1, 2, 6" and the weights in the third weight are "1, 1, 4, 2", then the process of multiplying the offline activation data and the weights in the third weight is "4×1, 1×1, 2×4, 6×2", and the resulting second importance scores are 4, 1, 8, and 12 respectively.

[0113] S302. Based on the second importance score, the third weight is pruned offline to obtain the fourth weight.

[0114] The fourth weight is the weight retained after filtering the weights in the third weight based on the second importance score.

[0115] In one possible implementation, the third weight is pruned offline based on the second importance score to obtain the fourth weight, including: determining the weights in the third weight whose second importance score is greater than a second threshold as the fourth weight.

[0116] The second threshold is a critical value for weight selection during the offline phase of the neural network model. For example, if the second threshold is set to 5, and the second importance scores are 4, 1, 8, and 12, then the fourth weight is determined to be "8, 12", meaning the fourth weight values ​​are "8, 12" respectively, and the fourth weight indices are "3, 4" respectively.

[0117] S303. Train the neural network model using the fourth weight, and adjust the fourth weight during the training process to obtain the first weight.

[0118] In this step, the neural network model using the fourth weight is trained. A target accuracy can be preset. The neural network model using the fourth weight is trained multiple times using a preset dataset, and the fourth weight is fine-tuned so that the accuracy of the neural network model reaches the target accuracy, thereby obtaining the first weight.

[0119] For example, after offline pruning of a neural network model, if the fourth weight is set to "8, 12", the accuracy of the neural network model will decrease. Training the neural network model using a preset dataset and adjusting the fourth weight to "9, 12" will allow its accuracy to reach the target accuracy, thus compensating for the accuracy loss caused by offline pruning.

[0120] The third weight is pruned offline based on the second importance score of each weight in the third weight. The weight is pruned by combining activation data and weight values, which improves the sparsity of the neural network model and transforms activation sparsity into weight sparsity. This avoids the hardware resource consumption caused by weight-activation joint pruning. The neural network model using the fourth weight is trained and the fourth weight is adjusted to obtain the first weight, which makes up for the accuracy loss caused by offline pruning.

[0121] Figure 4 Flowchart of the hybrid pruning method for the neural network model provided in this application Figure 4 ,like Figure 4 As shown, in this embodiment... Figure 1Based on the previous embodiment, this embodiment details the online pruning of the first weights according to the statistical characteristics of the current online activation data of the neural network model to obtain the second weights of the neural network model. The hybrid pruning method for the neural network model provided in this embodiment includes:

[0122] S401. Determine the statistical characteristics of online activation data.

[0123] Optionally, the online activation data can be divided into multiple groups of data according to channels or time slices, and the statistical characteristics of each group of data can be determined separately after determining the statistical characteristics of the online activation data.

[0124] In one possible implementation, the statistical characteristic is any one of the following or the result of operations on any one or more of the following: mean, extreme value, or norm.

[0125] For example, online activation data can be divided into multiple groups based on data time slices, and the mean of each group can be calculated to obtain the statistical characteristics of the online activation data. For instance, if a group of online activation data is "0, 1, 5, 6", and the statistical characteristic of this group is calculated by taking the mean, then the statistical characteristic of this group of online activation data is 3.

[0126] The statistical characteristics of online activation data can be determined by calculating the mean, extreme values, or norm. This eliminates the need for complex calculations, reduces the consumption of hardware resources in computing devices, and can accurately quantify the activity level of online activation data.

[0127] By determining the statistical characteristics of online activation data, we can identify the activity characteristics of the current online activation data, providing a data reference for pruning the neural network model.

[0128] S402. Based on statistical characteristics and the first weight, determine the first importance score of each weight in the first weight.

[0129] The first importance score is used to measure the importance of the current online active data and the combination of individual weights in this online inference phase, and can be used to determine whether the first weight should be retained in subsequent calculations.

[0130] In one possible implementation, the product or sum of the statistical features and the weights in the first weight can be determined as the first importance score of each weight in the first weight.

[0131] For example, the sum of the statistical features and the weights in the first weight can be determined as the first importance score of each weight in the first weight. For instance, if the statistical feature of a set of online activation data is 3 and the first weight is "4, 7, 5, 2", then the first importance scores of each weight in the first weight can be determined as "7, 10, 8, 5" respectively.

[0132] S403. Based on the first importance score, perform online pruning on the first weight to obtain the second weight.

[0133] The second weight is determined based on the first importance score during the current online inference phase. It is the weight value and weight index of the first weight that need to be retained. The second weight is only effective in this inference calculation and is used to guide this calculation. It does not change the original weight of the neural network model.

[0134] In one possible implementation, the weights whose first importance score is greater than a first threshold in the first weights can be determined as the second weights.

[0135] The first threshold is a pre-set judgment threshold in the online phase, used to determine the weights that need to be retained in the first weighting.

[0136] For example, if the first threshold is determined to be 7, and the first importance scores are "7, 10, 8, 5" in sequence, then the second weight values ​​can be determined to be "10, 8" in sequence, and the second weight indices can be determined to be "2, 4" in sequence.

[0137] In this embodiment, a first importance score is determined based on the statistical characteristics and first weight of the online activation data. The sparse information of the online activation data is combined with the first weight to dynamically filter invalid parameters. The weight parameters that need to be retained are determined based on the first importance score, thereby realizing weight filtering that changes with the online activation data and improving the sparsity of the neural network model.

[0138] The following combination Figure 5 and Figure 6 The hybrid pruning method for a neural network model provided in the embodiments of this application will be further described.

[0139] Figure 5 A schematic diagram of a hybrid pruning method for a neural network model provided in this application embodiment. Figure 1 The following is combined Figure 5 The possible implementations of this application are further described below.

[0140] During the offline training phase of the neural network model, the initial weights are first pruned based on the non-zero weights in the initial weights to obtain the third weights, for example, "7, 2, 1, 2". Then, based on the offline activation data and the third weights, the second importance score of each weight in the third weights is determined, and the weights with a second importance score greater than a second threshold are determined as the fourth weights. Finally, the neural network model using the fourth weights is trained to obtain the first weight, thus compensating for the accuracy loss caused by weight pruning.

[0141] For example, the third weight is "7, 2, 1, 2", and the offline activation data is "2, 1, 1, 3". Multiplying the offline activation data and the third weight yields second importance scores of "14, 2, 1, 6". With the second threshold TH1 set to 5, the weights in the third weight whose second importance scores are greater than the second threshold are designated as the fourth weight. The fourth weight, obtained by offline pruning of the third weight, has weight values ​​of "7, 2" and indices of "1, 4". To compensate for the accuracy loss caused by weight pruning, the neural network model using the fourth weight is trained, and the weight values ​​in the fourth weight are adjusted to "7, 3", while the index values ​​remain unchanged, thus obtaining the first weight. The weight values ​​of the first weight are "7, 3", and the weight index values ​​are "1, 4".

[0142] Figure 6 A schematic diagram of a hybrid pruning method for a neural network model provided in this application embodiment. Figure 2 The following is combined Figure 6 The possible implementations of this application are further described below.

[0143] Before performing online pruning on the neural network model, the statistical characteristics of the online activation data are first determined. Then, based on the statistical characteristics and the first weight, the first importance score of each weight in the first weight is determined. The first weight is then pruned according to the first importance score to obtain the second weight. Finally, the second weight is used to perform calculations with the online activation data, thereby guiding the computation unit to skip invalid calculations and reduce the computational load of the neural network model.

[0144] For example, if the online activation data is "1, 3, 2, 2", the statistical characteristic of the online activation data is determined by calculating the mean, and the statistical characteristic of the online activation data is "2". Based on the statistical characteristic and the first weight, the first importance score in the first weight is determined to be "14, 2". With the first threshold TH2 set to 10, the weights in the first weight whose first importance score is greater than the first threshold are determined as the second weight. The second weight obtained by online pruning the first weight has a value of "7" and an index of "1". Finally, calculations are performed based on the second weight and the online activation data to reduce the computational load of the neural network model.

[0145] Optionally, the online activation data can be data optimized based on the sparse characteristics of the input data. For example, if the effective information of the speech signal is mainly concentrated in the low-frequency band, while the effective information in the high-frequency band is relatively sparse, the online activation data can be optimized based on the sparse characteristics of the speech signal to increase the weight of the low-frequency speech signal and decrease the weight of the high-frequency speech signal, thereby further reducing the computational load of the neural network model.

[0146] Figure 7 This is a schematic diagram of the structure of a hybrid pruning device for a neural network model provided in an embodiment of this application, as shown below. Figure 7 As shown, the hybrid pruning device 70 for the neural network model provided in this embodiment includes an acquisition module 701 and a determination module 702, wherein:

[0147] The acquisition module 701 is used to acquire the first weights of the neural network model. The first weights are weights that have been pruned beforehand.

[0148] The determination module 702 is used to perform online pruning of the first weights based on the statistical characteristics of the current online activation data of the neural network model, so as to obtain the second weights of the neural network model.

[0149] In one possible implementation, the determining module 702 is further configured to:

[0150] Determine the statistical characteristics of online activation data;

[0151] Based on statistical characteristics and the first weight, determine the first importance score of each weight in the first weight;

[0152] The first weight is pruned online based on the first importance score to obtain the second weight.

[0153] In one possible implementation, the determining module 702 is further configured to:

[0154] The product or sum of the statistical characteristics and the weights in the first weight is determined as the first importance score of each weight in the first weight.

[0155] In one possible implementation, the determining module 702 is further configured to:

[0156] The weights whose importance score in the first weight is greater than the first threshold are determined as the second weights.

[0157] In one possible implementation, the statistical characteristic is any one of the following or the result of operations on any one or more of the following: mean, extreme value, or norm.

[0158] In one possible implementation, the acquisition module 701 is further configured to:

[0159] Based on the non-zero weights in the initial weights of the neural network model, weight pruning is performed on the initial weights to obtain the third weights;

[0160] Based on the offline activation data of the neural network model, the third weight is pruned offline to obtain the first weight.

[0161] In one possible implementation, the acquisition module 701 is further configured to:

[0162] Based on offline activation data and the third weight, determine the second importance score of each weight in the third weight;

[0163] The third weight is pruned offline based on the second importance score to obtain the fourth weight;

[0164] The neural network model using the fourth weight is trained, and the fourth weight is adjusted during the training process to obtain the first weight.

[0165] In one possible implementation, the acquisition module 701 is further configured to:

[0166] The product or sum of the offline activation data and the corresponding weights in the third weight is determined as the second importance score of each weight in the third weight.

[0167] In one possible implementation, the acquisition module 701 is further configured to:

[0168] The weights whose second importance score in the third weight is greater than the second threshold are determined as the fourth weights.

[0169] The hybrid pruning device for the neural network model provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0170] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the device 80 further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0171] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to perform the above-described method.

[0172] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0173] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0174] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0175] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0176] This application also provides a speech processing method, including: acquiring speech data; inputting the speech data into a neural network model to obtain the speech processing result output by the neural network model, wherein the neural network model is a model pruned using the hybrid pruning method of the aforementioned neural network model.

[0177] This application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface and executes the hybrid pruning method of the neural network model described above.

[0178] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0179] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0180] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0181] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0182] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0185] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0186] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0187] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A hybrid pruning method for a neural network model, characterized in that, The method includes: Obtain the first weight of the neural network model, where the first weight is a weight obtained by pre-pruning. The first weights are pruned online based on the statistical characteristics of the current online activation data of the neural network model to obtain the second weights of the neural network model.

2. The method according to claim 1, characterized in that, The step of pruning the first weights online based on the statistical characteristics of the current online activation data of the neural network model to obtain the second weights of the neural network model includes: Determine the statistical characteristics of the online activation data; Based on the statistical characteristics and the first weight, determine the first importance score of each weight in the first weight; The first weight is pruned online based on the first importance score to obtain the second weight.

3. The method according to claim 2, characterized in that, The step of determining the first importance score of each weight in the first weight based on the statistical characteristics and the first weight includes: The product or sum of the statistical feature and each weight in the first weight is determined as the first importance score of each weight in the first weight.

4. The method according to claim 2, characterized in that, The step of pruning the first weight online based on the first importance score to obtain the second weight includes: The weights whose first importance score is greater than the first threshold in the first weights are determined as the second weights.

5. The method according to any one of claims 1-4, wherein the statistical feature is any one of the following or the result of an operation on any one or more of the following: mean, extreme value, or norm.

6. The method according to any one of claims 1-4, characterized in that, Obtaining the first weight of the neural network model includes: Based on the non-zero weights in the initial weights of the neural network model, weight pruning is performed on the initial weights to obtain the third weights; Based on the offline activation data of the neural network model, the third weight is pruned offline to obtain the first weight.

7. The method according to claim 6, characterized in that, The step of offline pruning of the third weight based on the offline activation data of the neural network model to obtain the first weight includes: Based on the offline activation data and the third weight, determine the second importance score of each weight in the third weight; The third weight is pruned offline based on the second importance score to obtain the fourth weight; The neural network model using the fourth weight is trained, and the fourth weight is adjusted during the training process to obtain the first weight.

8. The method according to claim 7, characterized in that, The step of determining the second importance score of each weight in the third weight based on the offline activation data and the third weight includes: The product or sum of the offline activation data and the corresponding weights in the third weight is determined as the second importance score of each weight in the third weight.

9. The method according to claim 7, characterized in that, The step of offline pruning of the third weight based on the second importance score to obtain the fourth weight includes: The weights in the third weights whose second importance score is greater than the second threshold are determined as the fourth weights.

10. A speech processing method, characterized in that, include: Acquire voice data; The speech data is input into a neural network model to obtain the speech processing result output by the neural network model, wherein the neural network model is a model pruned using the method described in any one of claims 1-9.

11. A chip, characterized in that, The chip includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface and executes the method as described in any one of claims 1-9.

12. A hybrid pruning device for a neural network model, characterized in that, The device includes: an acquisition module and a determination module, wherein: The acquisition module is used to acquire the first weight of the neural network model, wherein the first weight is a weight obtained in advance through pruning; The determining module is used to perform online pruning on the first weights based on the statistical characteristics of the current online activation data of the neural network model, so as to obtain the second weights of the neural network model.

13. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-9.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images