A method and device for processing target tasks
By cutting and processing the channel importance analysis of neural network models, an efficient neural network model is generated, which solves the problem of low pruning efficiency of neural network models in the prior art, and is suitable for resource-constrained devices and real-time applications.
Patent Information
- Application Number
- CN202411187146.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The existing neural network model pruning technology has low processing efficiency, resulting in poor pruning processing efficiency of the model and cannot effectively meet the needs of resource-constrained devices and real-time applications.
By performing weight, local and global cropping processing based on the importance analysis of channel weight values and eigenvalues in the neural network model, an efficient neural network model is generated to handle tasks such as image recognition, natural language processing, and speech recognition.
It improves the efficiency and accuracy of the pruning processing of neural network models, reduces computing and storage requirements, and is suitable for resource-constrained devices and real-time applications.
Smart Images

Figure CN118690818B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a target task processing method and device, and also to an electronic device, a non-transitory computer-readable storage medium and a computer program product. Background Art
[0002] In recent years, with the rapid development of deep learning technology, neural network models have been increasingly used in various fields, such as image recognition, natural language processing, speech recognition, and autonomous driving. Different application scenarios have different requirements for the performance, efficiency, and resource requirements of neural network models, so they need to be optimized and adjusted in a targeted manner. Neural network models usually have a large number of parameters and complex structures, which leads to high computational and storage costs. They bring many challenges to resource-constrained devices and real-time requirements in practical applications. To this end, neural network model pruning technology has emerged to improve the efficiency of neural network models by reducing the parameters and computational complexity of neural network models. It is specifically reflected in the following aspects: Neural network model pruning can significantly reduce the computational complexity of the model, reduce the computational time during inference, and thus improve the response speed of the model. It is very important for applications that require real-time processing. Through pruning, the number of parameters of the neural network model is greatly reduced, thereby reducing the storage requirements of the neural network model. For devices with limited storage resources, this can effectively save storage space and facilitate the deployment and transmission of the model. Reducing the amount of computation and storage requirements directly leads to a reduction in energy consumption, which is very important for devices that need to run for a long time and have limited battery capacity. Through pruning technology, the energy consumption of the equipment can be reduced. Therefore, how to design a more efficient target task processing solution has become a technical problem that needs to be solved urgently. Summary of the invention
[0003] The present invention provides a target task processing method to solve the defect that the neural network model pruning technology in the prior art has high limitations, resulting in poor model pruning processing efficiency.
[0004] The present invention provides a target task processing method, comprising:
[0005] Determine the importance of the channels of each network layer in the neural network model to be pruned based on the weight values and characteristic values corresponding to the channels of each network layer in the neural network model to be pruned;
[0006] Based on the importance value of the channel, pruning the channels of each network layer in the neural network model to be pruned to obtain a neural network model after weight pruning;
[0007] Performing local pruning processing on the channels of each network layer in the neural network model after weight pruning to obtain a local pruned neural network model;
[0008] Performing global pruning processing on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model;
[0009] The target task to be processed is input into the globally pruned neural network model, and the target task to be processed is processed based on the globally pruned neural network model to obtain a processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0010] According to the target task processing method provided by the present invention, the weight values and characteristic values corresponding to the channels of each network layer in the neural network model to be pruned are used to determine the importance values of the channels of each network layer in the neural network model to be pruned, which specifically includes:
[0011] Determine the value of the first importance of the channel of each network layer in the neural network model to be pruned based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned;
[0012] Determine the value of the second importance of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned;
[0013] Based on the value of the first importance of the channel of each network layer in the neural network model to be pruned and the value of the second importance of the channel of each network layer in the neural network model to be pruned, the value of the importance of the channel of each network layer in the neural network model to be pruned is determined.
[0014] According to the target task processing method provided by the present invention, the first importance value of the channel of each network layer in the neural network model to be pruned is determined based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned, specifically including:
[0015] The weight values corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset weight importance analysis model to obtain the first importance values of the channels of the various network layers in the neural network model to be pruned output by the weight importance analysis model.
[0016] According to the target task processing method provided by the present invention, the determining of the second importance value of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned specifically includes:
[0017] The eigenvalues corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset eigenvalue importance analysis model to obtain the second importance values of the channels of the various network layers in the neural network model to be pruned output by the eigenvalue importance analysis model.
[0018] According to the target task processing method provided by the present invention, the channels of each network layer in the neural network model to be pruned are pruned based on the importance value of the channel to obtain the neural network model after weight pruning, which specifically includes:
[0019] The importance values of the channels of each network layer in the neural network model are compared and analyzed with the preset importance threshold. When there are values of the importance of the channels of each network layer in the neural network model that are less than the preset importance threshold, the destination channels whose importance values of the channels of each network layer in the neural network model are less than the preset importance threshold are deleted, and the connections associated with the destination channels in each network layer in the neural network model are deleted to obtain the neural network model after weight pruning.
[0020] According to the target task processing method provided by the present invention, the channels of each network layer in the weight pruned neural network model are locally pruned to obtain the locally pruned neural network model, specifically comprising:
[0021] All channels of each network layer in the neural network model after weight pruning are traversed to determine similar channels of each network layer in the neural network model after weight pruning, and similar channels of each network layer in the neural network model after weight pruning are pruned respectively to obtain a locally pruned neural network model.
[0022] According to the target task processing method provided by the present invention, all channels of each network layer in the neural network model after weight pruning are traversed, similar channels of each network layer in the neural network model after weight pruning are determined, and similar channels of each network layer in the neural network model after weight pruning are pruned respectively to obtain a locally pruned neural network model, specifically including:
[0023] Inputting the data in the first verification sample set into the weight pruned neural network model, obtaining the characteristic values of all channels corresponding to each network layer in the weight pruned neural network model, and sequentially determining the similarity values between the characteristic values of any two target channels in all the channels;
[0024] In a case where the similarity value between the feature values of the two target channels is greater than a preset similarity threshold, determining that the two target channels are non-similar channels, and retaining the two target channels;
[0025] When the similarity value between the feature values of the two target channels is less than or equal to a preset similarity threshold, the two target channels are determined to be similar channels, and one of the two target channels is pruned to obtain a locally pruned neural network model.
[0026] According to the target task processing method provided by the present invention, pruning one of the two target channels to obtain a locally pruned neural network model specifically includes:
[0027] Normalizing the elements in the feature maps corresponding to the feature values of the two target channels respectively so that the elements in the feature maps corresponding to the feature values of the two target channels are at the same magnitude; wherein the two target channels include a first target channel and a second target channel;
[0028] Determine a first boundary value of a first type element and a second type element in the first target channel;
[0029] Determine a second boundary value of the first type element and the second type element in the second target channel;
[0030] Determining a first quantity ratio of the first type of elements to the second type of elements in the first target channel according to the first limiting value;
[0031] Determining a second quantity ratio of the first type of elements to the second type of elements in the second target channel according to the second limiting value;
[0032] Compare and analyze the first quantity ratio and the second quantity ratio to obtain a comparison result;
[0033] According to the comparison result, deleting a target channel corresponding to a feature map having a smaller quantity ratio between the first quantity ratio and the second quantity ratio and a connection associated with the target channel;
[0034] After traversing all channels of all network layers in the neural network model after weight pruning, a locally pruned neural network model is obtained.
[0035] According to the target task processing method provided by the present invention, the channels of each network layer in the locally pruned neural network model are globally pruned to obtain the globally pruned neural network model, specifically comprising:
[0036] All channels of each network layer in the locally pruned neural network model are traversed to determine invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain a globally pruned neural network model.
[0037] According to the target task processing method provided by the present invention, all channels of each network layer in the locally pruned neural network model are traversed to determine the invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain the globally pruned neural network model, specifically comprising:
[0038] Inputting the data in the second verification sample set into the locally pruned neural network model to obtain a first loss value corresponding to the locally pruned neural network model;
[0039] Determine a second loss value corresponding to the locally pruned neural network model in a case where a first channel corresponding to a first network layer of the locally pruned neural network model is pre-deleted;
[0040] determining a first difference between the first loss value and the second loss value;
[0041] In the case where the first difference is less than or equal to a preset loss threshold, determining the first channel as an invalid associated channel, and deleting the first channel and the corresponding connection;
[0042] In the case where the first difference is greater than a preset loss threshold, determining that the first channel is a non-invalid associated channel, sequentially traversing the remaining channels corresponding to the first network layer of the neural network model after local pruning, and comparing and analyzing the corresponding loss values to obtain an intermediate neural network model;
[0043] All channels corresponding to the remaining network layers in the intermediate neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the neural network model after global pruning.
[0044] The target task processing method provided by the present invention further includes: when the first difference is greater than a preset loss threshold, pre-deleting all channels corresponding to the first network layer of the locally pruned neural network model,
[0045] Determine a third loss value corresponding to the locally pruned neural network model;
[0046] determining a second difference between the first loss value and the third loss value;
[0047] When the second difference is less than or equal to a preset loss threshold, all channels corresponding to the first network layer are determined as invalid associated channels, and all channels and corresponding connections corresponding to the first network layer are deleted;
[0048] When the second difference is greater than a preset loss threshold, it is determined that there are no invalid associated channels in the first network layer, and the remaining network layers of the locally pruned neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the globally pruned neural network model.
[0049] The present invention also provides a target task processing device, comprising:
[0050] The channel importance analysis unit is used to determine the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned.
[0051] The unstructured pruning processing unit is used to prune the channels of each network layer in the neural network model to be pruned based on the importance value of the channel to obtain the neural network model after weight pruning.
[0052] The local pruning processing unit is used to perform local pruning processing on the channels of each network layer in the neural network model after weight pruning to obtain the neural network model after local pruning.
[0053] A global pruning processing unit, used to perform global pruning processing on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model;
[0054] A target task processing unit is used to input the target task to be processed into the globally pruned neural network model, process the target task to be processed based on the globally pruned neural network model, and obtain a processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0055] According to the target task processing device provided by the present invention, the channel importance analysis unit is specifically used for:
[0056] Determine the value of the first importance of the channel of each network layer in the neural network model to be pruned based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned;
[0057] Determine the value of the second importance of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned;
[0058] Based on the value of the first importance of the channel of each network layer in the neural network model to be pruned and the value of the second importance of the channel of each network layer in the neural network model to be pruned, the value of the importance of the channel of each network layer in the neural network model to be pruned is determined.
[0059] According to the target task processing device provided by the present invention, the first importance value of the channel of each network layer in the neural network model to be pruned is determined based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned, specifically including:
[0060] The weight values corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset weight importance analysis model to obtain the first importance values of the channels of the various network layers in the neural network model to be pruned output by the weight importance analysis model.
[0061] According to the target task processing device provided by the present invention, the determining of the second importance value of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned specifically includes:
[0062] The eigenvalues corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset eigenvalue importance analysis model to obtain the second importance values of the channels of the various network layers in the neural network model to be pruned output by the eigenvalue importance analysis model.
[0063] According to the target task processing device provided by the present invention, the unstructured cutting processing unit is specifically used for:
[0064] The importance values of the channels of each network layer in the neural network model are compared and analyzed with the preset importance threshold. When there are values of the importance of the channels of each network layer in the neural network model that are less than the preset importance threshold, the destination channels whose importance values of the channels of each network layer in the neural network model are less than the preset importance threshold are deleted, and the connections associated with the destination channels in each network layer in the neural network model are deleted to obtain the neural network model after weight pruning.
[0065] According to the target task processing device provided by the present invention, the local cropping processing unit is specifically used for:
[0066] All channels of each network layer in the neural network model after weight pruning are traversed to determine similar channels of each network layer in the neural network model after weight pruning, and similar channels of each network layer in the neural network model after weight pruning are pruned respectively to obtain a locally pruned neural network model.
[0067] According to the target task processing device provided by the present invention, all channels of each network layer in the weight pruned neural network model are traversed to determine similar channels of each network layer in the weight pruned neural network model, and similar channels of each network layer in the weight pruned neural network model are pruned respectively to obtain a locally pruned neural network model, specifically including:
[0068] Inputting the data in the first verification sample set into the weight pruned neural network model, obtaining the characteristic values of all channels corresponding to each network layer in the weight pruned neural network model, and sequentially determining the similarity values between the characteristic values of any two target channels in all the channels;
[0069] In a case where the similarity value between the feature values of the two target channels is greater than a preset similarity threshold, determining that the two target channels are non-similar channels, and retaining the two target channels;
[0070] When the similarity value between the feature values of the two target channels is less than or equal to a preset similarity threshold, the two target channels are determined to be similar channels, and one of the two target channels is pruned to obtain a locally pruned neural network model.
[0071] According to the target task processing device provided by the present invention, the pruning of one of the two target channels to obtain a locally pruned neural network model specifically includes:
[0072] Normalizing the elements in the feature maps corresponding to the feature values of the two target channels respectively so that the elements in the feature maps corresponding to the feature values of the two target channels are at the same magnitude; wherein the two target channels include a first target channel and a second target channel;
[0073] Determine a first boundary value of a first type element and a second type element in the first target channel;
[0074] Determine a second boundary value of the first type element and the second type element in the second target channel;
[0075] Determining a first quantity ratio of the first type of elements to the second type of elements in the first target channel according to the first limiting value;
[0076] Determining a second quantity ratio of the first type of elements to the second type of elements in the second target channel according to the second limiting value;
[0077] Compare and analyze the first quantity ratio and the second quantity ratio to obtain a comparison result;
[0078] According to the comparison result, deleting a target channel corresponding to a feature map having a smaller quantity ratio between the first quantity ratio and the second quantity ratio and a connection associated with the target channel;
[0079] After traversing all channels of all network layers in the neural network model after weight pruning, a locally pruned neural network model is obtained.
[0080] According to the target task processing device provided by the present invention, the global clipping processing unit is specifically used for:
[0081] All channels of each network layer in the locally pruned neural network model are traversed to determine invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain a globally pruned neural network model.
[0082] According to the target task processing device provided by the present invention, all channels of each network layer in the locally pruned neural network model are traversed to determine the invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain the globally pruned neural network model, specifically including:
[0083] Inputting the data in the second verification sample set into the locally pruned neural network model to obtain a first loss value corresponding to the locally pruned neural network model;
[0084] Determine a second loss value corresponding to the locally pruned neural network model in a case where a first channel corresponding to a first network layer of the locally pruned neural network model is pre-deleted;
[0085] determining a first difference between the first loss value and the second loss value;
[0086] In the case where the first difference is less than or equal to a preset loss threshold, determining the first channel as an invalid associated channel, and deleting the first channel and the corresponding connection;
[0087] In the case where the first difference is greater than a preset loss threshold, determining that the first channel is a non-invalid associated channel, sequentially traversing the remaining channels corresponding to the first network layer of the neural network model after local pruning, and comparing and analyzing the corresponding loss values to obtain an intermediate neural network model;
[0088] All channels corresponding to the remaining network layers in the intermediate neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the neural network model after global pruning.
[0089] The target task processing device provided by the present invention further includes: when the first difference is greater than a preset loss threshold, pre-deleting all channels corresponding to the first network layer of the neural network model after local pruning,
[0090] Determine a third loss value corresponding to the locally pruned neural network model;
[0091] determining a second difference between the first loss value and the third loss value;
[0092] When the second difference is less than or equal to a preset loss threshold, all channels corresponding to the first network layer are determined as invalid associated channels, and all channels and corresponding connections corresponding to the first network layer are deleted;
[0093] When the second difference is greater than a preset loss threshold, it is determined that there are no invalid associated channels in the first network layer, and the remaining network layers of the locally pruned neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the globally pruned neural network model.
[0094] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the target task processing method as described in any one of the above items when executing the program.
[0095] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the target task processing method as described in any of the above items.
[0096] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the target task processing method as described in any one of the above items is implemented.
[0097] The target task processing method provided by the present invention determines the importance value of the channel of each network layer in the neural network model to be pruned by the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned, prunes the channel of each network layer in the neural network model to be pruned based on the value of the channel importance to obtain the neural network model after weight pruning, locally prunes the channel of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model, and globally prunes the channel of each network layer in the locally pruned neural network model to obtain the globally pruned neural network model, which can effectively improve the pruning efficiency and accuracy of the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS
[0098] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0099] Figure 1 It is one of the flow charts of the target task processing method provided by the present invention.
[0100] Figure 2 This is the second flowchart of the target task processing method provided by the present invention.
[0101] Figure 3 This is the third flow chart of the target task processing method provided by the present invention.
[0102] Figure 4 This is the fourth flow chart of the target task processing method provided by the present invention.
[0103] Figure 5It is a structural schematic diagram of a neural network model in the target task processing method provided by the present invention.
[0104] Figure 6 It is a structural schematic diagram of the target task processing device provided by the present invention.
[0105] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0106] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0107] Combine the following Figure 1-Figure 7 The target task processing method and device of the present invention are described, and the embodiments thereof are described in detail.
[0108] Based on the target task processing method described in the present invention, the embodiment is described in detail below. Figure 1 As shown, it is one of the flow charts of the target task processing method provided by the present invention, and the specific implementation process includes the following steps:
[0109] Step 101: Determine the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned.
[0110] In an embodiment of the present invention, a trained neural network model can be obtained in advance as a neural network model to be pruned and a verification sample set. The data in the verification sample set includes pictures and corresponding labels, and the pictures and corresponding labels are in a one-to-one correspondence. In the actual implementation process, it is necessary to pre-process the data in advance, input the pre-processed data into the original neural network model for training, and obtain a trained neural network model; wherein the data processing method is the same as the processing method when the neural network model is trained. Since the neural network model to be pruned has been trained, that is, the neural network model contains the weight values corresponding to the channels (i.e., channels) of each network layer and the values of the corresponding feature maps (i.e., feature values), therefore, before executing step 1, the weight values and feature values corresponding to the channels of each network layer in the neural network model memory can be obtained, and then the first importance values of the channels of each network layer in the neural network model to be pruned can be determined based on the weight values corresponding to the channels of each network layer in the neural network model to be pruned. And based on the feature values corresponding to the channels of each network layer in the neural network model to be pruned, the second importance values of the channels of each network layer in the neural network model to be pruned are determined. Then, based on the first importance value of the channel of each network layer in the neural network model to be pruned and the second importance value of the channel of each network layer in the neural network model to be pruned, the importance value of the channel of each network layer in the neural network model to be pruned is determined. The network layer may be a convolutional layer, a pooling layer, a fully connected layer, etc. in the neural network model.
[0111] Among them, when determining the first importance value of the channel of each network layer in the neural network model to be pruned based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned, the weight value corresponding to each channel of each network layer in the neural network model to be pruned can be input into a preset weight importance analysis model to obtain the first importance value of each channel of each network layer in the neural network model to be pruned output by the weight importance analysis model.
[0112] When determining the second importance value of the channel of each network layer in the neural network model to be pruned based on the eigenvalues corresponding to the channels of each network layer in the neural network model to be pruned, the eigenvalues corresponding to the channels of each network layer in the neural network model to be pruned can be input into a preset eigenvalue importance analysis model to obtain the second importance value of each channel of each network layer in the neural network model to be pruned output by the eigenvalue importance analysis model.
[0113] Specifically, the formula corresponding to the weight importance analysis model is as follows:
[0114]
[0115] In an embodiment of the present invention, in the process of calculating the first importance value of the channel of each network layer in the neural network model to be pruned, the rank of the influencing factor weight and the mean of the weight are introduced to fully consider the characteristic information of each channel, thereby preventing the channel from being deleted by mistake.
[0116] Specifically, the algorithm corresponding to the eigenvalue importance analysis model is as follows:
[0117]
[0118] Based on the first importance value of the channel of each network layer in the neural network model to be pruned and the second importance value of the channel of each network layer in the neural network model to be pruned, the formula corresponding to the importance value of the channel of each network layer in the neural network model to be pruned is calculated as follows:
[0119]
[0120] Among the above two formulas, , ,and represents the hyperparameter, Indicates the weight corresponding to the channel The value corresponding to the jth row and the kth column is, Indicates the weight corresponding to the channel The value of the rank (i.e. the size of the rank); Indicates the weight corresponding to the channel The value corresponding to the jth row and the kth column is express The value of the corresponding norm; This is the value of the second importance; That is the value of first importance; is the importance value of one of the channels in one of the network layers. The weight value includes the weight corresponding to the channel The value corresponding to the jth row and the kth column and the weight corresponding to the channel The value of the rank.
[0121] Step 102: based on the importance value of the channel, prune the channels of each network layer in the neural network model to be pruned to obtain a neural network model after weight pruning.
[0122] Specifically, the importance values of the channels of each network layer in the neural network model can be compared and analyzed with a preset importance threshold. If there are values of the importance of the channels of each network layer in the neural network model that are less than the preset importance threshold, the destination channels whose importance values of the channels of each network layer in the neural network model are less than the preset importance threshold are deleted, and the connections associated with the destination channels in each network layer in the neural network model are deleted to obtain a neural network model after weight pruning. Figure 2 As shown, in judging the value of importance (i.e. ) is greater than or equal to its threshold (i.e., the preset importance threshold), if not, delete the corresponding channel and delete the affected connection, if so, execute the next step; enter the local pruning optimization system (i.e., step 103), further prune the weight channel corresponding to each layer; further, enter the global pruning optimization system (i.e., step 104), according to the global dependency, further prune the weight channel corresponding to each layer, and obtain the pruned neural network model; STEP9: fine-tune the globally pruned neural network model; obtain the final globally pruned neural network model. In the determination of unstructured pruning, the present invention introduces the importance of the weight value and the importance of the eigenvalue (i.e., the value of the feature map) corresponding to the weight value to jointly determine the importance of the channel, and introduces influencing factors such as weight rank and weight mean in the importance of the weight value, fully considering the characteristics of the channel, thereby effectively preventing the channel from being deleted by mistake.
[0123] Step 103: Perform local pruning on the channels of each network layer in the neural network model after weight pruning to obtain a local pruned neural network model.
[0124] In an embodiment of the present invention, all channels of each network layer in the neural network model after weight pruning can be traversed, and similar channels of each network layer in the neural network model after weight pruning can be determined respectively, and then similar channels of each network layer in the neural network model after weight pruning can be pruned respectively to obtain a locally pruned neural network model.
[0125] Specifically, the data in the preset first verification sample set can be input into the neural network model after weight pruning to obtain the characteristic values of all channels corresponding to each network layer in the neural network model after weight pruning, and the similarity values between the characteristic values of any two target channels in all the channels are determined in turn. When the similarity value between the characteristic values of the two target channels is greater than the preset similarity threshold, the two target channels are determined to be non-similar channels and the two target channels are retained. When the similarity value between the characteristic values of the two target channels is less than or equal to the preset similarity threshold, the two target channels are determined to be similar channels, and one of the two target channels is pruned to obtain a neural network model after local pruning.
[0126] In the process of pruning one of the two target channels to obtain the neural network model after local pruning, each element in the feature map corresponding to the feature values of the two target channels can be normalized respectively so that each element in the feature map corresponding to the feature values of the two target channels is at the same magnitude; wherein the two target channels include a first target channel and a second target channel. Determine a first limit value of a first type element and a second type element in the first target channel. Determine a second limit value of a first type element and a second type element in the second target channel. According to the first limit value, determine a first quantity ratio of the first type element and the second type element in the first target channel; according to the second limit value, determine a second quantity ratio of the first type element and the second type element in the second target channel. Compare and analyze the first quantity ratio with the second quantity ratio to obtain a comparison result; according to the comparison result, delete the target channel corresponding to the feature map with a smaller quantity ratio between the first quantity ratio and the second quantity ratio and the connection associated with the target channel; after traversing all channels of all network layers in the neural network model after weight pruning, obtain the neural network model after local pruning.
[0127] The complete optimization process of the local cutting process of the present invention is as follows: Figure 3 As shown, the main steps are as follows: STEP1: Select the first verification sample set; wherein the data in the first verification sample set includes pictures and corresponding labels, and the pictures and corresponding labels correspond to each other. STEP2: Preprocess the data; wherein the data processing method is the same as the processing method when the neural network model is trained; STEP3: Input the preprocessed data into the neural network model after weight pruning; STEP4: Obtain the eigenvalues of all channels of the network layer; STEP5: Calculate the similarity between two different eigenvalues; wherein the value calculation of the similarity is determined by the following formula:
[0128]
[0129] in, express In the i Line j The value corresponding to the column, express The value corresponding to the i-th row and j-th column is, express The mean of express The mean of ; s is the similarity value between the feature values of any two target channels in the network layer; n is the i In the row i The corresponding integer (n≥1); m is the j In the row j The corresponding integer (m ≥ 1).
[0130] Further, STEP6: calculate whether the value of its similarity is greater than a set threshold (i.e., a preset similarity threshold), if not, retain the channel (i.e., retain the two target channels), if yes, execute the next step; STEP7: normalize the eigenvalues of the two target channels respectively, so that the elements in the feature maps corresponding to the eigenvalues of the two target channels are at the same order of magnitude; STEP8: select the boundary value of large elements (i.e., first type elements) and small elements (i.e., second type elements); for example, the boundary value can be set to 0.6; STEP9: calculate the ratio of the number of large elements to the number of small elements; the calculation formula corresponding to the ratio of the number of large elements to the number of small elements is as follows:
[0131]
[0132] in, Indicates the number of large elements, Represents the number of small elements, Indicates the current element value. Indicates the set limit value.
[0133] The calculating the quantity ratio of the number of large elements to the number of small elements includes: determining a first quantity ratio of the first type of elements to the second type of elements in the first target channel according to the first limiting value; and determining a second quantity ratio of the first type of elements to the second type of elements in the second target channel according to the second limiting value. In the embodiment of the present invention, it is the first quantity ratio or the second quantity ratio.
[0134] STEP10: Delete the channels corresponding to the feature maps with smaller ratios, and the connections of their channels; STEP11: Check whether all channels of this layer have been compared. If so, execute the next step. If not, return to execute STEP5 until the feature values corresponding to all channels of this layer have been compared; STEP12: Prune the channels of other layers in turn; STEP13: Wait to obtain the pruned neural network model; STEP14: The program ends. In the process of structured pruning, the present invention adopts a combination of local optimization pruning and global optimization pruning to further improve the pruning efficiency of the model. Local pruning can preferentially remove similar channels in the model layer and reduce the number of traversals in global pruning. In the local pruning, the present invention further determines the channels to be discarded by calculating the ratio of the number of large elements to the number of small elements, which is more conducive to retaining effective features and preventing accidental deletion of channels.
[0135] Step 104: Perform global pruning on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model.
[0136] In an embodiment of the present invention, all channels of each network layer in the locally pruned neural network model can be traversed to determine the invalid associated channels of each network layer in the locally pruned neural network model, and then the invalid associated channels of each network layer in the locally pruned neural network model can be pruned to obtain a globally pruned neural network model.
[0137] Specifically, the data in the second verification sample set is input into the locally pruned neural network model to obtain the first loss value corresponding to the locally pruned neural network model; in the case of pre-deleting the first channel corresponding to the first network layer of the locally pruned neural network model, the second loss value corresponding to the locally pruned neural network model is determined. The first difference between the first loss value and the second loss value is determined; in the case that the first difference is less than or equal to the preset loss threshold, the first channel is determined to be an invalid associated channel, and the first channel and the corresponding connection are deleted. In the case that the first difference is greater than the preset loss threshold, the first channel is determined to be a non-invalid associated channel, and the remaining channels corresponding to the first network layer of the locally pruned neural network model are traversed in turn, and the corresponding loss values are compared and analyzed to obtain an intermediate neural network model; all channels corresponding to the remaining network layers in the intermediate neural network model are traversed in turn, and compared and analyzed according to the loss values to obtain a globally pruned neural network model.
[0138] In addition, when the first difference is greater than the preset loss threshold, it also includes: pre-deleting all channels corresponding to the first network layer of the locally pruned neural network model, determining the third loss value corresponding to the locally pruned neural network model; determining the second difference between the first loss value and the third loss value; when the second difference is less than or equal to the preset loss threshold, determining all channels corresponding to the first network layer as invalid associated channels, and deleting all channels and corresponding connections corresponding to the first network layer. When the second difference is greater than the preset loss threshold, it is determined that there are no invalid associated channels in the first network layer, and the remaining network layers of the locally pruned neural network model are traversed in turn, and compared and analyzed according to the loss values to obtain the globally pruned neural network model.
[0139] The optimization flow chart of the global clipping process of the present invention is as follows: Figure 4As shown, the main steps are as follows: STEP 1: Select a second verification sample set; wherein the data in the second verification sample set also includes pictures and corresponding labels, and the pictures and corresponding labels correspond one to one. STEP2: Preprocess the data; wherein the data processing method is the same as the processing method during the training of the neural network model; STEP3: Input the processed data into the neural network model after local pruning; STEP4: Calculate the overall loss value L1 (i.e., the first loss value) of the locally pruned neural network model based on the image and label value; the calculation of the loss value L1 adopts the loss calculation method during the training of the locally pruned neural network model; STEP5: Delete the first channel of the first network layer of the locally pruned neural network model; STEP6: Calculate the loss value L2 (i.e., the second loss value) of the model after pre-deleting the first channel; wherein the calculation method of the loss value L2 is the same as the calculation method of the loss value L1; STEP7: Calculate the first difference between the loss value L1 and the loss value L2, whether the first difference is less than or equal to the set threshold (Juin preset loss threshold), if so, delete the first channel and the corresponding connection, if not, execute the next step; STEP 8: Delete all channels of the subsequent network layer one by one; wherein, mainly determine the relationship between channels in different network layers; STEP9: Recalculate the loss value L3 (i.e., the third loss value) of the neural network model; STEP10: Calculate the second difference between the loss value L1 and the loss value L3, whether the second difference is less than or equal to the set threshold (i.e., the preset loss threshold), if so, delete all selected channels and corresponding connections, otherwise proceed to the next step; STEP11: Determine whether all channels of all subsequent network layers have been deleted one by one, if otherwise, execute STEP8, if so, execute the next step; STEP12: If the second difference between L1 and L3 is greater than the set threshold, execute the next network layer; STEP13: Determine whether each channel of each subsequent network layer has been deleted one by one, if otherwise, delete each channel of each network layer in turn, and return to STEP8, if so, execute the next step; STEP14: Wait to obtain the pruned neural network model; STEP15: Execution ends. Among them, the relationship between each channel in each network layer is as follows: Figure 5 As shown, layer 1 channel 1 can be the first channel of the first network layer, layer 1 channel 2 can be the second channel of the first network layer, and other descriptions are not repeated here. In the global optimization process, the present invention uses layer-by-layer main channel pruning, compares the ratio of the entire model loss function and the original loss function, can further determine the dependency between the layers, and further delete invalid associated channels.
[0140] Step 105: input the target task to be processed into the globally pruned neural network model, process the target task to be processed based on the globally pruned neural network model, and obtain a processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0141] In an embodiment of the present invention, when the target task to be processed is a speech recognition task, multiple pieces of speech data to be recognized can be input into the globally pruned neural network model, and speech recognition processing can be performed on the multiple pieces of speech data to be recognized based on the convolution kernels of each network layer after the corresponding mixed and pruned processing in the globally pruned neural network model, to obtain the recognition processing results of the corresponding speech data. The convolution kernels of each network layer after the corresponding mixed and pruned processing are obtained by mixed and pruned according to the characteristics of the speech recognition task (such as focusing on recognizing speech data) and the importance values of the corresponding channels.
[0142] In the case where the target task to be processed is an autonomous driving task, the multiple materials used for autonomous driving to be identified (such as the current driving road conditions or driving instructions) can be input into the globally pruned neural network model, and based on the convolution kernels of each network layer after the corresponding mixed clipping processing in the globally pruned neural network model, the multiple materials used for autonomous driving to be identified are respectively subjected to autonomous driving analysis and processing to obtain the autonomous driving analysis and processing results of the corresponding multiple materials used for autonomous driving (such as vehicle control instructions). Among them, the convolution kernels of each network layer after the corresponding mixed clipping processing are obtained by mixed clipping according to the characteristics of the autonomous driving task (such as focusing on identifying the relevant driving road conditions or driving instruction data for autonomous driving) and the importance values of the corresponding channels.
[0143] In the case where the target task to be processed is a natural language processing task, multiple natural language processing tasks to be identified can be input into the globally pruned neural network model, and the multiple natural language processing tasks to be identified can be processed respectively based on the convolution kernels of each network layer after the corresponding mixed pruned processing in the globally pruned neural network model to obtain the processing results of the corresponding natural language processing tasks. Among them, natural language processing tasks include text generation, text classification, sentiment analysis, machine translation, etc. Among them, the convolution kernels of each network layer after the corresponding mixed pruned processing are obtained by mixed pruned according to the characteristics of the natural language processing task (such as focusing on identifying text-related data) and the importance value of the corresponding channel.
[0144] In the case where the target task to be processed is an image recognition task, the multiple image data to be recognized can be input into the globally pruned neural network model, and based on the convolution kernels of each network layer after the corresponding mixed cropping processing in the globally pruned neural network model, the multiple image data to be recognized are respectively subjected to image recognition processing to obtain the recognition processing results of the corresponding image data. The convolution kernels of each network layer after the corresponding mixed cropping processing are obtained by mixed cropping according to the characteristics of the image recognition task (such as focusing on recognizing image data) and the importance values of the corresponding channels.
[0145] In the specific implementation process, the globally pruned neural network model can be deployed on a processor in an electronic device with computing capabilities, so that the processor in the electronic device can dynamically prune each input target task through the globally pruned neural network model, and the electronic device deployed with the globally pruned neural network model is used to process image recognition tasks, natural language processing tasks, speech recognition tasks, autonomous driving tasks and other target tasks. The electronic device can first select a globally pruned neural network model suitable for processing the target task from the existing multiple types of globally pruned neural network models, or it can construct the network layer convolution kernel (a convolution kernel can also be called a channel) of the globally pruned neural network model that meets the target task. The selected neural network model or the self-constructed neural network model has different network layer convolution kernels after training for different target task training samples. Among them, the electronic device can be a smart watch, a smart phone, a smart car machine, etc.
[0146] The present invention adopts a method combining structured pruning and unstructured pruning, which can not only improve the pruning efficiency of the model, but also prevent uneven structured distribution, resulting in no reduction in required computing power. The importance of the channels of each network layer in the neural network model to be pruned is determined by the weight values and characteristic values corresponding to the channels of each network layer in the neural network model to be pruned, and the channels of each network layer in the neural network model to be pruned are pruned based on the importance values of the channels to obtain a neural network model after weight pruning, and the channels of each network layer in the neural network model after weight pruning are locally pruned to obtain a locally pruned neural network model, and the channels of each network layer in the locally pruned neural network model are globally pruned to obtain a globally pruned neural network model, which can effectively improve the pruning efficiency and accuracy of the neural network model.
[0147] The target task processing device provided by the present invention is described below. The target task processing device described below and the target task processing method described above can be referred to in correspondence with each other. Figure 6As shown, it is a schematic diagram of the structure of the target task processing device provided by the present invention. The target task processing device of the present invention specifically includes the following parts:
[0148] The channel importance analysis unit 601 is used to determine the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned.
[0149] The unstructured pruning processing unit 602 is used to prune the channels of each network layer in the neural network model to be pruned based on the importance value of the channel to obtain the neural network model after weight pruning.
[0150] The local pruning processing unit 603 is used to perform local pruning processing on the channels of each network layer in the neural network model after weight pruning to obtain the neural network model after local pruning.
[0151] The global pruning processing unit 604 is used to perform global pruning processing on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model.
[0152] The target task processing unit 605 is used to input the target task to be processed into the globally pruned neural network model, process the target task to be processed based on the globally pruned neural network model, and obtain the processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0153] According to the target task processing device provided by the present invention, the channel importance analysis unit is specifically used for:
[0154] Determine the value of the first importance of the channel of each network layer in the neural network model to be pruned based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned;
[0155] Determine the value of the second importance of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned;
[0156] Based on the value of the first importance of the channel of each network layer in the neural network model to be pruned and the value of the second importance of the channel of each network layer in the neural network model to be pruned, the value of the importance of the channel of each network layer in the neural network model to be pruned is determined.
[0157] According to the target task processing device provided by the present invention, the first importance value of the channel of each network layer in the neural network model to be pruned is determined based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned, specifically including:
[0158] The weight values corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset weight importance analysis model to obtain the first importance values of the channels of the various network layers in the neural network model to be pruned output by the weight importance analysis model.
[0159] According to the target task processing device provided by the present invention, the determining of the second importance value of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned specifically includes:
[0160] The eigenvalues corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset eigenvalue importance analysis model to obtain the second importance values of the channels of the various network layers in the neural network model to be pruned output by the eigenvalue importance analysis model.
[0161] According to the target task processing device provided by the present invention, the unstructured cutting processing unit is specifically used for:
[0162] The importance values of the channels of each network layer in the neural network model are compared and analyzed with the preset importance threshold. When there are values of the importance of the channels of each network layer in the neural network model that are less than the preset importance threshold, the destination channels whose importance values of the channels of each network layer in the neural network model are less than the preset importance threshold are deleted, and the connections associated with the destination channels in each network layer in the neural network model are deleted to obtain the neural network model after weight pruning.
[0163] According to the target task processing device provided by the present invention, the local cropping processing unit is specifically used for:
[0164] All channels of each network layer in the neural network model after weight pruning are traversed to determine similar channels of each network layer in the neural network model after weight pruning, and similar channels of each network layer in the neural network model after weight pruning are pruned respectively to obtain a locally pruned neural network model.
[0165] According to the target task processing device provided by the present invention, all channels of each network layer in the weight pruned neural network model are traversed to determine similar channels of each network layer in the weight pruned neural network model, and similar channels of each network layer in the weight pruned neural network model are pruned respectively to obtain a locally pruned neural network model, specifically including:
[0166] Inputting the data in the first verification sample set into the weight pruned neural network model, obtaining the characteristic values of all channels corresponding to each network layer in the weight pruned neural network model, and sequentially determining the similarity values between the characteristic values of any two target channels in all the channels;
[0167] In a case where the similarity value between the feature values of the two target channels is greater than a preset similarity threshold, determining that the two target channels are non-similar channels, and retaining the two target channels;
[0168] When the similarity value between the feature values of the two target channels is less than or equal to a preset similarity threshold, the two target channels are determined to be similar channels, and one of the two target channels is pruned to obtain a locally pruned neural network model.
[0169] According to the target task processing device provided by the present invention, the pruning of one of the two target channels to obtain a locally pruned neural network model specifically includes:
[0170] Normalizing the elements in the feature maps corresponding to the feature values of the two target channels respectively so that the elements in the feature maps corresponding to the feature values of the two target channels are at the same magnitude; wherein the two target channels include a first target channel and a second target channel;
[0171] Determine a first boundary value of a first type element and a second type element in the first target channel;
[0172] Determine a second boundary value of the first type element and the second type element in the second target channel;
[0173] Determining a first quantity ratio of the first type of elements to the second type of elements in the first target channel according to the first limiting value;
[0174] Determining a second quantity ratio of the first type of elements to the second type of elements in the second target channel according to the second limiting value;
[0175] Compare and analyze the first quantity ratio and the second quantity ratio to obtain a comparison result;
[0176] According to the comparison result, deleting a target channel corresponding to a feature map having a smaller quantity ratio between the first quantity ratio and the second quantity ratio and a connection associated with the target channel;
[0177] After traversing all channels of all network layers in the neural network model after weight pruning, a locally pruned neural network model is obtained.
[0178] According to the target task processing device provided by the present invention, the global clipping processing unit is specifically used for:
[0179] All channels of each network layer in the locally pruned neural network model are traversed to determine invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain a globally pruned neural network model.
[0180] According to the target task processing device provided by the present invention, all channels of each network layer in the locally pruned neural network model are traversed to determine the invalid associated channels of each network layer in the locally pruned neural network model, and the invalid associated channels of each network layer in the locally pruned neural network model are pruned respectively to obtain the globally pruned neural network model, specifically including:
[0181] Inputting the data in the second verification sample set into the locally pruned neural network model to obtain a first loss value corresponding to the locally pruned neural network model;
[0182] Determine a second loss value corresponding to the locally pruned neural network model in a case where a first channel corresponding to a first network layer of the locally pruned neural network model is pre-deleted;
[0183] determining a first difference between the first loss value and the second loss value;
[0184] In the case where the first difference is less than or equal to a preset loss threshold, determining the first channel as an invalid associated channel, and deleting the first channel and the corresponding connection;
[0185] In the case where the first difference is greater than a preset loss threshold, determining that the first channel is a non-invalid associated channel, sequentially traversing the remaining channels corresponding to the first network layer of the neural network model after local pruning, and comparing and analyzing the corresponding loss values to obtain an intermediate neural network model;
[0186] All channels corresponding to the remaining network layers in the intermediate neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the neural network model after global pruning.
[0187] The target task processing device provided by the present invention further includes: when the first difference is greater than a preset loss threshold, pre-deleting all channels corresponding to the first network layer of the neural network model after local pruning,
[0188] Determine a third loss value corresponding to the locally pruned neural network model;
[0189] determining a second difference between the first loss value and the third loss value;
[0190] When the second difference is less than or equal to a preset loss threshold, all channels corresponding to the first network layer are determined as invalid associated channels, and all channels and corresponding connections corresponding to the first network layer are deleted;
[0191] When the second difference is greater than a preset loss threshold, it is determined that there are no invalid associated channels in the first network layer, and the remaining network layers of the locally pruned neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the globally pruned neural network model.
[0192] The target task processing device provided by the present invention determines the importance value of the channel of each network layer in the neural network model to be pruned by the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned, prunes the channel of each network layer in the neural network model to be pruned based on the value of the channel importance to obtain the neural network model after weight pruning, locally prunes the channel of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model, and globally prunes the channel of each network layer in the locally pruned neural network model to obtain the globally pruned neural network model, which can effectively improve the pruning efficiency and accuracy of the neural network model.
[0193] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device (ie, electronic device) may include: a processor (processor) 701, a communication interface (Communications Interface) 704, a memory (memory) 702 and a communication bus 703, wherein the processor 701, the communication interface 704, and the memory 702 communicate with each other through the communication bus 703. The processor 701 can call the logic instructions in the memory 702 to execute the target task processing method, which includes: determining the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned; based on the importance value of the channel, pruning the channel of each network layer in the neural network model to be pruned to obtain the neural network model after weight pruning; locally pruning the channels of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model; globally pruning the channels of each network layer in the locally pruned neural network model to obtain the globally pruned neural network model; inputting the target task to be processed into the globally pruned neural network model, processing the target task to be processed based on the globally pruned neural network model, and obtaining the processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0194] In addition, the logic instructions in the above-mentioned memory 702 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.
[0195] On the other hand, the present application also provides a computer program product, which includes a computer program, which can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target task processing method provided by the above methods, the method including: determining the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned; based on the importance value of the channel, pruning the channel of each network layer in the neural network model to be pruned to obtain a weight-pruned neural network model; locally pruning the channel of each network layer in the neural network model after weight pruning to obtain a locally pruned neural network model; globally pruning the channel of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model; inputting the target task to be processed into the globally pruned neural network model, processing the target task to be processed based on the globally pruned neural network model, and obtaining the processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
[0196] On the other hand, the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored program, wherein the program executes the target task processing method provided by the above methods when it is run, the method comprising: determining the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned; based on the importance value of the channel, pruning the channel of each network layer in the neural network model to be pruned to obtain the neural network model after weight pruning; locally pruning the channel of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model; globally pruning the channel of each network layer in the locally pruned neural network model to obtain the globally pruned neural network model; inputting the target task to be processed into the globally pruned neural network model, processing the target task to be processed based on the globally pruned neural network model, and obtaining the processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, autonomous driving task.
[0197] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0198] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A target task processing method, characterized in that: include: Determine the importance of the channels of each network layer in the neural network model to be pruned based on the weight values and characteristic values corresponding to the channels of each network layer in the neural network model to be pruned; Based on the importance value of the channel, pruning the channels of each network layer in the neural network model to be pruned to obtain a neural network model after weight pruning; The channels of each network layer in the neural network model after weight pruning are locally pruned to obtain the locally pruned neural network model; the locally pruned channels of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model specifically includes: traversing all channels of each network layer in the neural network model after weight pruning, determining similar channels of each network layer in the neural network model after weight pruning, and pruning similar channels of each network layer in the neural network model after weight pruning to obtain the locally pruned neural network model; Performing global pruning on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model; performing global pruning on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model specifically includes: traversing all channels of each network layer in the locally pruned neural network model, determining invalid associated channels of each network layer in the locally pruned neural network model, and pruning the invalid associated channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model; The target task to be processed is input into the globally pruned neural network model, and the target task to be processed is processed based on the globally pruned neural network model to obtain a processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
2. The target task processing method according to claim 1, characterized in that: The determining the importance of the channels of each network layer in the neural network model to be pruned based on the weight values and characteristic values corresponding to the channels of each network layer in the neural network model to be pruned specifically includes: Determine the value of the first importance of the channel of each network layer in the neural network model to be pruned based on the weight value corresponding to the channel of each network layer in the neural network model to be pruned; Determine the value of the second importance of the channel of each network layer in the neural network model to be pruned based on the characteristic value corresponding to the channel of each network layer in the neural network model to be pruned; Based on the value of the first importance of the channel of each network layer in the neural network model to be pruned and the value of the second importance of the channel of each network layer in the neural network model to be pruned, the value of the importance of the channel of each network layer in the neural network model to be pruned is determined.
3. The target task processing method according to claim 2, characterized in that: The determining, based on the weight values corresponding to the channels of each network layer in the neural network model to be pruned, the values of the first importance of the channels of each network layer in the neural network model to be pruned specifically includes: The weight values corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset weight importance analysis model to obtain the first importance values of the channels of the various network layers in the neural network model to be pruned output by the weight importance analysis model.
4. The target task processing method according to claim 2, characterized in that: The determining, based on the characteristic values corresponding to the channels of each network layer in the neural network model to be pruned, the values of the second importance of the channels of each network layer in the neural network model to be pruned specifically includes: The eigenvalues corresponding to the channels of the various network layers in the neural network model to be pruned are input into a preset eigenvalue importance analysis model to obtain the second importance values of the channels of the various network layers in the neural network model to be pruned output by the eigenvalue importance analysis model.
5. The target task processing method according to claim 1, characterized in that: The traversing of all channels of each network layer in the weight pruned neural network model, determining similar channels of each network layer in the weight pruned neural network model, and pruning similar channels of each network layer in the weight pruned neural network model to obtain a locally pruned neural network model specifically includes: Inputting the data in the first verification sample set into the weight pruned neural network model, obtaining the characteristic values of all channels corresponding to each network layer in the weight pruned neural network model, and sequentially determining the similarity values between the characteristic values of any two target channels in all the channels; In a case where the similarity value between the feature values of the two target channels is greater than a preset similarity threshold, determining that the two target channels are non-similar channels, and retaining the two target channels; When the similarity value between the feature values of the two target channels is less than or equal to a preset similarity threshold, the two target channels are determined to be similar channels, and one of the two target channels is pruned to obtain a locally pruned neural network model.
6. The target task processing method according to claim 5, characterized in that: The pruning of one of the two target channels to obtain a locally pruned neural network model specifically includes: Normalizing the elements in the feature maps corresponding to the feature values of the two target channels respectively so that the elements in the feature maps corresponding to the feature values of the two target channels are at the same magnitude; wherein the two target channels include a first target channel and a second target channel; Determine a first boundary value of a first type element and a second type element in the first target channel; Determine a second boundary value of the first type element and the second type element in the second target channel; Determining a first quantity ratio of the first type of elements to the second type of elements in the first target channel according to the first limiting value; Determining a second quantity ratio of the first type of elements to the second type of elements in the second target channel according to the second limiting value; Compare and analyze the first quantity ratio and the second quantity ratio to obtain a comparison result; According to the comparison result, deleting a target channel corresponding to a feature map having a smaller quantity ratio between the first quantity ratio and the second quantity ratio and a connection associated with the target channel; After traversing all channels of all network layers in the neural network model after weight pruning, a locally pruned neural network model is obtained.
7. The target task processing method according to claim 1, characterized in that: The pruning of the channels of each network layer in the neural network model to be pruned based on the importance value of the channel to obtain the neural network model after weight pruning specifically includes: The importance values of the channels of each network layer in the neural network model are compared and analyzed with the preset importance threshold. When there are values of the importance of the channels of each network layer in the neural network model that are less than the preset importance threshold, the destination channels whose importance values of the channels of each network layer in the neural network model are less than the preset importance threshold are deleted, and the connections associated with the destination channels in each network layer in the neural network model are deleted to obtain the neural network model after weight pruning.
8. The target task processing method according to claim 1, characterized in that: The traversing of all channels of each network layer in the locally pruned neural network model, determining invalid associated channels of each network layer in the locally pruned neural network model, and pruning the invalid associated channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model specifically includes: Inputting the data in the second verification sample set into the locally pruned neural network model to obtain a first loss value corresponding to the locally pruned neural network model; Determine a second loss value corresponding to the locally pruned neural network model in a case where a first channel corresponding to a first network layer of the locally pruned neural network model is pre-deleted; determining a first difference between the first loss value and the second loss value; In the case where the first difference is less than or equal to a preset loss threshold, determining the first channel as an invalid associated channel, and deleting the first channel and the corresponding connection; In the case where the first difference is greater than a preset loss threshold, determining that the first channel is a non-invalid associated channel, sequentially traversing the remaining channels corresponding to the first network layer of the neural network model after local pruning, and comparing and analyzing the corresponding loss values to obtain an intermediate neural network model; All channels corresponding to the remaining network layers in the intermediate neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the neural network model after global pruning.
9. The target task processing method according to claim 8, characterized in that: Also includes: When the first difference is greater than a preset loss threshold, all channels corresponding to the first network layer of the locally pruned neural network model are pre-deleted, Determine a third loss value corresponding to the locally pruned neural network model; determining a second difference between the first loss value and the third loss value; When the second difference is less than or equal to a preset loss threshold, all channels corresponding to the first network layer are determined as invalid associated channels, and all channels and corresponding connections corresponding to the first network layer are deleted; When the second difference is greater than a preset loss threshold, it is determined that there are no invalid associated channels in the first network layer, and the remaining network layers of the locally pruned neural network model are traversed in turn, and comparison and analysis are performed respectively according to the loss values to obtain the globally pruned neural network model.
10. A target task processing device, characterized in that: include: A channel importance analysis unit, used to determine the importance value of the channel of each network layer in the neural network model to be pruned based on the weight value and characteristic value corresponding to the channel of each network layer in the neural network model to be pruned; An unstructured pruning processing unit, used for pruning the channels of each network layer in the neural network model to be pruned based on the importance value of the channel, to obtain a neural network model after weight pruning; A local pruning processing unit is used to perform local pruning processing on the channels of each network layer in the neural network model after weight pruning to obtain the neural network model after local pruning; the local pruning processing unit is specifically used to traverse all channels of each network layer in the neural network model after weight pruning, determine similar channels of each network layer in the neural network model after weight pruning, and perform pruning processing on similar channels of each network layer in the neural network model after weight pruning to obtain the neural network model after local pruning; A global pruning processing unit is used to perform global pruning processing on the channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model; the global pruning processing unit is specifically used to traverse all channels of each network layer in the locally pruned neural network model, determine invalid associated channels of each network layer in the locally pruned neural network model, and prune the invalid associated channels of each network layer in the locally pruned neural network model to obtain a globally pruned neural network model; A target task processing unit is used to input the target task to be processed into the globally pruned neural network model, process the target task to be processed based on the globally pruned neural network model, and obtain a processing result of the corresponding target task; wherein the target task includes any one of the following: image recognition task, natural language processing task, speech recognition task, and autonomous driving task.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the target task processing method according to any one of claims 1 to 9 is implemented.
12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target task processing method according to any one of claims 1 to 9 is implemented.
13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the target task processing method according to any one of claims 1 to 9 is implemented.
