Model compression method and apparatus, computer-readable medium, and electronic device

Abstract

The present disclosure provides a model compression method and device, a computer readable medium and an electronic device, and relates to the technical field of artificial intelligence. The method comprises: obtaining a pre-trained initial neural network model; performing depth pruning processing on the initial neural network model to obtain a depth pruned initial neural network model; performing width pruning processing on the depth pruned initial neural network model to obtain a width pruned initial neural network model; and determining the width pruned initial neural network model that meets a preset model compression parameter as a target neural network model. The present disclosure can effectively compress the redundant structure in the initial neural network model, can effectively reduce the model volume of the target neural network model while ensuring the performance and accuracy of the target neural network model, and thus improves the applicable scenarios of the target neural network model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to a model compression method, a model compression apparatus, a computer-readable medium, and an electronic device. Background Technology

[0002] With the continuous improvement of science and technology, deep learning (DL) technology has developed rapidly. The goal of deep learning is to learn the inherent patterns and representational layers of sample data. The implementation of deep learning generally relies on various types of neural network structures. However, as the complexity of tasks increases and the amount of data grows, the neural network structures in deep learning models become increasingly complex, often containing a lot of redundant networks. How to optimize the redundant networks in deep learning models is a huge challenge we face today.

[0003] Currently, neural network optimization mainly adopts the channel pruning scheme. However, this method has limited compression capabilities for neural network models, and the resulting neural network models still contain redundant networks. At the same time, it cannot effectively guarantee the performance and accuracy of the pruned neural network. Summary of the Invention

[0004] The purpose of this disclosure is to provide a model compression method, a model compression device, a computer-readable medium, and an electronic device, thereby effectively reducing the model size of the target neural network model while ensuring its performance and accuracy.

[0005] According to a first aspect of this disclosure, a model compression method is provided, comprising:

[0006] Obtain a pre-trained initial neural network model;

[0007] The initial neural network model is subjected to deep pruning to obtain a deeply pruned initial neural network model;

[0008] The initial neural network model after depth pruning is subjected to width pruning to obtain the initial neural network model after width pruning.

[0009] The initial neural network model after width pruning that meets the preset model compression parameters is determined as the target neural network model.

[0010] According to a second aspect of this disclosure, a model compression apparatus is provided, comprising:

[0011] The model acquisition module is used to acquire the pre-trained initial neural network model;

[0012] The deep pruning module is used to perform deep pruning on the initial neural network model to obtain a deep-pruned initial neural network model.

[0013] The width pruning module is used to perform width pruning on the initial neural network model after depth pruning to obtain the initial neural network model after width pruning.

[0014] The model output module is used to determine the initial neural network model after width pruning, which meets the preset model compression parameters, as the target neural network model.

[0015] According to a third aspect of this disclosure, a computer-readable medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method described above.

[0016] According to a fourth aspect of this disclosure, an electronic device is provided, characterized in that it comprises:

[0017] Processor; and

[0018] Memory is used to store one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the methods described above.

[0019] One embodiment of this disclosure provides a model compression method that can obtain a pre-trained initial neural network model, perform deep pruning on the initial neural network model to obtain a deeply pruned initial neural network model, then perform width pruning on the deeply pruned initial neural network model to obtain a width-pruned initial neural network model, and finally determine the width-pruned initial neural network model that meets preset model compression parameters as the target neural network model. On the one hand, by first performing deep pruning on the initial neural network model and then performing width pruning on the initial neural network model, the compression of the neural network model from coarse-grained to fine-grained can be achieved, improving the compression strength of the initial neural network model and effectively reducing the model size of the target neural network model. On the other hand, the stepwise compression from coarse-grained to fine-grained allows for better control of the performance and accuracy of the neural network model during the compression process, and the pruned neural network model can be further verified by using preset model compression parameters, further improving the performance and accuracy of the output target neural network model.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0022] Figure 1 A schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied is shown;

[0023] Figure 2 The illustration schematically shows a flowchart of a model compression method according to an exemplary embodiment of the present disclosure;

[0024] Figure 3 This schematic diagram illustrates a process for implementing deep pruning in an exemplary embodiment of the present disclosure.

[0025] Figure 4 This illustration schematically depicts a process for determining the optimal subnetwork through neural network structure search in an exemplary embodiment of the present disclosure.

[0026] Figure 5 This schematic diagram illustrates a process for implementing width pruning in an exemplary embodiment of the present disclosure.

[0027] Figure 6 This illustration schematically shows a process diagram for weight channel pruning of the optimal subnetwork obtained by search in an exemplary embodiment of the present disclosure;

[0028] Figure 7 This schematic diagram illustrates the composition of a model compression apparatus in an exemplary embodiment of the present disclosure.

[0029] Figure 8 A schematic diagram of an electronic device to which embodiments of the present disclosure may be applied is shown. Detailed Implementation

[0030] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0031] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] Figure 1 A schematic diagram of a system architecture for an exemplary application environment in which an embodiment of the model compression method and apparatus of this disclosure can be applied is shown.

[0033] like Figure 1 As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables. Terminal devices 101, 102, and 103 may be various electronic devices with artificial intelligence computing capabilities, including but not limited to desktop computers, laptops, smartphones, and tablets. It should be understood that... Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0034] The model compression method provided in this embodiment is generally executed in server 105, and correspondingly, the model compression device is generally located in server 105. However, it is readily understood by those skilled in the art that the model compression method provided in this embodiment can also be executed in terminal devices 101, 102, and 103, and correspondingly, the model compression device can also be located in terminal devices 101, 102, and 103. This exemplary embodiment does not impose any special limitations on this.

[0035] One technical solution provides a model pruning method combining layer pruning and channel pruning, comprising the following steps: Step 1, sparsification training; Step 2, layer-wise adaptive threshold lookup; Step 3, determining the sparsity rate of each layer based on the threshold; Step 4, calculating layer importance ranking; Step 5, layer pruning; Step 6, channel pruning; and Step 7, fine-tuning. This solution primarily employs pruning to optimize the model at the convolutional layer and weight / channel dimensions, using statistical algorithms to calculate the importance of convolutional layers and channels, and then pruning based on this importance. However, in practice, due to the large granularity of convolutional layers, statistically calculated importance is difficult to accurately represent layer importance, often leading to incorrect pruning, affecting model accuracy, and making it difficult to truly measure layer importance.

[0036] Another technical solution provides a neural network model compression method based on structure search and channel pruning. For the dataset and task, a lightweight network with decent accuracy but lower parameter and computational costs compared to conventional convolutional neural networks is first trained. Each layer of the network is assigned a scaling factor to measure its importance. Simultaneously, the scaling factors for each layer are initialized using a large-variance Gaussian distribution, L1 norm regularization is applied, and a subgradient optimization algorithm is used for sparsity training. Layers with scaling factors close to 0 are pruned, and the model is fine-tuned. If the accuracy returns to the baseline or the decrease is within 5%, the next round of model compression and pruning is performed; otherwise, the process ends. This solution mainly uses network search to build the initial model and then prunes it based on the searched model. This method generates new network structures during the model search process, requiring significant computational and time costs. It cannot optimize pre-designed artificial networks, limiting its application scenarios. Furthermore, the searched model often fails to achieve the accuracy of manually designed models on many tasks.

[0037] Neural Architecture Search (NAS) is a process that automates the process of manually tuning neural networks to discover more complex architectures. NAS uses search strategies to test and evaluate a large number of network architectures in the search space and selects the architecture that best meets the given problem objective by maximizing the fitness function.

[0038] Currently, related technologies mainly employ pruning or NAS (Neural Allocation) schemes for model compression and optimization. For well-trained, high-quality models, the only current solution is pruning, but the accuracy of the output model cannot be guaranteed. Furthermore, models searched using NAS often fail to perform well in many tasks, sometimes even showing lower accuracy than manually designed models. NAS has limited practical applications, and related technologies struggle to simultaneously compress manually designed models while maintaining accuracy.

[0039] Based on one or more problems in the related technologies, this disclosure first provides a model compression method. The following describes the model compression method and model compression apparatus of the exemplary embodiments of this disclosure in detail, taking the execution of the method by a server as an example.

[0040] Figure 2 The present exemplary embodiment shows a schematic flowchart of a model compression method, which may include the following steps S210 to S240:

[0041] In step S210, a pre-trained initial neural network model is obtained.

[0042] In one exemplary embodiment, the initial neural network model refers to a convolutional neural network (CNN) model built and trained for a deep learning task. For example, the initial neural network model can be a CNN for object detection or a CNN for speech recognition; this example embodiment does not impose any special limitations on this. Generally, initial neural network models designed and trained manually have computational redundancy when processing specific deep learning tasks, often involving massive computational loads. However, due to limitations in the computing power of terminal devices or servers, computationally intensive initial neural network models are difficult to implement.

[0043] Convolutional Neural Networks (CNNs) are deep feedforward neural networks that incorporate convolutional computations. CNNs possess representation learning capabilities, enabling shift-invariant classification of input information based on their hierarchical structure. CNNs can be constructed by mimicking biological visual perception mechanisms and can perform both supervised and unsupervised learning. The shared parameters of convolutional kernels within their hidden layers and the sparsity of inter-layer connections allow CNNs to learn gridded features (such as pixels and audio) with relatively low computational cost.

[0044] In step S220, the initial neural network model is subjected to deep pruning to obtain a deep-pruned initial neural network model.

[0045] In an exemplary embodiment, depth pruning refers to the process of pruning the depth layers of an initial neural network model. The depth layers of a neural network model generally refer to the number of convolutional layers in the vertical direction.

[0046] Deep pruning can be a process of pruning convolutional layers in an initial neural network model. For example, the usefulness of a convolutional layer can be determined based on its computational cost in the initial neural network model, retaining useful convolutional layers and deleting those that do not participate in computation, thus pruning the convolutional layers in the initial neural network model. Of course, deep pruning can also be a process of performing network search on the convolutional layers in the initial neural network model based on Neural Architecture Search (NAS) technology to select the optimal subnetwork. This example embodiment does not impose any special limitations on the method of implementing deep pruning.

[0047] Compared to the initial neural network model before deep pruning, the initial neural network model after deep pruning effectively reduces the number of convolutional layers, thereby reducing the redundancy of the initial neural network model at a coarse-grained level.

[0048] In step S230, the initial neural network model after depth pruning is subjected to width pruning to obtain the initial neural network model after width pruning.

[0049] In an exemplary embodiment, width pruning refers to the process of pruning the width of the initial neural network model. The width of the neural network model generally refers to the number of channels in each convolutional layer in the horizontal direction.

[0050] Width pruning can be a process of pruning convolutional layers in an initial neural network model. For example, convolutional channels can be pruned based on their weights, retaining more important channels and deleting less important ones, thus pruning the convolutional channels of each convolutional layer in the initial neural network model. Alternatively, a pre-defined search agent (a type of reinforcement learning network) can determine the pruning rate of each convolutional layer and delete convolutional channels based on that rate. This example embodiment does not impose any special limitations on the method of implementing width pruning.

[0051] Compared to the initial neural network model before width pruning, the initial neural network model after width pruning effectively reduces the number of convolutional channels in the convolutional layers, thereby reducing redundancy and computational cost at a finer granular level.

[0052] In step S240, the initial neural network model after width pruning that meets the preset model compression parameters is determined as the target neural network model.

[0053] In an exemplary embodiment, the preset model compression parameter refers to the conditional parameter used to verify whether the initial neural network model after width pruning meets the performance and accuracy requirements. For example, the preset model compression parameter can be a model execution efficiency threshold, an output result accuracy threshold, or a model compression rate threshold, etc. The preset model compression parameter can be customized according to the actual situation. This example embodiment does not make any special limitations on the specific form of the preset model compression parameter.

[0054] A target neural network model refers to a neural network model obtained by compressing an initial neural network model. Compared to the initial neural network model, the target neural network model has a simpler network structure, less redundancy, and its performance and accuracy remain within a certain range of the initial neural network model. In other words, the target neural network model can achieve the same task as the pre-trained initial neural network model, while having less redundancy and requiring less computational power, effectively expanding the applicable scenarios of the target neural network model.

[0055] By first performing depth pruning on the initial neural network model and then performing width pruning, the neural network model can be compressed from coarse-grained to fine-grained, increasing the compression strength of the initial neural network model and effectively reducing the size of the target neural network model. At the same time, the stepwise compression from coarse-grained to fine-grained allows for better control of the performance and accuracy of the neural network model during the compression process. Furthermore, the pruned neural network model can be further validated by using preset model compression parameters, further improving the performance and accuracy of the output target neural network model.

[0056] The technical content of steps S210 to S240 will be described in detail below.

[0057] In one exemplary embodiment, the preset model compression parameters may include a model execution efficiency threshold and an output result accuracy threshold. The specific parameters of the model execution efficiency threshold and the output result accuracy threshold can be customized according to the actual situation. This example embodiment does not impose any special limitations on this.

[0058] Optionally, the following steps can be used to select an initial neural network model that meets the preset model compression parameters after width pruning and determine it as the target neural network model: The initial neural network model after width pruning can be verified to determine the model execution efficiency and output accuracy of the initial neural network model after width pruning; if the model execution efficiency is greater than or equal to the model execution efficiency threshold and the output accuracy is greater than or equal to the output accuracy threshold, then the initial neural network model after width pruning is determined as the target neural network model.

[0059] The verification operation refers to the operation of verifying the verification data obtained in the initial neural network model after width pruning. A pre-set verification dataset can be input into the initial neural network model after width pruning to determine the model execution efficiency and output accuracy of the initial neural network model after width pruning.

[0060] If the model execution efficiency is greater than or equal to the model execution efficiency threshold and the output accuracy is greater than or equal to the output accuracy threshold, then the initial neural network model after this round of width pruning is determined to meet the requirements and is used as the target neural network model for the final output.

[0061] If the model execution efficiency is less than the model execution efficiency threshold, or the output accuracy is less than the output accuracy threshold, or both the model execution efficiency and output accuracy are less than the output accuracy threshold, the initial neural network model after width pruning in this round is considered unacceptable. Therefore, model compression, i.e., depth pruning and width pruning, can be performed on the initial neural network model after width pruning in this round until the model execution efficiency of the initial neural network model after width pruning is greater than or equal to the model execution efficiency threshold, and the output accuracy is greater than or equal to the output accuracy threshold. The initial neural network model after width pruning is then used as the final output target neural network model.

[0062] By setting preset model compression parameters, it is possible to monitor the performance and accuracy of the neural network model during the model compression process, effectively ensuring the performance (model execution efficiency) and accuracy (output result accuracy) of the final output target neural network model.

[0063] In one exemplary embodiment, it can be achieved through Figure 3 The steps described above implement deep pruning of the initial neural network model:

[0064] Step S310: Based on the availability of convolutional layers in the initial neural network model, construct a neural network model search space, the neural network model search space including 2 NN subnetworks, where N is the number of convolutional layers;

[0065] Step S320: Randomly sample and train the sub-network in the search space of the neural network model to obtain a trained sub-network model;

[0066] Step S330: Based on the evolutionary search method and the preset model compression parameters, search for the subnetworks in the subnetwork model, determine the target subnetwork, and use the target subnetwork as the initial neural network model after deep pruning.

[0067] The neural network model search space refers to a predefined space based on the convolutional layers in the initial neural network model, used to search for and determine the optimal sub-network structure for completing deep learning tasks. The neural network model search space can include 2... N There are N subnetworks, where N can represent the number of convolutional layers in the initial neural network model.

[0068] The search space of a neural network model can be constructed based on the availability of convolutional layers in the initial neural network model. For example, the computational cost of the convolutional layer (i.e., the number of multiplications and summs) can be determined, and the availability of the convolutional layer can be determined based on the computational cost (i.e., the number of multiplications and summs). Of course, the availability of the convolutional layer can also be determined based on the convolutional channels and the weights of the convolutional channels. This example embodiment is not limited to this.

[0069] Optionally, the computational cost (MultiplyAccumulate, MAC, i.e., the number of times multiply and accumulate) of each sub-network in the search space of the neural network model can be determined, and random sampling and network training can be performed on sub-networks with computational cost greater than or equal to a preset computational cost threshold to obtain a trained sub-network model.

[0070] Evolutionary search algorithms are heuristic search algorithms whose main components are mutation, recombination, and selection (including parent selection and elimination of samples) algorithm design modules. For different deep learning tasks, completing the mutation, recombination, and selection modules allows the framework of the evolutionary search algorithm to be built. Based on the evolutionary search method and preset model compression parameters, subnetworks in the subnetwork model can be searched to determine the target subnetwork. The target subnetwork is a subnetwork that can achieve the deep learning task and meet the accuracy and compression ratio requirements. The target subnetwork is then used as the initial neural network model after deep pruning.

[0071] By using Neural Architecture Search (NAS) technology to prune the convolutional layers of the initial neural network model, the network complexity can be reduced while maintaining the accuracy of the initial neural network model. This results in a better network structure for the pre-trained initial neural network model, thereby improving network performance.

[0072] Figure 4 The illustration schematically depicts a process for determining the optimal subnetwork through neural network structure search in an exemplary embodiment of the present disclosure.

[0073] refer to Figure 4 As shown, in step S410, input the pre-trained initial neural network model.

[0074] Step S420: Construct the neural network model search space based on the availability of convolutional layers. The neural network model search space can be constructed based on the availability of each convolutional layer in the initial neural network model. The neural network model search space can include 2... N There are 1 sub-networks, where N can represent the number of convolutional layers in the initial neural network model;

[0075] Step S430: Train the sub-networks in the search space of the neural network model. Random sampling and training can be performed on sub-networks in the search space whose computational complexity falls within a certain range to obtain trained sub-network models.

[0076] Step S440: Perform a network search based on accuracy and compression ratio metrics. Using accuracy and compression ratio metrics as a benchmark, an evolutionary search algorithm can be employed to search for sub-networks in the search space of the neural network model, thereby pruning the convolutional layers of the initial neural network model.

[0077] Step S450 yields the optimal subnetwork that satisfies both accuracy and compression ratio criteria. This optimal subnetwork can be used as the initial neural network model after deep pruning.

[0078] Given an existing manually designed neural network model, re-searching for a new neural network model using Neural Architecture Search (NAS) is time-consuming and uncertain in terms of achieving the accuracy of the manually designed model. It may also introduce new network structures, impacting model design and training efficiency. Furthermore, manually designed models often exhibit significant redundancy, high power consumption, and long runtime, limiting their application on low-performance computing platforms such as mobile devices. This example implementation, however, employs NAS search to prune and optimize the convolutional layers of an existing trained neural network model, improving performance while maintaining accuracy.

[0079] In one exemplary embodiment, it can be achieved through Figure 5 The steps described in the document implement width pruning of the initial neural network model after depth pruning, refer to... Figure 5 As shown, it can specifically include:

[0080] Step S510: Calculate the performance parameters of the convolution kernels of each sub-network in the initial neural network model after deep pruning;

[0081] Step S520: Prune the convolutional channels of each convolutional layer in the initial neural network model after depth pruning according to the performance parameters to obtain the initial neural network model after width pruning.

[0082] Here, performance parameters refer to parameters that measure the performance of the convolutional kernels in each sub-network. For example, performance parameters can be the computational cost of the convolution kernel, i.e., the cumulative accumulation count (MAC), or the parameter count of the weights. For instance, the parameter count of the weights can be represented as i × O × K × K, where i can represent the input channel, O can represent the output channel, and K can represent the size of the convolutional kernel. Of course, other parameters can also be used to represent the performance parameters of the convolutional kernels in each sub-network, and this example embodiment does not impose any special limitations on this.

[0083] Optionally, the performance parameters of the convolutional kernels of each subnetwork in the initial deep-pruned neural network model can be calculated as follows: The convolutional computation cost and the number of weight parameters of the convolutional kernels in each subnetwork of the initial deep-pruned neural network model can be calculated; the performance parameters of the convolutional kernels are determined based on the ratio of the convolutional computation cost to the number of weight parameters. A larger ratio indicates that the convolutional kernel has a high computational cost and few parameters, indicating good performance, and the corresponding convolutional channels can be retained. Conversely, if the ratio is less than a certain threshold, the convolutional kernel has poor performance, and the corresponding convolutional channels can be pruned.

[0084] Optionally, the absolute values ​​of the weights of each convolutional layer can be calculated according to the convolutional channels, and sorted according to the absolute values. Then, the convolutional channels of each convolutional layer in the initial neural network model after depth pruning can be pruned according to the performance parameters to obtain the initial neural network model after width pruning.

[0085] By further pruning the convolutional channels of the initial neural network model after pruning using Neural Architecture Search (NAS) technology, redundant data in the channel dimensions of the initial neural network model is removed, thereby maximizing the optimization of the initial neural network model while ensuring its performance and accuracy.

[0086] Optionally, when pruning the convolutional channels of the initial neural network model, a neural network model search space can be constructed, and then the optimal convolutional channel can be searched and determined through the neural network model search space to achieve the pruning of the convolutional channels.

[0087] In an exemplary embodiment, after obtaining an initial neural network model with width pruning that satisfies preset model compression parameters, a training dataset can be acquired, and the initial neural network model with width pruning can be fine-tuned and trained using the training dataset to further improve the accuracy of the initial neural network model with width pruning, thereby obtaining the final output target neural network model.

[0088] Figure 6 The illustration shows a flowchart of weight channel pruning for the optimal subnetwork obtained by search in an exemplary embodiment of the present disclosure.

[0089] refer to Figure 6 As shown, in step S610, the optimal subnetwork found is input, which is the initial neural network model after deep pruning.

[0090] Step S620: Calculate the ratio of convolution computation to weight parameter quantity for each convolution. After obtaining the optimal subnetwork after NAS convolutional layer pruning, the MAC computation of each convolution and the number of weight parameters i×O×K×K can be calculated, where i can represent the input channel, O can represent the output channel, and K can represent the size of the convolution kernel. The ratio of convolution computation to weight parameter quantity for each convolution can be expressed as MAC / Weight.

[0091] Step S630: Calculate and sort the absolute values ​​of the weights for each convolution. Calculate the absolute value of the weights for each convolutional layer by channel and sort them according to the absolute values;

[0092] Step S640: Select pruning channels based on the ratio of convolution computation to weight parameters for each convolution. Channels can be filtered based on the MAC / Weight ratio; those smaller than the ratio are pruned, while a higher MAC / Weight ratio indicates higher computation and fewer parameters, and these should be retained.

[0093] Step S650, performance and accuracy verification. The performance and accuracy of the initial neural network model with pruned channels can be verified. If the performance and accuracy requirements are met, the target neural network model is determined. If the performance and accuracy requirements are not met, return to step S610.

[0094] Step S660, fine-tuning training. A training dataset can be obtained, and the final compressed target neural network model can be fine-tuned using the training dataset to further improve the accuracy of the target neural network model.

[0095] In summary, this exemplary embodiment allows for the acquisition of a pre-trained initial neural network model. Deep pruning is then performed on this initial neural network model to obtain a deeply pruned initial neural network model. Furthermore, width pruning is performed on the deeply pruned initial neural network model to obtain a width-pruned initial neural network model. Finally, the width-pruned initial neural network model that meets preset model compression parameters is determined as the target neural network model. On one hand, by first performing deep pruning on the initial neural network model and then width pruning, compression of the neural network model from coarse-grained to fine-grained can be achieved, increasing the compression strength of the initial neural network model and effectively reducing the model size of the target neural network model. On the other hand, the gradual compression from coarse-grained to fine-grained allows for better control of the performance and accuracy of the neural network model during the compression process. Furthermore, the preset model compression parameters are used to further verify the pruned neural network model, further improving the performance and accuracy of the output target neural network model.

[0096] In this exemplary embodiment, coarse-to-fine model compression of the network model is achieved by combining NAS convolutional layer pruning and weight channel pruning. This solves the problem that models trained through the pre-training process have high power consumption and long processing time in practical applications, enabling pre-trained neural network models to be applied to hardware platforms with different performance levels, such as for the compression of semantic segmentation models in portrait blurring projects.

[0097] Currently, portrait blurring projects are mainly applied to high-performance mobile phones. However, due to hardware limitations, they cannot be widely applied to low-end mobile platforms. Furthermore, the semantic segmentation algorithms in blurring technology have large models and high accuracy requirements, limiting the project's widespread application. In this example implementation, pruning convolutional layers using NAS search reduces network complexity while maintaining model accuracy, resulting in a better network structure and improved network performance. After NAS layer pruning, weight channel pruning is further applied to remove redundancy in the channel dimensions, maximizing model optimization. For users, the optimized model using the model compression method summarized in this example implementation makes mobile phone functions smoother, especially for many deep learning-based camera functions, which run more smoothly and quickly. It also allows mid-to-low-end models to utilize a variety of practical functions, effectively avoiding lag and waiting time during mobile phone use and improving the user experience.

[0098] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0099] Further reference Figure 7 As shown, this example embodiment also provides a model compression device 700, including a model acquisition module 710, a depth pruning module 720, a width pruning module 730, and a model output module 740. Wherein:

[0100] The model acquisition module 710 is used to acquire the pre-trained initial neural network model;

[0101] The deep pruning module 720 is used to perform deep pruning on the initial neural network model to obtain a deep-pruned initial neural network model.

[0102] The width pruning module 730 is used to perform width pruning on the initial neural network model after depth pruning to obtain the initial neural network model after width pruning.

[0103] The model output module 740 is used to determine the initial neural network model after width pruning, which meets the preset model compression parameters, as the target neural network model.

[0104] In an exemplary embodiment, the preset model compression parameters may include a model execution efficiency threshold and an output result accuracy threshold; the model output module 740 may be used for:

[0105] The initial neural network model after width pruning is validated to determine the model execution efficiency and output accuracy of the initial neural network model after width pruning.

[0106] If the model execution efficiency is greater than or equal to the model execution efficiency threshold, and the output accuracy is greater than or equal to the output accuracy threshold, then the initial neural network model after width pruning is determined as the target neural network model.

[0107] In one exemplary embodiment, the deep pruning module 720 can be used to:

[0108] Based on the availability of convolutional layers in the initial neural network model, a neural network model search space is constructed, which includes 2... N N subnetworks, where N is the number of convolutional layers;

[0109] Random sampling and network training are performed on the sub-networks in the search space of the neural network model to obtain trained sub-network models;

[0110] Based on the evolutionary search method and the preset model compression parameters, the subnetworks in the subnetwork model are searched to determine the target subnetwork, and the target subnetwork is used as the initial neural network model after deep pruning.

[0111] In one exemplary embodiment, the deep pruning module 720 can be used to:

[0112] Determine the computational cost of each subnetwork in the search space of the neural network model;

[0113] Random sampling and network training are performed on subnetworks whose computational cost is greater than or equal to a preset computational cost threshold to obtain trained subnetwork models.

[0114] In one exemplary embodiment, the width pruning module 730 can be used to:

[0115] Calculate the performance parameters of the convolution kernels of each subnetwork in the initial neural network model after deep pruning;

[0116] Based on the performance parameters, the convolutional channels of each convolutional layer in the initial neural network model after depth pruning are pruned to obtain the initial neural network model after width pruning.

[0117] In one exemplary embodiment, the width pruning module 730 can be used to:

[0118] Calculate the convolution computation cost and weight parameter quantity of the convolution kernels of each subnetwork in the initial neural network model after deep pruning;

[0119] The performance parameters of the convolution kernel are determined based on the ratio of the convolution computation amount to the weight parameter amount.

[0120] In one exemplary embodiment, the model output module 740 can be used to:

[0121] Obtain the training dataset;

[0122] The target neural network model is obtained by fine-tuning the initial neural network model after width pruning, which meets the preset model compression parameters, using the training dataset.

[0123] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0124] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0125] Exemplary embodiments of this disclosure also provide an electronic device. This electronic device may be the aforementioned terminal devices 101, 102, 103, and server 105. Generally, the electronic device may include a processor and a memory, the memory being used to store executable instructions of the processor, the processor being configured to perform the aforementioned model compression method by executing the executable instructions.

[0126] The following is based on Figure 8 Taking a mobile terminal 800 as an example, the construction of this electronic device will be described by way of example. Those skilled in the art will understand that, apart from components specifically designed for mobile purposes, Figure 8 The structure can also be applied to fixed types of equipment.

[0127] like Figure 8 As shown, the mobile terminal 800 may specifically include: a processor 801, a memory 802, a bus 803, a mobile communication module 804, an antenna 1, a wireless communication module 805, an antenna 2, a display screen 806, a camera module 807, an audio module 808, a power module 809, and a sensor module 810.

[0128] Processor 801 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, an encoder, a decoder, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). The model compression method in this exemplary embodiment can be executed by an AP, GPU, or DSP. When the method involves neural network-related processing, it can be executed by an NPU. For example, the NPU can load neural network parameters and execute neural network-related algorithm instructions.

[0129] An encoder encodes (compresses) images or videos to reduce data size for easier storage or transmission. A decoder decodes (decompresses) the encoded data to restore the original image or video data. The mobile terminal 800 can support one or more encoders and decoders, such as image formats like JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), and BMP (Bitmap), and video formats like MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0130] The processor 801 can be connected to the memory 802 or other components via the bus 803.

[0131] The memory 802 can be used to store executable program code, which includes instructions. The processor 801 executes various functional applications and data processing of the mobile terminal 800 by running the instructions stored in the memory 802. The memory 802 can also store application data, such as images, videos, and other files.

[0132] The communication functions of the mobile terminal 800 can be implemented through a mobile communication module 804, antenna 1, a wireless communication module 805, antenna 2, a modem processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 804 can provide 3G, 4G, and 5G mobile communication solutions for use on the mobile terminal 800. The wireless communication module 805 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication for use on the mobile terminal 800.

[0133] The display screen 806 is used to implement display functions, such as displaying the user interface, images, and videos. The camera module 807 is used to implement shooting functions, such as capturing images and videos. The audio module 808 is used to implement audio functions, such as playing audio and capturing voice. The power module 809 is used to implement power management functions, such as charging the battery, supplying power to the device, and monitoring battery status.

[0134] The sensor module 810 may include one or more sensors to implement corresponding sensing and detection functions. For example, the sensor module 810 may include an inertial sensor, which is used to detect the motion posture of the mobile terminal 800 and output inertial sensing data.

[0135] Exemplary embodiments of this disclosure also provide a computer-readable storage medium having a program product stored thereon capable of implementing the methods described above in this specification. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0136] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0137] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0138] Furthermore, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0139] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0140] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A model compression method applied to the compression of semantic segmentation models in portrait blurring projects, characterized in that, include: Obtain a pre-trained initial neural network model; The initial neural network model is subjected to deep pruning to obtain a deeply pruned initial neural network model, including: constructing a neural network model search space based on the availability of convolutional layers in the initial neural network model, wherein the neural network model search space includes 2 N N is the number of convolutional layers in a sub-network; random sampling and network training are performed on the sub-networks in the search space of the neural network model to obtain trained sub-network models; based on the evolutionary search method and preset model compression parameters, the sub-networks in the sub-network models are searched to determine the target sub-network, and the target sub-network is used as the initial neural network model after deep pruning. The initial neural network model after depth pruning is subjected to width pruning to obtain an initial neural network model after width pruning. This includes: calculating the convolutional computation cost and weight parameter quantity of the convolutional kernels of each sub-network in the initial neural network model after depth pruning; determining the performance parameters of the convolutional kernels based on the ratio of the convolutional computation cost to the weight parameter quantity; and pruning the convolutional channels of each convolutional layer in the initial neural network model after depth pruning based on the performance parameters to obtain an initial neural network model after width pruning. The initial neural network model after width pruning that meets the preset model compression parameters is determined as the target neural network model; The method combines depth pruning and width pruning to significantly reduce model size and computational load while maintaining model accuracy. This enables the compressed target neural network model to adapt to hardware platforms with different performance levels, thereby improving the efficiency of portrait blurring on mobile terminals.

2. The method according to claim 1, characterized in that, The preset model compression parameters include a model execution efficiency threshold and an output result accuracy threshold; determining the initial neural network model after width pruning that meets the preset model compression parameters as the target neural network model includes: The initial neural network model after width pruning is validated to determine the model execution efficiency and output accuracy of the initial neural network model after width pruning. If the model execution efficiency is greater than or equal to the model execution efficiency threshold, and the output accuracy is greater than or equal to the output accuracy threshold, then the initial neural network model after width pruning is determined as the target neural network model.

3. The method according to claim 1, characterized in that, The step of randomly sampling and training sub-networks in the search space of the neural network model to obtain trained sub-network models includes: Determine the computational cost of each subnetwork in the search space of the neural network model; Random sampling and network training are performed on subnetworks whose computational cost is greater than or equal to a preset computational cost threshold to obtain trained subnetwork models.

4. The method according to claim 1, characterized in that, The step of determining the initial neural network model after width pruning, which satisfies the preset model compression parameters, as the target neural network model includes: Obtain the training dataset; The target neural network model is obtained by fine-tuning the initial neural network model after width pruning, which meets the preset model compression parameters, using the training dataset.

5. A model compression device, used for compressing semantic segmentation models in a portrait blurring project, characterized in that, include: The model acquisition module is used to acquire the pre-trained initial neural network model; The deep pruning module is used to perform deep pruning on the initial neural network model to obtain a deep-pruned initial neural network model. This includes: constructing a neural network model search space based on the availability of convolutional layers in the initial neural network model, wherein the neural network model search space includes 2... N N is the number of convolutional layers in a sub-network; random sampling and network training are performed on the sub-networks in the search space of the neural network model to obtain trained sub-network models; based on the evolutionary search method and preset model compression parameters, the sub-networks in the sub-network models are searched to determine the target sub-network, and the target sub-network is used as the initial neural network model after deep pruning. A width pruning module is used to perform width pruning on the initial neural network model after depth pruning to obtain a width-pruned initial neural network model. This includes: calculating the convolutional computation cost and weight parameter quantity of the convolutional kernels in each sub-network of the initial neural network model after depth pruning; determining the performance parameters of the convolutional kernels based on the ratio of the convolutional computation cost to the weight parameter quantity; and pruning the convolutional channels of each convolutional layer in the initial neural network model after depth pruning based on the performance parameters to obtain a width-pruned initial neural network model. The model output module is used to determine the initial neural network model after width pruning that meets the preset model compression parameters as the target neural network model; The device combines depth pruning and width pruning to significantly reduce model size and computational load while maintaining model accuracy. This enables the compressed target neural network model to adapt to hardware platforms with different performance levels, improving the efficiency of portrait blurring on mobile terminals.

6. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

7. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 4 by executing the executable instructions.

Images