Pruning optimization method and system for deep neural network

Abstract

The invention provides a pruning optimization method and system for a deep neural network, and the method comprises the following steps: carrying out the quantification of an initial neural network model, and generating a quantized neural network model; carrying out pre-training on the quantized neural network model, and carrying out first pruning processing so as to compress the model and generate a pre-trained neural network model; converting the pre-trained neural network model into a computational graph, performing second pruning processing on the computational graph, searching a standard solution, and generating a pruned computational graph; and performing fine tuning optimization on the pruned calculation graph, allocating a memory, generating a pruned and optimized neural network model, and executing the pruned and optimized neural network model on the mobile equipment. The first pruning compresses the model on the premise of ensuring the precision, and the second pruning only searches the standard solution, so that the search space is greatly reduced, the reasoning efficiency is improved, the hardware memory usage amount is reduced to the minimum, and compared with the prior art, the reasoning speed is remarkably improved, and no precision loss exists.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a pruning optimization method and system for deep neural networks. Background Technology

[0002] As intelligent application services shift from the cloud to mobile devices such as laptops and smartphones, they face increasing computational and storage challenges. Running deep neural network (DNN) inference and real-time high-precision tasks on mobile platforms requires substantial hardware and software resources, exceeding the capabilities of mobile devices. Deep neural networks contain complex structures and a large number of parameters, including many unnecessary or unimportant ones. Redundancy can be removed through model compression and pruning techniques to improve the efficiency of deep neural network inference.

[0003] Existing technologies, including unstructured and structured pruning, have been extensively researched and promoted. While unstructured pruning maintains accuracy, it lacks hardware efficiency; while structured pruning, although hardware-friendly, sacrifices accuracy. Furthermore, efficient utilization of memory buffers on mobile devices, particularly in AI inference and accelerator optimization, remains a technical bottleneck. Existing pruning optimization methods either lack optimization for mobile platforms, require specific hardware, or involve trade-offs between compression ratio and accuracy, leading to limitations in practical applications.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, this application provides a pruning optimization method and system for deep neural networks. By combining quantization and weight compression techniques, it improves hardware efficiency while maintaining accuracy to enhance deep neural network (DNN) inference on mobile devices. Furthermore, by adopting a recursive depth-first search strategy, it searches only canonical solutions, thereby significantly reducing the search space, improving memory allocation efficiency, accelerating deep neural network inference, and improving the application efficiency of deep neural networks on mobile devices.

[0006] This application is achieved through the following technical solution:

[0007] The first aspect of this application is to provide a pruning optimization method for deep neural networks, comprising the following steps:

[0008] The initial neural network model is quantized to generate a quantized neural network model.

[0009] The quantized neural network model is pre-trained, and the first pruning process is performed to compress the model and generate a pre-trained neural network model.

[0010] The pre-trained neural network model is converted into a computational graph, and a second pruning process is performed on the computational graph to search for a normalized solution and generate a pruned computational graph.

[0011] The pruned computation graph is fine-tuned and optimized, memory is allocated, a pruned and optimized neural network model is generated, and it is executed on a mobile device.

[0012] Specifically, the step of quantizing the initial neural network model to generate a quantized neural network model includes the following steps:

[0013] Map the original layers in the initial neural network model to low-precision layers;

[0014] Calculate the original layer weight matrix of the original layer, and quantize each of the original layer weight matrices to improve computational efficiency;

[0015] Randomly replace the original layer and the low-precision layer, and calculate the precision loss for each replacement;

[0016] The replacement scheme with the least accuracy loss is selected to optimize accuracy and ensure that the quantized neural network model maintains the same accuracy as the initial neural network model.

[0017] Specifically, the steps of pre-training the quantized neural network model, performing the first pruning process, compressing the model, and generating a pre-trained neural network model include:

[0018] Kernel pruning and layer pruning are performed on the quantized neural network model, some convolutional kernels and convolutional layers are randomly deleted, and then the training process is executed to generate several result data.

[0019] The result data with high precision is filtered out, and the kernel pruning and layer pruning processes corresponding to the result data with high precision are retained to form the pre-trained neural network model.

[0020] Specifically, the kernel pruning includes: obtaining the kernel weight matrix of the convolution kernel;

[0021] Randomly set some weight values ​​in the convolution kernel weight matrix to zero, and retain the non-zero weight values;

[0022] The layer pruning includes: obtaining the number of input channels, where each input channel corresponds to a numerical value;

[0023] Randomly set some of the inputs to zero, and retain the values ​​corresponding to the non-zeroed inputs to perform inference calculations;

[0024] Record the zeroing process and the corresponding calculation results.

[0025] Specifically, the step of converting the pre-trained neural network model into a computational graph, performing a second pruning process on the computational graph, searching for a normalized solution, and generating a pruned computational graph includes the following steps:

[0026] The pre-trained neural network model is converted into a computational graph using a compiler;

[0027] An intermediate layer is derived from the computation graph, the intermediate layer including a search tree;

[0028] A pruning rule based on a recursive depth-first search strategy is determined, and the search tree is pruned and optimized according to the pruning rule to generate the pruned computation graph.

[0029] Specifically, the pruning rules based on the recursive depth-first search strategy include: offset pruning rules and offset priority pruning rules;

[0030] The offset pruning rule introduces an offset adjustment threshold for the pruning process to achieve flexible control over model complexity or resource allocation.

[0031] The offset priority pruning rule combines the priority and the offset amount to distinguish the importance of different parameters or modules, thereby pruning redundant parts more efficiently.

[0032] Specifically, the step of pruning and optimizing the search tree according to the pruning rules includes the following steps:

[0033] List the elements in the search tree and determine the priority and offset of each element, wherein the priority is sorted alphabetically.

[0034] The elements are searched using an optical method, following the order of increasing offset.

[0035] For elements with the same priority, the element with the smallest alphabetical order is searched first to reduce the search space, and the search process for elements with larger alphabetical order is pruned to remove redundancy.

[0036] Specifically, it also includes the steps of: verifying and evaluating the pruned and optimized neural network model, and testing its accuracy and running time.

[0037] A second aspect of this application is to provide a pruning optimization system for deep neural networks, comprising:

[0038] The preprocessing module is used to quantize the initial neural network model and generate a quantized neural network model.

[0039] The pre-training module is used to pre-train the quantized neural network model, perform the first pruning process, thereby compressing the model and generating a pre-trained neural network model.

[0040] The pruning optimization module is used to convert the pre-trained neural network model into a computational graph, perform a second pruning process on the computational graph, search for a normalized solution, and generate a pruned computational graph.

[0041] The execution module is used to fine-tune and optimize the pruned computation graph, allocate memory, generate a pruned and optimized neural network model, and execute it on the mobile device.

[0042] A third aspect of this application is to provide a storage medium, which is a non-transitory computer-readable storage medium, storing a program for a deep neural network pruning optimization method as described above, wherein the program, when executed by a processor, performs the following steps:

[0043] The initial neural network model is quantized to generate a quantized neural network model.

[0044] The quantized neural network model is pre-trained, and the first pruning process is performed to compress the model and generate a pre-trained neural network model.

[0045] The pre-trained neural network model is converted into a computational graph, and a second pruning process is performed on the computational graph to search for a normalized solution and generate a pruned computational graph.

[0046] The pruned computation graph is fine-tuned and optimized, memory is allocated, a pruned and optimized neural network model is generated, and it is executed on a mobile device.

[0047] Compared with the prior art, this application has the following technical effects:

[0048] This application performs the first pruning during the pre-training stage, compressing the model while ensuring accuracy. Then, it adopts a recursive depth-first search strategy to search only canonical solutions in the computation graph, which greatly reduces the search space, improves inference efficiency, and minimizes the resource usage on mobile devices. Compared with the existing TensorFlow Lite and TVM methods, the inference speed is significantly improved without any loss of accuracy. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a deep neural network pruning optimization method according to an embodiment of this application.

[0051] Figure 2 This is a flowchart based on the Mobimind framework, which is an embodiment of this application.

[0052] Figure 3 This is a schematic diagram of pruning during the pre-training stage of an embodiment of this application.

[0053] Figure 4 This is a schematic diagram of a search method based on an optical solver, as described in an embodiment of this application.

[0054] Figure 5 This is a performance comparison chart between the pruned and optimized neural network model of this application and existing neural network models.

[0055] Figure 6 This is a schematic diagram of a deep neural network pruning optimization system according to an embodiment of this application.

[0056] Figure 7 This is a flowchart illustrating the quantization process of an initial neural network model, as described in an embodiment of this application.

[0057] Figure 8 This is a flowchart of the first pruning process in a deep neural network pruning optimization method according to an embodiment of this application.

[0058] Figure 9 This is a flowchart of the second pruning process in a deep neural network pruning optimization method according to an embodiment of this application.

[0059] Figure 10 This is a flowchart illustrating the recursive depth-first search strategy in the second pruning process of an embodiment of this application. Detailed Implementation

[0060] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0062] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0063] Existing pruning techniques fall at two extremes in the design space: unstructured pruning allows arbitrary weight pruning, achieving high compression rates but limiting hardware parallelism, while structured pruning (such as filter / channel pruning) is hardware-friendly but leads to greater accuracy loss. Therefore, real-time deep neural network (DNN) inference on mobile devices requires an end-to-end framework that strikes a balance between high accuracy and hardware efficiency.

[0064] In this application, a deep neural network model is transformed into a computation graph by a compiler. Typically, compilers follow a similar process: (1) representing the model as a computation graph; (2) decomposing the computation graph into smaller, more manageable computational units (operators); and (3) allocating operators to specific hardware components. When compilers are tasked with scheduling multiple operators simultaneously, they encounter challenges related to memory allocation in the accelerator's temporary memory. This challenge involves managing buffers with overlapping active scopes, a key challenge in memory allocation.

[0065] The challenge of neural network pruning lies in balancing accuracy and hardware efficiency. In this application, the inference framework achieves high accuracy through kernel pruning and layer pruning, while leveraging compiler optimizations to improve execution efficiency.

[0066] The goal is to identify the most effective weight pruning strategy to accelerate DNNs on mobile devices and to develop a corresponding full-stack acceleration framework. In this application, 16-bit floating-point (16FP) notation is used on GPUs to represent weights and intermediate results. Combined with low-precision techniques, this format is widely supported on mobile devices and does not compromise the accuracy of the DNN.

[0067] The compiler in this application optimizes the structured pruning process by focusing on kernel mode and layers, integrating hardware and software to form a multi-level caching system. Hardware ensures fast memory access and efficient local data usage, while software optimization enables fine-tuning of performance. Specifically, the first pruning, also known as structured pruning, is performed on convolutional kernels and layers during the pre-training phase to compress the model. Then, the pre-trained neural network model is compiled into a computational graph, and a second pruning step optimizes the graph, integrating hardware and software to form a multi-level caching system.

[0068] like Figure 1 The diagram shown is a flowchart of a deep neural network pruning optimization method according to an embodiment of this application, including the following steps:

[0069] S1, quantize the initial neural network model to generate a quantized neural network model;

[0070] S2, pre-train the quantized neural network model, perform the first pruning process to compress the model and generate a pre-trained neural network model;

[0071] S3, the pre-trained neural network model is converted into a computation graph, the computation graph is pruned a second time, a normalized solution is searched, and a pruned computation graph is generated;

[0072] S4. Fine-tune and optimize the pruned computation graph, allocate memory, generate the pruned and optimized neural network model, and execute it on the mobile device.

[0073] Specifically, such as Figure 7 As shown, step S1, which involves quantizing the initial neural network model to generate a quantized neural network model, includes the following steps:

[0074] S11, Map the original layers in the initial neural network model to low-precision layers;

[0075] S12, calculate the original layer weight matrix of the original layer, and quantize each of the original layer weight matrices to improve computational efficiency;

[0076] S13, randomly replace the original layer and the low-precision layer, and calculate the precision loss for each replacement;

[0077] S14. Select the replacement scheme with the least accuracy loss to achieve accuracy optimization and ensure that the quantized neural network model maintains the same accuracy as the initial neural network model.

[0078] Convolutional kernel quantization is an effective method for lightweighting models, allowing neural networks to operate efficiently at low precision through digital recoding. Quantization refers to converting floating-point parameters (such as weights and activation values) in a neural network model into low-precision integer representations, for example, from 32-bit floating-point numbers to 8-bit integers. This reduces the model's storage requirements, speeds up inference, and lowers power consumption, which is especially important when deployed on edge devices. In this application, the quantizer design leverages specific hardware features, such as tensor cores, to accelerate low-precision operations, ensuring a balance between computational speed and accuracy. The optimal configuration providing the lowest precision loss is identified by strategically replacing original and low-precision layers.

[0079] Specifically, Tensor Cores are the "deep learning acceleration engines" of NVIDIA GPUs. They solve general problems through dedicated hardware, significantly improve the efficiency of matrix operations, and are the core technology for training large models (such as GPT and LLaMA) and inference.

[0080] Specifically, the original neural network layers are mapped to low-precision layers. Let L... k Let k represent the k-th layer of the original neural network, where k = 1, ..., N, and N represents the total number of layers. The weights of each layer are... Where P k and Q k These are the height and width of the convolution kernel, C. k It is the number of input channels, C k+1 This refers to the number of output channels. In this application, the weight matrix of each original layer is quantized to improve computational efficiency. Low-precision mapping. It can be represented as:

[0081]

[0082] Where Q represents quantization operation, W k This indicates the weight of each layer, and P represents the quantization bit width, for example, 16-bit FP16 or 8-bit INT8.

[0083] GPU acceleration and low-precision computation are performed on a single GPU, with low-precision computation accelerated by dedicated hardware units such as tensor cores. For example, matrix multiplication A×B can be represented as:

[0084] C = A × B ， (2)

[0085] Where A∈R m×k , B∈Rk×n The generated matrix C∈R m×n It is low-precision, such as FP16. In one embodiment of this application, for a certain mobile device, the GPU automatically utilizes tensor cores for efficient computation to significantly improve matrix operation efficiency through dedicated hardware. Typically, a mobile device has only one GPU, and all computations are performed on this device. This application optimizes task scheduling and layer computation order to maximize GPU utilization.

[0086] Random replacement and precision optimization: During quantization, this application performs random replacement between the original layer and the low-precision layer. For each replacement, the precision loss between the low-precision layer and the original layer is calculated. The precision loss Δ(W) k The calculation method for ) is as follows:

[0087] Δ(W k )=|Acc(low-precision(W k ))-Acc(original(W k ))|,(3)

[0088] Specifically, compare the accuracy before and after the replacement, and select the replacement scheme with the least loss of accuracy.

[0089] The baseline for accuracy optimization: Select the top 3 layers with the smallest accuracy loss. Calculate the accuracy loss for each replacement Δk, and then select the layer with the smallest loss.

[0090]

[0091] The calculation in step (4) ensures that the accuracy loss of these three layers is minimized, and ensures that the quantization model maintains an accuracy close to that of the original model, thereby ensuring the accuracy in the pruning optimization process.

[0092] like Figure 2 The diagram shown is a flowchart of a deep neural network pruning optimization method based on the Mobimind framework according to an embodiment of this application.

[0093] MOBIMIND, developed by Mobimind.ai, is an artificial intelligence platform focusing on Natural Language Processing (NLP) and Natural Language Understanding (NLU) technologies, aiming to automate and optimize interactions with text and language data. This application employs a pruning optimization method for deep neural networks on smart devices based on the Mobimind inference framework.

[0094] Specifically, in one embodiment of this application, the MobiMind inference framework has two stages: (1) Pattern-based pre-training: the training process is optimized using specific patterns, and tensor cores technology is used to accelerate low-precision operations to support selective pruning; (2) Code generation and optimization: the pre-trained neural network model is converted into a computation graph, graph optimization is achieved through pruning, and then memory matching and code generation are performed.

[0095] The quantized neural network model undergoes pre-training and optimization. The first pruning is performed during the pre-training stage to compress the model. After being compiled into a computational graph, a second pruning is performed. Then, memory is allocated according to the cache size of the hardware system to form a pruned and optimized neural network model, which is then executed on the hardware device.

[0096] Specifically, such as Figure 8 As shown, step S2 involves pre-training the quantized neural network model, performing the first pruning process, compressing the model, and generating a pre-trained neural network model, including the following steps:

[0097] S21, perform kernel pruning and layer pruning on the quantized neural network model, randomly delete some convolutional kernels and convolutional layers, and then execute the training process to generate several result data;

[0098] S22, the result data with high precision is filtered out, and the kernel pruning and layer pruning processes corresponding to the result data with high precision are retained to form the pre-trained neural network model.

[0099] Specifically, each convolutional layer includes several convolutional kernels, and the convolutional kernel weight matrix corresponding to each convolutional kernel includes several weight values. First, kernel pruning is performed by randomly deleting several weight values ​​to achieve kernel pruning. Then, several convolutional layers are randomly deleted to achieve layer pruning, thereby achieving the effect of simplifying the model.

[0100] Compared to irregular structures, certain regular patterns can achieve high accuracy and functional approximation capabilities. Based on these two characteristics, this application proposes two pattern-based pruning techniques: kernel pruning and layer pruning. Here, "kernel" refers to the convolutional kernel in a neural network.

[0101] Kernel pruning: This involves pruning a fixed number of weights for each kernel in a convolutional filter, leaving a specific "kernel pattern" with non-zero weights.

[0102] like Figure 3The diagram shown illustrates a pruning process in a 4-graph mode according to an embodiment of this application. Each convolutional kernel has a 3*3 kernel weight matrix, comprising 9 weight values, which is the most commonly used size. After kernel pruning, only 4 non-zero weights are retained. This method can be applied to other kernel sizes and fully connected (FC) layers.

[0103] Specifically, the kernel pruning includes: obtaining the kernel weight matrix of the convolution kernel;

[0104] Randomly set some weight values ​​in the convolution kernel weight matrix to zero, and retain the non-zero weight values.

[0105] Predefined patterns allow the compiler to reorder and generate code at the filter and kernel levels, optimizing instruction-level parallelism by grouping kernels with the same pattern. This approach is applicable to embedded processor architectures such as GPUs or CPUs and can be applied to any predefined pattern, not just the 4-entry pattern used in the embodiments of this application.

[0106] Layer pruning focuses on severing connections between certain input and output channels, equivalent to removing the corresponding convolutional kernels. In a convolutional layer, the correlation between input channel i and output channel j is represented by the i-th convolutional kernel of filter j. Layer pruning aims to overcome the limited pruning rate achieved through convolutional kernel pattern pruning.

[0107] Specifically, layered pruning includes:

[0108] Obtain the number of user input channels, with each input channel corresponding to a numerical value;

[0109] Randomly set some of the inputs to zero, and retain the values ​​corresponding to the non-zeroed inputs to perform inference calculations;

[0110] Record the process of setting the value to zero and the calculation results;

[0111] Different zeroing processes are used depending on the scenario.

[0112] Specifically, zeroing can be understood as the core operation of the first pruning process. The requirements for zeroing strength and sparsity vary in different application scenarios. For example, in the scenario of object detection, the zeroing strength is greater and the sparsity is higher. That is, during pruning, the intensity of deleting some convolutional kernels and convolutional layers is greater.

[0113] like Figure 3 The diagram shows the process of the first pruning during the pre-training phase. This involves randomly deleting some convolutional kernels and layers, or randomly setting some convolutional kernels and layers to zero, thus compressing the model.

[0114] Specifically, in the pre-training phase, the training loop is at least 50 times. In each loop, several convolutional kernels and convolutional layers are randomly deleted from the neural network model to perform training, generating several result data. The data with higher accuracy is selected, and the set consisting of the data and the corresponding pruning process is taken as the first pruning process. The resulting neural network model is the pre-trained neural network model.

[0115] After pre-training, the MobiMind framework uses a compiler to transform the pre-trained neural network model into a computation graph. Combining heuristic methods with solver-based methods, a recursive depth-first search strategy is adopted to explore only canonical solutions, thereby significantly reducing the search space, improving memory allocation efficiency, and achieving pruning optimization.

[0116] Specifically, the MobiMind framework provides an efficient and flexible solution for mobile CPUs and GPUs, generating optimized execution code for deep neural networks to maximize the performance of edge devices.

[0117] Specifically, in this application, the MobiMind framework converts deep neural networks (DNNs) into computational graphs to implement structured optimization methods.

[0118] Specifically, such as Figure 9 As shown, step S3 involves converting the pre-trained neural network model into a computational graph, performing a second pruning process on the computational graph, searching for a normalized solution, and generating a pruned computational graph. This includes the following steps:

[0119] S31, The pre-trained neural network model is converted into a computation graph using a compiler;

[0120] S32, Derive an intermediate layer from the computation graph, the intermediate layer including a search tree;

[0121] S33, determine the pruning rules based on the recursive depth-first search strategy, and perform pruning optimization on the search tree according to the pruning rules to generate the pruned computation graph.

[0122] Specifically, in one embodiment, in step S33, the pruning rules based on the recursive depth-first search strategy include: offset pruning rules and offset priority pruning rules;

[0123] The offset pruning rule introduces an offset adjustment threshold for the pruning process to achieve flexible control over model complexity or resource allocation.

[0124] The offset priority pruning rule combines the priority and the offset amount to distinguish the importance of different parameters or modules, thereby pruning redundant parts more efficiently.

[0125] Specifically, such as Figure 10As shown, in one embodiment, step S33, which involves pruning and optimizing the search tree according to the pruning rules, includes the following steps:

[0126] S331, List the elements in the search tree and determine the priority and offset of each element, wherein the priority is sorted alphabetically;

[0127] S332, The element is searched using optical resolution, and the element is searched in ascending order of the offset;

[0128] S333, for elements with the same priority, prioritize searching the element with the smallest alphabetical order to reduce the search space, and perform pruning on the search process of the element with the largest alphabetical order to remove redundancy.

[0129] like Figure 4 As shown, this application proposes a method for static assignment of computational graphs, namely the LightSolver Approaches, which focuses on efficiently solving problems, reducing the search space, significantly shortening the search time, and improving the efficiency of pruning optimization.

[0130] Specifically, in order to optimize static allocation, while ensuring reliability and integrity, this application strengthens the constrained solution space by restricting the search space to a search lattice to adapt to the buffer filling problem, in which a specific structure must be satisfied.

[0131] Specifically, this application includes two pruning rules: the offset pruning rule and the offset priority pruning rule.

[0132] Offset pruning rules adjust the pruning threshold or priority by introducing an offset, thus providing more flexible control over model complexity or resource allocation. The adjustment value introduced during pruning is used to dynamically adjust the sensitivity or priority of pruning. Specifically, in this application, for a given set of elements, the offset pruning rule trims the search space by focusing on the offsets of the elements. If two elements have the same offset, the element with the smaller lexicographical order of its label is given priority.

[0133] Offset priority pruning rules combine priority and offset to differentiate the importance of different parameters or modules, thereby pruning redundant parts more efficiently. Priority- and offset-based pruning strategies achieve finer-grained pruning control by quantifying the importance and risk of objects. Specifically, in one embodiment, if the offset of element A is greater than that of element B, A is searched first. If two elements have the same offset, they are searched alphabetically, prioritizing elements with smaller alphabetical order.

[0134] Specifically, in one embodiment, the pruning process of the search tree includes:

[0135] For a set of elements X ∈1 (O ∈1 ), ..., X ∈n (O ∈n ), if and only if: O ∈1 ≤O ∈2 ≤···≤O ∈n and If O ∈i =O ∈j , then ∈ i <∈ j The search is performed with offset priority. This ensures that elements with smaller offsets appear first, and if offsets are the same, they are ordered alphabetically. Here, ∈1, ..., ∈n represent n elements, X ∈n (O ∈n ) represents the element representation. X uses 26 English letters to represent the order, and O uses numbers to represent the offset. For example, A(2) means that the offset of element A is 2, and A is the first in the order. For elements with the same offset, element A(2) is given priority in the search.

[0136] Specifically, such as Figure 4 As shown, in one embodiment, five elements A(2), B(1), C(2), D(4), E(2), F(4), and G(0) are set, with the numbers in parentheses representing offsets. This application applies pruning rules for centralized searching, and the goal is not to sort the elements comprehensively, but to efficiently guide the search.

[0137] Specifically, the pruning strategy includes the following steps:

[0138] 1. Initial exploration: First, consider all elements and prune according to the element's offset value. The elements are searched in the following order: G(0), B(1), A(2), C(2), E(2), D(4), F(4).

[0139] 2. Pruning elements with the same offset: For elements with the same offset value (such as A, C, and E, all with an offset of 2), prioritize the element with the smallest alphabetical order to reduce the search space. For example, the search order for E(2), A(2), C(2) is: A(2), C(2), E(2). Similarly, for D(4) and F(4), prioritize searching D(4), then F(4).

[0140] 3. Pruning Process: This application employs a backtracking method instead of sorting all elements. By focusing on the standard solution, the search space is reduced. The strategy is to systematically explore elements and eliminate infeasible options based on their offsets and labels. This reduces the complexity of the problem by progressively narrowing the candidate set at each step of the exploration, thus achieving more efficient pruning.

[0141] Specifically, after step S4, the deep neural network pruning and optimization method of this application further includes step S5, which verifies and evaluates the pruned and optimized neural network model, and tests its accuracy and running time.

[0142] To further illustrate the superior technical performance of the deep neural network model pruning optimization method proposed in this application, comparative experiments were conducted to compare the method with existing DNN inference acceleration frameworks TFLite, TVM, and MNN, and to evaluate the performance of the MobiMind-based pruning optimization method.

[0143] The evaluation platform and running configuration for the comparative test: a 12th generation Intel(R) Core(TM) i9-12950HX laptop equipped with a 3080Ti-m. All tests were run 50 times with different inputs (images), with the CPU using 8 threads and the GPU using all pipelines.

[0144] Since the results from multiple runs did not show significant differences, this experiment only presents the average time. Because convolutional layers are the most time-consuming part, accounting for over 95% of the total execution time (90% for the VGG network), our evaluation focuses on convolutional layers. All runs were optimized for performance; for example, all intensive runs were optimized using Winograd, and all GPU runs used 16-bit floating-point numbers.

[0145] Accuracy Validation and Analysis: We evaluated the accuracy of the quantizer-based joint kernel pruning and layer pruning method on VGG-16 and ResNet-50 datasets using the ImageNet ILSVRC-2012 and CIFAR-10 datasets. Starting with the pre-trained model, we limited the number of training epochs for kernel mode and layer pruning to 120—consistent with the original DNN training in PyTorch and significantly lower than the number of epochs used in iterative pruning.

[0146] Table 1. Accuracy Comparison of the Top Five Kernel Pruning Modes

[0147] Network Original DNN 8-image mode 12-image mode VGG16 91.8％ 92.2％ 92.3％ ResNet50 92.6％ 92.8％ 93.1％

[0148] Table 1 shows a comparison of the top five accuracies when kernel mode pruning is applied to the convolutional layers of VGG-16 and ResNet-50 using the ImageNet dataset. The reference group is the original DNN model without using any modes. The accuracy results show examples for 8-image and 12-image modes, respectively. The results indicate that accuracy improves when the number of images is sufficiently high, typically 8 to 12, which is attributed to the compatibility of pruning with the human visual system and its ability to reduce overfitting.

[0149] Table 2 shows the convolution weight reduction based on joint kernel pattern pruning (the set contains 8 patterns) and connectivity pruning.

[0150]

[0151]

[0152] Table 2 compares the execution time latency of VGG16 and ResNet-50 on mobile devices using the ImageNet dataset. For VGG16, 3×3 convolutional kernels were used. The results show that using quantizers significantly reduces the runtime of ResNet-50, decreasing the inference time of convolutional layers by 20%–30%, while accuracy (precision) remains largely unchanged. Table 2 shows no decrease in accuracy, and the runtime is comparable to that of quantizers on convolutional layers.

[0153] like Figure 5 The figure shows a performance comparison between the pruning optimization method of this application and existing methods. The x-axis represents different pre-trained DNN models; the y-axis represents the average DNN inference runtime for a single input. As can be seen from the figure, the deep neural network pruning optimization method of this application has a shorter runtime and higher inference efficiency.

[0154] like Figure 6 As shown, a second aspect of this application is to provide a deep neural network pruning optimization system 10, comprising:

[0155] Preprocessing module 11 is used to quantize the initial neural network model and generate a quantized neural network model;

[0156] The pre-training module 12 is used to pre-train the quantized neural network model, perform the first pruning process, thereby compressing the model and generating a pre-trained neural network model.

[0157] The pruning optimization module 13 is used to convert the pre-trained neural network model into a computational graph, perform a second pruning process on the computational graph, search for a normalized solution, and generate a pruned computational graph.

[0158] Execution module 14 is used to fine-tune and optimize the pruned computation graph, allocate memory, generate a pruned and optimized neural network model, and execute it on a mobile device.

[0159] The verification module 15 is used to verify and evaluate the pruned and optimized neural network model, and to test its accuracy and running time.

[0160] A third aspect of this application is to provide a storage medium, which is a non-transitory computer-readable storage medium, storing a program for a deep neural network pruning optimization method as described above, wherein the program, when executed by a processor, performs the following steps:

[0161] S1, quantize the initial neural network model to generate a quantized neural network model;

[0162] S2, pre-train the quantized neural network model, perform the first pruning process to compress the model and generate a pre-trained neural network model;

[0163] S3, the pre-trained neural network model is converted into a computation graph, the computation graph is pruned a second time, a normalized solution is searched, and a pruned computation graph is generated;

[0164] S4, fine-tune and optimize the pruned computation graph, allocate memory, generate the pruned and optimized neural network model, and execute it on the mobile device;

[0165] S5, verify and evaluate the pruned and optimized neural network model, and test its accuracy and running time.

[0166] Compared with the prior art, this application has the following technical effects:

[0167] (1) Use the compiler to seamlessly bridge the efficiency gap between structured pruning and semi-structured pruning.

[0168] (2) A recursive depth-first search strategy is adopted, which searches only the normal solution, greatly reducing the search space.

[0169] (3) The Mobimind-based inference framework minimizes resource usage on mobile devices and significantly improves inference speed without loss of accuracy compared to existing TensorFlow Lite and TVM methods.

[0170] In summary, this application provides a pruning optimization method for deep neural networks, comprising the following steps: quantizing an initial neural network model to generate a quantized neural network model; pre-training the quantized neural network model and performing a first pruning process to compress the model and generate a pre-trained neural network model; converting the pre-trained neural network model into a computation graph, performing a second pruning process on the computation graph, searching for canonical solutions, and generating a pruned computation graph; fine-tuning and optimizing the pruned computation graph, allocating memory, generating a pruned and optimized neural network model, and executing it on a mobile device. This application performs a first pruning during the pre-training stage, compressing the model while maintaining accuracy. Then, it employs a recursive depth-first search strategy, searching only for canonical solutions in the computation graph, which greatly reduces the search space, improves inference efficiency, and minimizes resource usage on mobile devices. Compared with existing TensorFlow Lite and TVM methods, it significantly improves inference speed without loss of accuracy.

[0171] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions are not in essence a departure from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A pruning optimization method for deep neural networks, characterized in that, Includes the following steps: The initial neural network model is quantized to generate a quantized neural network model. The quantized neural network model is pre-trained, and the first pruning process is performed to compress the model and generate a pre-trained neural network model. The pre-trained neural network model is converted into a computational graph, and a second pruning process is performed on the computational graph to search for a normalized solution and generate a pruned computational graph. The pruned computation graph is fine-tuned and optimized, memory is allocated, a pruned and optimized neural network model is generated, and it is executed on a mobile device.

2. The pruning optimization method for a deep neural network according to claim 1, characterized in that, The step of quantizing the initial neural network model to generate a quantized neural network model includes the following steps: Map the original layers in the initial neural network model to low-precision layers; Calculate the original layer weight matrix of the original layer, and quantize each of the original layer weight matrices; Randomly replace the original layer and the low-precision layer, and calculate the precision loss for each replacement; The replacement scheme with the least accuracy loss is selected to optimize accuracy and ensure that the quantized neural network model maintains the same accuracy as the initial neural network model.

3. The pruning optimization method for a deep neural network according to claim 1, characterized in that, The steps of pre-training the quantized neural network model, performing the first pruning process, compressing the model, and generating a pre-trained neural network model include: Kernel pruning and layer pruning are performed on the quantized neural network model, some convolutional kernels and convolutional layers are randomly deleted, and then the training process is executed to generate several result data. The result data with high precision is selected, and the kernel pruning and layer pruning processes corresponding to the result data with high precision are retained to form the pre-trained neural network model.

4. The pruning optimization method for a deep neural network according to claim 3, characterized in that, The kernel pruning includes: obtaining the kernel weight matrix of the convolution kernel; Randomly set some weight values ​​in the convolution kernel weight matrix to zero, and retain the non-zero weight values; The layer pruning includes: obtaining the number of input channels, where each input channel corresponds to a numerical value; Randomly set some of the inputs to zero, and retain the values ​​corresponding to the non-zeroed inputs to perform inference calculations; Record the zeroing process and the corresponding calculation results.

5. The pruning optimization method for a deep neural network according to claim 1, characterized in that, The process of converting the pre-trained neural network model into a computational graph, performing a second pruning process on the computational graph, searching for a normalized solution, and generating a pruned computational graph includes the following steps: The pre-trained neural network model is converted into a computational graph using a compiler; An intermediate layer is derived from the computation graph, the intermediate layer including a search tree; A pruning rule based on a recursive depth-first search strategy is determined, and the search tree is pruned and optimized according to the pruning rule to generate the pruned computation graph.

6. The pruning optimization method for a deep neural network according to claim 5, characterized in that, The pruning rules based on the recursive depth-first search strategy include: offset pruning rules and offset priority pruning rules; The offset pruning rule introduces an offset adjustment threshold for the pruning process to achieve flexible control over model complexity or resource allocation. The offset priority pruning rule combines the priority and the offset amount to distinguish the importance of different parameters or modules, thereby pruning redundant parts more efficiently.

7. The pruning optimization method for a deep neural network according to claim 6, characterized in that, The step of pruning and optimizing the search tree according to the pruning rules includes the following steps: List the elements in the search tree and determine the priority and offset of each element, wherein the priority is sorted alphabetically. The elements are searched using an optical method, following the order of increasing offset. For elements with the same priority, the element with the smallest alphabetical order is searched first to reduce the search space, and the search process for elements with larger alphabetical order is pruned to remove redundancy.

8. The pruning optimization method for a deep neural network according to claim 1, characterized in that, It also includes the steps of: verifying and evaluating the pruned and optimized neural network model, and testing its accuracy and running time.

9. A pruning optimization system for deep neural networks, characterized in that, include: The preprocessing module is used to quantize the initial neural network model and generate a quantized neural network model. The pre-training module is used to pre-train the quantized neural network model, perform the first pruning process, thereby compressing the model and generating a pre-trained neural network model. The pruning optimization module is used to convert the pre-trained neural network model into a computational graph, perform a second pruning process on the computational graph, search for a normalized solution, and generate a pruned computational graph. The execution module is used to fine-tune and optimize the pruned computation graph, allocate memory, generate a pruned and optimized neural network model, and execute it on the mobile device.

10. A storage medium, characterized in that, The storage medium is a non-transitory computer-readable storage medium storing a program for a deep neural network pruning optimization method as described in any one of claims 1-8, wherein the program, when executed by a processor, performs the following steps: The initial neural network model is quantized to generate a quantized neural network model. The quantized neural network model is pre-trained, and the first pruning process is performed to compress the model and generate a pre-trained neural network model. The pre-trained neural network model is converted into a computational graph, and a second pruning process is performed on the computational graph to search for a normalized solution and generate a pruned computational graph. The pruned computation graph is fine-tuned and optimized, memory is allocated, a pruned and optimized neural network model is generated, and it is executed on a mobile device.

Images