Neural network model cutting method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a cutting method and device of a neural network model, equipment, a medium and a program product. The method comprises the following steps: based on a network structure of a target model, determining an output error amount after cutting of the target model corresponding to a cutting rate of each network layer in the network structure; based on the additive characteristic of the network structure, determining that the total output error amount of the target model after cutting is the sum of all the output error amounts after cutting; based on the network structure, determining that the constraint condition of the cutting action is that the sum of the cut sizes of the network layers is not greater than the target cut size; on the premise that the constraint condition is met, the optimal value of the cutting rate corresponding to each network layer in the network structure is solved by taking the minimum output error total amount after cutting as a target; and cutting each network layer in the network structure based on the cutting rate optimal value corresponding to each network layer in the network structure. According to the method, the optimal cutting rate of each layer is determined by taking the overall output effect of the model as a target, and more accurate cutting operation is realized.

Description

Technical Field

[0001] This invention pertains to machine learning, and particularly relates to a method, apparatus, device, computer storage medium, and computer program product for pruning a neural network model. Background Technology

[0002] With the development of neural networks, the parameter scale of neural network models has become increasingly large, resulting in extremely high resource consumption during model training and application, high operating costs, and difficulty in deployment on mobile devices with limited computing resources. To reduce the computational burden and resource consumption of neural networks, model pruning techniques for trimming neural network parameters have emerged. Currently, model pruning techniques mainly target the output accuracy of each pruned layer, and cannot guarantee the overall performance of the neural network model. Summary of the Invention

[0003] This invention provides a method, apparatus, device, computer storage medium, and computer program product for pruning a neural network model. It can determine the optimal pruning rate of each layer in the network with the overall output effect of the model as the target, thereby achieving more accurate pruning operations and better pruning results.

[0004] In a first aspect, embodiments of the present invention provide a method for pruning a neural network model, comprising: Based on the network structure of the target model, determine the pruning rate of each network layer in the corresponding network structure and the output error of the target model after pruning; Based on the additivity of network structure, the total output error of the target model after pruning is determined to be the sum of the pruning rates of each network layer in the corresponding network structure and the output error of the target model after pruning. Based on the network structure, the constraint condition for the pruning action is that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size. Under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, we solve for the optimal value of the pruning rate corresponding to each network layer in the network structure. Based on the optimal pruning rate of each network layer in the network structure, each network layer in the network structure is pruned.

[0005] Secondly, embodiments of the present invention provide a neural network model pruning device, comprising: The single-layer pruning error determination module is configured to determine the pruning error of the target model after pruning, based on the network structure of the target model and the pruning rate of each network layer in the network structure. The error total determination module is configured to determine the total output error of the target model after pruning based on the additivity property of the network structure. This total output error is the sum of the pruning rates of each network layer in the corresponding network structure and the total output error of the target model after pruning. The constraint determination module is configured to determine the constraint condition for the pruning action based on the network structure, which is that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size. The optimal planning solution module is configured to find the optimal pruning rate for each network layer in the network structure, with the goal of minimizing the total number of output errors after pruning, while satisfying the constraints. The pruning module is configured to prune each network layer in the network structure based on the optimal pruning rate corresponding to each network layer in the network structure.

[0006] Thirdly, embodiments of the present invention provide a neural network model pruning device, the device comprising: a processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the neural network model pruning method as described above.

[0007] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the above-described pruning method for a neural network model.

[0008] Fifthly, embodiments of the present invention provide a computer program product, including a computer program, which, when executed, implements the above-described pruning method for a neural network model.

[0009] The neural network model pruning method, apparatus, device, and computer storage medium of this invention determine the total output error of the target model after pruning based on the additivity of the network structure. This is the sum of the output errors of each network layer after pruning. The pruning size determines the constraints, thereby minimizing the total output error of the target model after pruning while satisfying the constraints. The optimal pruning rate for each network layer is calculated and used for pruning. This assigns a more precise pruning rate to each network layer in the target model, ensuring high accuracy of the target model's output while achieving a high pruning rate. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1This is a schematic diagram illustrating the creation process of the neural network model provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for pruning a neural network model according to an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the additivity characteristics of a visual converter model provided in an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the effect of a pruning method for a neural network model using existing technology. Figure 5 This is a schematic diagram illustrating the effect of a neural network model pruning method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a neural network model pruning device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a neural network model pruning device provided in an embodiment of the present invention. Detailed Implementation

[0012] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0013] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0014] Figure 1 This is a schematic diagram illustrating the creation process of the neural network model provided in an embodiment of the present invention. For example... Figure 1As shown, model pruning, occurring before model deployment, is a crucial step in model optimization. It aims to reduce the size of the neural network, decrease its computational burden and resource consumption, and make it easier to deploy the model to mobile devices. Currently, model pruning techniques primarily target the output accuracy of each pruned layer, and cannot guarantee the overall performance of the neural network model.

[0015] To address the problems of existing technologies, embodiments of the present invention provide a method, apparatus, device, and computer storage medium for pruning neural network models. The embodiments of the present invention use minimizing the total output error of the pruned target model as the objective to solve for the optimal pruning rate for each network layer, thereby performing more precise pruning on each network layer and ensuring high output accuracy while achieving a high pruning rate.

[0016] The method for pruning neural network models provided in this embodiment of the invention will be introduced below.

[0017] Figure 2 This is a flowchart illustrating a neural network model pruning method provided in an embodiment of the present invention. Figure 2 As shown, the method may include the following steps S201-S205.

[0018] S201: Based on the network structure of the target model, determine the pruning rate of each network layer in the corresponding network structure and the output error of the target model after pruning.

[0019] Specifically, the target model can include mainstream transformer neural networks, such as LLaVA. Multiple network layers in the target model form a network structure, and network layers with at least one of layer weights and layer activations can be pruned. Specifically, layer weights refer to the weights of all parameterized network layers in the neural network model, including projection layers, QKV, FC1, FC2, etc. Layer activations refer to the outputs of any intermediate layer in the network, including token features of each intermediate layer in the neural network model, as well as query, key, and value. Other non-parameterized layers in the neural network model, such as attention, softmax, and layernorm layers, are usually not pruned. Therefore, in some embodiments, each network layer in the network structure is a network layer that includes at least one of layer weights and layer activations.

[0020] Specifically, the post-pruning output error refers to the difference between the output of the target model after pruning a network layer according to a certain pruning ratio and the output of the target model before pruning that network layer. It can be represented by Euclidean distance. The post-pruning output error of the target model corresponding to a specific network layer will change with the pruning ratio of that network layer. The post-pruning output error of the target model corresponding to the pruning ratio of any network layer in the network structure can be expressed as the Euclidean distance between the output of the target model before pruning and the output of the target model after pruning that network layer according to the pruning ratio.

[0021] Specifically, set Define the number of network layers. This is the output of the last layer of the original, uncropped network, i.e., the output of the target model before pruning. The Euclidean distance between the output of the last layer of the pruned network, i.e., the pruned output, and the output can be expressed as: Furthermore, the layer weights of the i-th layer are pruned using a pruning rate. The output error after cropping can be expressed as: Similarly, the activation values ​​of the i-th layer of the network are pruned using a pruning rate. The output error after cropping can be expressed as: , where i is the index of the network layer being pruned. The cutting rate typically ranges from [value range missing]. .

[0022] Specifically, in some embodiments, determining the pruning rate of each network layer in the corresponding network structure and the pruned output error of the target model may include: traversing multiple given pruning rates distributed within a preset pruning rate range, and for each network layer in the network structure, determining the pruned output error of the target model after pruning the network layer according to each given pruning rate through network inference. This network inference can be implemented based on a calibration dataset. The preset pruning rate range can be [0,1], or a range not greater than 1 and not less than 0. Multiple given pruning rates are selected within the preset pruning rate range, and these multiple given pruning rates can be evenly distributed within the preset pruning rate range. For example, a pruning rate interval K can be set, and then pruning rates can be uniformly sampled within the preset pruning rate range. , K is used as multiple given sampling rates. For example, if the preset clipping rate range is [0,1], K is set to 100. In practice, the preset clipping rate range and the number of given clipping rates can be set according to the number of model parameters and the size of the dataset. Furthermore, under a given clipping rate, the output error after clipping of any network layer is determined through network inference. After traversing all given clipping rates, a set of output error after clipping that varies with the clipping rate of that network layer can be obtained.

[0023] S202: Based on the additivity of network structure, the total output error of the target model after pruning is determined to be the sum of the output errors of all pruned models.

[0024] Specifically, the network structure of a neural network model determines whether it possesses additivity. The prerequisite for additivity is that the expected values ​​of any two network layers are independent when their respective layer weights or activation values ​​are pruned; that is, for the _th ... Layer network layer ( ): , in The output result of the previous layer before pruning the i-th layer of the network. Output results after clipping The expected value of the difference, similarly, The output of the previous layer before pruning the j-th layer of the network. Output after cropping The expected value of the difference.

[0025] Neural network models that hold this premise have the property of additiveness, such as the VisionTransformers (ViTs) model. Figure 3 This is a schematic diagram illustrating the additivity characteristics of a visual converter model provided in an embodiment of the present invention. For example... Figure 3 As shown, for the initial converter model The improved model was obtained by making modifications. Because this model has the property of additiveness, the original parameters in the network structure are... , and In addition , and Then, the parameters are directly updated to , and .

[0026] The additivity property indicates that the joint effect on the output of the target model caused by pruning multiple network layers (i.e., the total output error of the target model after pruning) is equivalent to the sum of the individual effects on the output of the target model caused by pruning each network layer individually (i.e., the pruning rate of each network layer in the corresponding network structure and the amount of output error of the target model after pruning). Therefore, we have: , in, Output the total error after pruning the target model. The lower right corner indicates the layer weights and activation values ​​of the network layers from layer 1 to layer l after pruning.

[0027] S203: Based on the network structure, the constraint condition for the pruning action is determined to be that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size.

[0028] Specifically, the pruned size of the layer parameters of the i-th network layer can be expressed as: The pruned size of the activation values ​​of the i-th network layer can be expressed as: , The target size after cropping from the entire network. The constraints can then be expressed as: .

[0029] S204: Under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, find the optimal value of the pruning rate corresponding to each network layer in the network structure.

[0030] Specifically, under the premise of satisfying the constraints, with the objective of minimizing the total number of output errors after pruning, the corresponding optimization objective function to be solved can be expressed as: ; .

[0031] Furthermore, based on the relationship between the total number of pruned output errors of the target model determined in step S202 and the total number of pruned output errors, the total number of pruned output errors is decomposed using the additivity property to obtain the pruned output error corresponding to the pruning rate of each network layer's layer parameters and layer activation values, i.e.: ; .

[0032] Furthermore, the optimization objective function is decomposed into multiple linear independent terms representing the output error after pruning, becoming a linear programming problem. This significantly reduces the difficulty of solving the initial optimization objective function, which can be solved using algorithms for various linear programming problems. The solution is based on the fact that step S201 has already obtained the specific values ​​corresponding to different pruning rates for each output error after pruning. Based on these values, the rate-distortion output error curves of the layer weights and activation values ​​of each network layer corresponding to the pruning rate can be obtained.

[0033] Specifically, in some embodiments, the objective function can be solved using the Lagrange method. Therefore, step S204 may include: under the premise of satisfying constraints, with the objective of minimizing the total output error after pruning, solving for the optimal pruning rate corresponding to each network layer in the network structure using the Lagrange method. Specifically, a Lagrange operator λ∈R^+ is defined, and the pruning rate value at the point where the slope of the rate-distortion output error curve of each network layer's layer weights and activation values ​​equals λ is found to be the optimal pruning rate corresponding to that layer.

[0034] Specifically, in some embodiments, the optimization objective function can be searched for an optimal solution that satisfies the constraints using a traversal scheme such as a binary search method within a certain range. Therefore, step S204 may include: under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, solving for the optimal value of the pruning rate corresponding to each network layer in the network structure using a binary search method. Specifically, in the actual operation of the binary search method, an exponential variable can be set separately. Let the operator again ,right Solve by binary search .

[0035] In some embodiments, the objective function can also be solved using other numerical methods, including dynamic programming, divide-and-conquer algorithms, etc., without limitation.

[0036] S205: Based on the optimal pruning rate of each network layer in the network structure, prune each network layer in the network structure.

[0037] Specifically, after determining the optimal pruning rate for each network layer, the corresponding network layer is pruned using the optimal value. In some embodiments, pruning any network layer in the network structure may include: calculating the importance score of each layer's weights and activation values, and pruning the network layer based on the importance score and the optimal pruning rate corresponding to the network layer. The importance score here can be an L1 score or a first-order importance score, etc. L1 scores and first-order importance scores are known in the art and will not be described in detail here.

[0038] Figure 4 This is a schematic diagram illustrating the effect of a pruning method for a neural network model using existing technology. For example... Figure 4 As shown, the existing pruning method prunes each layer individually, only considering the output accuracy of the current layer and the impact of the current layer's output on the next layer (e.g., ...). Figure 4 In this method, the influence considered by the indicator method and the dashed circle corresponding to the clipping action color only appear after the clipping action. This method does not consider that the influence of each layer will be passed down to the last layer of the model and thus affect the result. In some cases, although the output error of the current layer is low, as the error accumulates, the error will reach a large value when it is passed to the last layer, making it impossible to guarantee the accuracy of the entire model.

[0039] Figure 5 This is a schematic diagram illustrating the effect of a neural network model pruning method provided in an embodiment of the present invention. For example... Figure 5 As shown, this embodiment of the invention considers the impact of the pruning actions of each network layer on the output of the entire target model (i.e., the amount of output error after pruning, such as...). Figure 5 The method considers the influence of the cropping action (the dotted coil corresponding to the cropping action color is passed layer by layer to the last layer of the network), and uses the additivity property to add the output influence of the cropping action of all network layers to obtain the total output after cropping. It proposes a joint cropping method that comprehensively considers the cropping output error of all layers and minimizes the total output error after cropping. It assigns a more accurate cropping rate to each network layer in the target model, which can ensure the high accuracy of the target model's output while achieving a high cropping rate.

[0040] Table 1 below provides the pruning results of the pruning method provided in the embodiments of the present invention on mainstream datasets and models.

[0041] Table 1. Pruning results on mainstream datasets and models. As shown in Table 1, the pruning method provided by the embodiments of the present invention prunes two common models, ResNet and VGG, according to a given overall pruning rate requirement, and validates the results using two mainstream datasets, CIFAR-10 and ImageNet. The accuracy and computational cost are shown in Table 1. According to the results in Table 1, the pruning method provided by the embodiments of the present invention can still maintain good accuracy when pruned to 73%, especially for the ResNet model, where the performance loss is still less than 1% while achieving a high pruning rate of 73%.

[0042] Furthermore, existing technologies propose a post-training pruning method and calibration improvements for the Vision Transformer (ViTs) model to optimize model performance. This method processes each network layer separately, ignoring the inter-layer interference. After pruning the Vision Transformer model according to the method in this embodiment, the model's accuracy and pruning rate already meet the requirements, eliminating the need for further post-training pruning and calibration improvements.

[0043] Figure 6 This is a schematic diagram of the structure of a neural network model pruning device provided in an embodiment of the present invention. Figure 6 As shown, the device may include a single-layer trimming error determination module 601, an error total determination module 602, a constraint determination module 603, an optimal planning solution module 604, and a trimming module 605.

[0044] The single-layer pruning error determination module 601 is configured to determine the pruning rate of each network layer in the corresponding network structure and the pruning error of the target model based on the network structure of the target model.

[0045] The error total determination module 602 is configured to determine the total output error of the target model after pruning based on the additivity property of the network structure. This total error is the sum of the pruning rates of each network layer in the corresponding network structure and the output error of the target model after pruning.

[0046] The constraint determination module 603 is configured to determine the constraint condition for the pruning action based on the network structure, namely that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size.

[0047] The optimal planning solution module 604 is configured to solve for the optimal pruning rate of each network layer in the network structure, with the goal of minimizing the total number of output errors after pruning, while satisfying the constraints.

[0048] The pruning module 605 is configured to prune each network layer in the network structure based on the optimal pruning rate corresponding to each network layer in the network structure.

[0049] In some embodiments, the pruning output error of the target model as the pruning rate of any network layer in the network structure changes can be: the Euclidean distance between the target model's output before pruning and the target model's output after pruning the network layer based on the pruning rate.

[0050] In some embodiments, the layer error quantity driving module 601 may be further configured to: traverse multiple given pruning rates distributed within a preset pruning rate range, and for each network layer in the network structure, determine the pruned output error quantity of the target model after the network layer is pruned according to each given pruning rate through network inference.

[0051] In some embodiments, the constraint determination module 603 may be further configured to: under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, solve for the optimal value of the pruning rate corresponding to each network layer in the network structure using the Lagrange method.

[0052] In some embodiments, the constraint determination module 603 may be further configured to: under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, solve for the optimal value of the pruning rate corresponding to each network layer in the network structure by means of a binary search method.

[0053] In some embodiments, each network layer in the network structure is a network layer that includes at least one of layer weights and layer activation values.

[0054] In some embodiments, the acquisition module 605 may be further configured to: calculate the importance score of each layer weight and each layer activation value in the network layer for each network layer in the network structure, and prune the network layer based on the optimal value of the importance score and the pruning rate corresponding to the network layer.

[0055] Figure 7 This is a schematic diagram of a neural network model pruning device provided in an embodiment of the present invention. The pruning device may include a processor 701 and a memory 702 storing computer program instructions.

[0056] Specifically, the processor 701 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0057] Memory 702 may include mass storage for data or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 702 may include removable or non-removable (or fixed) media, or memory 702 may be non-volatile solid-state memory. Memory 702 may be internal or external to the integrated gateway disaster recovery device.

[0058] In one example, memory 702 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0059] Memory 702 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0060] The processor 701 reads and executes computer program instructions stored in the memory 702 to achieve... Figure 2 The pruning method of the neural network model in the illustrated embodiment.

[0061] In one example, the trimming device may also include a communication interface 703 and a bus 704. For example, Figure 3 As shown, the processor 701, memory 702, and communication interface 703 are connected through bus 704 and complete communication with each other.

[0062] The communication interface 703 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0063] Bus 704 includes hardware, software, or both, that couples components of a trimmed device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 704 may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0064] Furthermore, in conjunction with the neural network model pruning methods described in the above embodiments, this invention can be implemented using a computer storage medium. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the neural network model pruning methods described in the above embodiments.

[0065] This application also provides a computer program product, including a computer program that, when executed, implements any of the neural network model pruning methods described in the above embodiments.

[0066] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0067] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0068] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0069] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0070] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for pruning a neural network model, the method comprising: include: Based on the network structure of the target model, determine the pruning rate of each network layer in the network structure and the pruning output error of the target model; Based on the additivity of the network structure, the total output error of the target model after pruning is determined to be the sum of the pruning rates of each network layer in the network structure and the total output error of the target model after pruning. Based on the network structure, the constraint condition for the pruning action is determined to be that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size. Under the premise of satisfying the above constraints, with the goal of minimizing the total number of output errors after pruning, the optimal value of the pruning rate corresponding to each network layer in the network structure is solved. Based on the optimal pruning rate of each network layer in the network structure, each network layer in the network structure is pruned.

2. The tailoring method of claim 1, wherein, The pruning output error of the target model as the pruning rate of any network layer in the network structure changes is: the Euclidean distance between the output result of the target model before pruning and the output result of the target model after pruning the network layer based on the pruning rate.

3. The tailoring method of claim 1, wherein, The determination of the pruning rate of each network layer in the network structure and the pruned output error of the target model includes: By iterating through multiple given pruning rates distributed within a preset pruning rate range, for each network layer in the network structure, the post-pruning output error of the target model after the network layer is pruned according to each given pruning rate is determined through network inference.

4. The tailoring method of claim 1, wherein, Under the premise of satisfying the constraints, and with the objective of minimizing the total number of output errors after pruning, the optimal value of the pruning rate corresponding to each network layer in the network structure is determined, including: Under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, the optimal value of the pruning rate corresponding to each network layer in the network structure is solved by the Lagrange method.

5. The tailoring method of claim 1, wherein, Under the premise of satisfying the constraints, and with the objective of minimizing the total number of output errors after pruning, the optimal value of the pruning rate corresponding to each network layer in the network structure is determined, including: Under the premise of satisfying the constraints, with the goal of minimizing the total number of output errors after pruning, the optimal value of the pruning rate corresponding to each network layer in the network structure is solved by binary search method.

6. The tailoring method according to any one of claims 1 to 5, characterized in that, Each network layer in the network structure is a network layer that includes at least one of layer weights and layer activation values.

7. The cutting method according to claim 6, characterized in that, The step of pruning each network layer in the network structure based on the optimal value of the pruning rate corresponding to each network layer in the network structure includes: For each network layer in the network structure, calculate the importance score of each layer weight and each layer activation value, and prune the network layer based on the optimal value of the importance score and the pruning rate corresponding to the network layer.

8. A pruning device for a neural network model, characterized in that, include: The single-layer pruning error determination module is configured to determine the pruning error of the target model after pruning based on the network structure of the target model and the pruning rate of each network layer in the network structure. The total error determination module is configured to determine, based on the additivity of the network structure, the total output error of the target model after pruning as the sum of the pruning rates of each network layer in the network structure and the total output error of the target model after pruning. The constraint determination module is configured to determine the constraint condition for the pruning action based on the network structure, namely that the sum of the pruned sizes corresponding to the pruning rates of each network layer in the network structure is not greater than the target pruned size. The optimal planning and solving module is configured to solve for the optimal pruning rate of each network layer in the network structure, with the goal of minimizing the total number of output errors after pruning, while satisfying the constraints. The pruning module is configured to prune each network layer in the network structure based on the optimal pruning rate corresponding to each network layer in the network structure.

9. A pruning device for a neural network model, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the pruning method of the neural network model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the pruning method for the neural network model as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program, which, when executed, implements the pruning method for the neural network model as described in any one of claims 1-7.

Images