Network model processing method, device, computer equipment and readable storage medium

Abstract

The present application relates to a network model processing method, device, computer equipment and readable storage medium. The method includes: determining the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; for each layer structure, determining the sparse mask according to the weight sparsity rate corresponding to the targeted layer structure; performing single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain the sparse weight elements of the targeted layer structure after single-layer pruning; performing single-layer quantization on the sparse weight elements to obtain the target weight elements corresponding to the targeted layer structure; updating the target network model according to the target weight elements corresponding to each layer structure to obtain the optimized target network model for image processing. The use of this method can reduce overhead costs while ensuring model accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a network model processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, neural network models have achieved remarkable results in various fields, including image recognition, speech recognition, and natural language processing. However, neural network models often contain a large number of redundant weight parameters, which places high demands on the model's storage space and computing resources when deployed in edge devices. This leads to high latency in hardware storage and transmission bandwidth, and slow model inference. To reduce the complexity of neural network models and improve operational efficiency, pruning techniques, which reduce computational complexity by removing redundant parameters and structures in the model, and quantization techniques, which simplify the model's computational and storage resources by replacing high-precision values ​​propagated in the model with low-precision data, have become two important model compression techniques.

[0003] However, traditional pruning and quantization techniques are decoupled during deployment, each requiring its own overhead, complicating the entire model workflow. An integrated offline design combining pruning and quantization is highly valuable for practical application deployment. Recently, some approaches have considered the joint optimization effects of quantization and pruning, but these methods require complex training and fine-tuning, consume more data and computing resources, and have significant disadvantages in deployment. Summary of the Invention

[0004] Based on this, it is necessary to provide a network model processing method, device, computer equipment, computer-readable storage medium and computer program product that can reduce overhead costs while ensuring model accuracy in response to the above technical problems.

[0005] In a first aspect, the present application provides a network model processing method, comprising:

[0006] Obtaining a target network model, and determining the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight element and the weight element of the same layer, and the weight element of the same layer and the weight element belong to the same layer structure;

[0007] For each layer structure in each layer structure, determining a sparse mask for the layer structure according to a weight sparsity rate corresponding to the layer structure;

[0008] Performing single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain sparse weight elements of the targeted layer structure after single-layer pruning;

[0009] Perform single-layer quantization on the sparse weight elements to obtain the target weight elements corresponding to the targeted layer structure;

[0010] The target network model is updated according to the target weight elements corresponding to each layer structure to obtain an optimized target network model; the optimized target network model includes an image processing model, which is used to process the input image to output the image processing result.

[0011] In a second aspect, the present application further provides a network model processing device, comprising:

[0012] A layer sparsity rate determination module is used to obtain a target network model and determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight element and the weight element of the same layer, and the weight element of the same layer and the weight element belong to the same layer structure;

[0013] A sparse mask determination module is used to determine, for each layer structure in each layer structure, a sparse mask for the layer structure according to a weight sparsity rate corresponding to the layer structure;

[0014] A single-layer pruning module is used to perform single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain the sparse weight elements of the targeted layer structure after the single-layer pruning;

[0015] A single-layer quantization module is used to perform single-layer quantization on sparse weight elements to obtain target weight elements corresponding to the target layer structure;

[0016] The model updating module is used to update the target network model according to the target weight elements corresponding to each layer structure to obtain an optimized target network model; the optimized target network model includes an image processing model, which is used to process the input image to output the image processing result.

[0017] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above network model processing method when executing the computer program.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above network model processing method when executed by a processor.

[0019] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above network model processing method when executed by a processor.

[0020] The above-mentioned network model processing method, device, computer equipment, computer-readable storage medium and computer program product determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model. The relative sensitivity parameters are determined based on the numerical relationship between the weight elements and the same-layer weight elements belonging to the same layer structure. For each layer structure, the sparse mask of the layer structure is determined according to the weight sparsity rate corresponding to the layer structure. Based on the sparse mask, the weight elements included in the layer structure are pruned in a single layer to obtain the sparse weight elements of the layer structure after single-layer pruning. The sparse weight elements are quantized in a single layer to obtain the target weight elements corresponding to the layer structure. After traversing each layer structure to obtain the corresponding target weight elements, the target network model is updated according to the target weight elements corresponding to each layer structure to obtain the optimized target network model, so that the input image can be processed by the optimized target network model. The weight sparsity rate corresponding to each layer structure is determined based on the relative sensitivity parameters of the weight elements in the target network model. The weight sparsity rate can be adaptively determined based on the numerical relationship of the weight elements in the same layer structure to ensure the reasonable distribution of the sparsity rate among the various layer structures. Then, based on the weight sparsity rate of each layer structure, single-layer pruning and single-layer quantization are alternately optimized in turn according to the level of the layer structure, realizing the logical integration of pruning and quantization processing. There is no need to train and fine-tune the model, which reduces the overhead cost of network model processing while ensuring the model accuracy of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A diagram illustrating an application environment of a network model processing method in one embodiment;

[0023] Figure 2 Schematic diagram of a flow chart of a network model processing method in one embodiment;

[0024] Figure 3 A schematic diagram of a process for determining a weight sparsity rate in one embodiment;

[0025] Figure 4 A schematic diagram of a process for deploying a network model on an edge device in one embodiment;

[0026] Figure 5 1 is a flow chart of a network model processing method in another embodiment;

[0027] Figure 6 A schematic diagram of a process for determining a weight sparsity rate for a layer structure in one embodiment;

[0028] Figure 7 A schematic diagram of a process for iteratively performing single-layer pruning and single-layer quantization processing on a layer structure in one embodiment;

[0029] Figure 8 is a structural block diagram of a network model processing device in one embodiment;

[0030] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] The network model processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104 or placed on the cloud or other network servers.

[0033] The server 104 can obtain a target network model, specifically from a data storage system. The server 104 determines the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model. The relative sensitivity parameters are determined based on the numerical relationship between the weight elements and the weight elements of the same layer structure. For each layer structure, the server 104 determines a sparse mask for the layer structure based on the weight sparsity rate corresponding to the layer structure. Based on the sparse mask, the server 104 performs single-layer pruning on the weight elements included in the layer structure to obtain the sparse weight elements of the layer structure after single-layer pruning. The server 104 performs single-layer quantization on the sparse weight elements to obtain the target weight elements corresponding to the layer structure. After traversing each layer structure to obtain the corresponding target weight elements, the server 104 updates the target network model according to the target weight elements corresponding to each layer structure to obtain an optimized target network model. The server 104 can deploy the optimized target network model to the terminal 102 so that the terminal 102 can directly apply the optimized target network model locally. For example, if the target network model is a pre-trained image recognition model, the server 104 can deploy the optimized image recognition model to the terminal 102 after pruning and quantizing the image recognition model, so that the terminal 102 can recognize the image uploaded by the user based on the optimized image recognition model.

[0034] Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0035] In an exemplary embodiment, Figure 2 As shown, a network model processing method is provided, which is applied to Figure 1 The server in FIG. 1 is taken as an example to illustrate the method, including the following steps 202 to 210. Among them:

[0036] Step 202, obtain the target network model, and determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight elements and the weight elements of the same layer, and the weight elements of the same layer belong to the same layer structure.

[0037] The target network model can be an artificial neural network model for compression optimization. Specifically, it can include models built based on various algorithms, such as at least one of a feedforward neural network (FNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), or a generative adversarial network (GAN). The target network model can include at least two layers, each of which can include different weights. Weights are parameters connecting different layers in the target network model and determine the signal transmission strength between them. Each layer can function as a neuron, each receiving weighted input from neurons in the previous layer and generating output through an activation function. Weights are core parameters of the target network model, determining how the target network model processes input data and generates corresponding outputs.

[0038] The relative sensitivity parameter is used to measure the importance of the weight element in the layer structure to which it belongs. The relative sensitivity parameter is determined based on the numerical relationship between the weight element and the weight element of the same layer. The weight element of the same layer refers to the weight element that belongs to the same layer structure as the weight element. The relative sensitivity parameter is determined based on the numerical value of the weight elements included in the same layer structure, specifically based on the numerical relationship between each weight element. The weight sparsity rate refers to the sparsity rate of the weight elements corresponding to the layer structure, that is, the proportion of the number of weight elements with a value of 0 in the layer structure. The higher the weight sparsity rate, the greater the proportion of 0, which can effectively reduce the number of model processing and thus improve the model processing efficiency.

[0039] Optionally, the server can obtain the target network model that needs to be compressed and optimized, and determine the weight elements included in each layer structure in the target network model. For each layer structure, the server can determine the relative sensitivity parameters of each weight element in the layer structure in turn. The specific server can determine the weight elements belonging to the same layer structure, and determine the relative sensitivity parameters of each weight element based on the numerical relationship of each weight element. The server can determine the weight sparsity rate of each layer structure based on the relative sensitivity parameters of each weight element. The specific server can use the relative sensitivity parameters of all weight elements in the target network model to screen all weight elements, and determine the weight sparsity rate of the layer structure based on the weight elements retained after screening. In some embodiments, when screening all weight elements, it can be ensured that each layer structure includes at least one weight element to avoid the entire layer structure being pruned.

[0040] Step 204 : For each layer structure in each layer structure, determine a sparse mask for the layer structure according to a weight sparsity rate corresponding to the layer structure.

[0041] Among them, the sparse mask is calculated based on the weight sparsity rate and is used to prune the weight elements of the layer structure. Specifically, the relatively unimportant values ​​in the weight elements can be set to zero through the relative sensitivity parameter, thereby performing pruning based on the sparse mask. Exemplarily, the server can traverse each layer structure in the target network model, that is, perform compression processing on each layer structure separately. Specifically, the server can determine the weight sparsity rate of the layer structure, and perform mask calculation based on the weight sparsity rate to determine the sparse mask of the layer structure.

[0042] Step 206 : Perform single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain sparse weight elements of the targeted layer structure after single-layer pruning.

[0043] Among them, single-layer pruning is a hierarchical pruning process for the layer structure, that is, pruning is performed on the layer structure as a whole. Single-layer pruning can be implemented step by step in the layer, so that each layer structure can be traversed to achieve independent pruning of each layer structure. Sparse weight elements are the pruning results obtained after the weight elements in the layer structure are subjected to single-layer pruning. Optionally, for the targeted layer structure, the server can perform single-layer pruning on the included weight elements based on the sparse mask of the layer structure. Specifically, single-layer pruning can be performed on the layer structure based on the alternating direction multiplication method to remove some weight elements in the layer structure to obtain sparse weight elements after single-layer pruning.

[0044] Step 208: Perform single-layer quantization on the sparse weight elements to obtain target weight elements corresponding to the targeted layer structure.

[0045] Among them, single-layer quantization is a hierarchical quantization process for the layer structure, that is, the layer structure is quantized as a whole. Single-layer quantization can be implemented step by step in the layer, so that each layer structure can be traversed to achieve independent quantization of each layer structure. The target weight element is the optimization result obtained after the original weight element in the layer structure is compressed and optimized by single-layer pruning and single-layer quantization. Optionally, the server can perform single-layer quantization on the sparse weight elements of the layer structure. Specifically, it can perform single-layer quantization on the sparse weight elements based on the alternating direction multiplication method to obtain the target weight element corresponding to the layer structure.

[0046] In step 210, the target network model is updated according to the target weight elements corresponding to each layer structure to obtain an optimized target network model; the optimized target network model includes an image processing model, which is used to process the input image to output an image processing result.

[0047] Exemplarily, the server traverses each layer structure in the target network model, and each layer structure can independently perform hierarchical pruning and hierarchical quantization processing in sequence to obtain corresponding target weight elements. The server can update the target network model through the target weight elements corresponding to each layer structure. Specifically, the weight elements in the target network model can be updated to the target weight elements corresponding to each layer structure, thereby compressing and optimizing the target network model to obtain an optimized target network model. The optimized target network model includes an image processing model, which is used to process the input image to output the image processing result. For example, the image processing model may include an image recognition model to identify the object category included in the image from the input image; for example, the image processing model may include an image classification model to perform image classification on the input image.

[0048] In the above-mentioned network model processing method, the weight sparsity rate corresponding to each layer structure is determined based on the relative sensitivity parameters of each weight element included in each layer structure in the target network model. The relative sensitivity parameters are determined based on the numerical relationship between the weight element and the same-layer weight element belonging to the same layer structure. For each layer structure, the sparse mask of the layer structure is determined according to the weight sparsity rate corresponding to the layer structure. Based on the sparse mask, the weight elements included in the layer structure are pruned in a single layer to obtain the sparse weight elements of the layer structure after single-layer pruning. The sparse weight elements are quantized in a single layer to obtain the target weight elements corresponding to the layer structure. After traversing each layer structure to obtain the corresponding target weight elements, the target network model is updated according to the target weight elements corresponding to each layer structure to obtain the optimized target network model. The weight sparsity rate corresponding to each layer structure is determined based on the relative sensitivity parameters of the weight elements in the target network model. The weight sparsity rate can be adaptively determined based on the numerical relationship of the weight elements in the same layer structure to ensure the reasonable distribution of the sparsity rate among the various layer structures. Then, based on the weight sparsity rate of each layer structure, single-layer pruning and single-layer quantization are alternately optimized in turn according to the level of the layer structure, realizing the logical integration of pruning and quantization processing. There is no need to train and fine-tune the model, which reduces the overhead cost of network model processing while ensuring the model accuracy of the network model.

[0049] In an exemplary embodiment, Figure 3 As shown, the process of determining the weight sparsity rate, that is, determining the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model, includes steps 302 to 308.

[0050] Step 302: Determine a target sparsity ratio for the entire target network model.

[0051] Among them, the target sparsity rate is the overall sparsity rate for compression optimization of the target network model, that is, the degree of compression optimization required for the target network model as a whole. Optionally, the server can determine the target sparsity rate of the target network model as a whole, and the target sparsity rate can be flexibly configured by the user according to actual needs. The target sparsity rate is the compression optimization requirement of the target network model as a whole. The target sparsity rate can be allocated to each layer structure of the target network model, that is, each layer structure can be configured with a different weighted sparsity rate so that the target network model as a whole can meet the target sparsity rate.

[0052] Step 304 : For each weight element included in each layer structure in the target network model, a relative sensitivity parameter of the weight element is calculated based on the numerical relationship between the weight element and the weight elements in the same layer.

[0053] Exemplarily, the server can calculate the corresponding relative sensitivity parameter for each weight element. Specifically, for each weight element, the server can determine the weight elements of the same layer structure as the weight element, and based on the numerical relationship between the weight element and the weight elements of the same layer, specifically based on the numerical size relationship between the weight element and the weight elements of the same layer, calculate the relative sensitivity parameter of the weight element. After traversing each weight element, the server can obtain the relative sensitivity parameter of each weight element in the target network model.

[0054] Step 306 : Filter the weight elements to obtain adaptive weight elements based on the target sparsity rate and the relative sensitivity parameters of the weight elements in the target network model.

[0055] Among them, the adaptive weight elements are weight elements that need to be retained that are screened out from the various weight elements of the target network model. Optionally, the server can perform a global screening of the weight elements in the target network model based on the target sparsity rate. Specifically, the server can screen the weight elements based on their respective relative sensitivity parameters to obtain the adaptive weight elements that are retained. In some embodiments, the server can sort the relative sensitivity parameters of the weight elements according to their numerical values, and remove the weight elements corresponding to values ​​less than the global threshold, thereby retaining the adaptive weight elements.

[0056] Step 308: Determine the weight sparsity rate corresponding to each layer structure based on the adaptive weight element.

[0057] Optionally, the adaptive weight elements retained after screening may come from different layer structures, and the number of weight elements retained in different layer structures may be different. The server may determine the weight sparsity rate of each layer structure based on the adaptive weight elements. Specifically, the server may determine the layer structure to which each adaptive weight element belongs, and count the number of adaptive weight elements retained in each layer structure, thereby calculating the corresponding weight sparsity rate based on the number of adaptive weight elements.

[0058] In this embodiment, the server screens out adaptive weight elements from each weight element based on the overall target sparsity of the target network model and the relative sensitivity parameters of each weight element, and determines the weight sparsity of each layer structure based on the adaptive weight elements. The server can adaptively calculate the non-uniform sparsity for each layer structure of the target network model, alleviate the problem of accuracy loss caused by unreasonable sparsity rate distribution, ensure the rationality of sparsity rate distribution, and thus ensure the model accuracy of the network model.

[0059] In an exemplary embodiment, the sparse mask of the targeted layer structure is determined based on the weight sparsity rate corresponding to the targeted layer structure, including: determining the number of pruning iterations for the targeted layer structure; when the number of pruning iterations does not meet the pruning end condition, based on the weight sparsity rate corresponding to the targeted layer structure, updating the pre-pruning weight elements of the targeted layer structure for this pruning iteration to obtain the sparse mask of the targeted layer structure for this pruning iteration.

[0060] The pruning iteration count is the number of times the weight elements of the layer structure have been pruned. The pruning end condition is used to determine whether to continue pruning the weight elements of the layer structure. Specifically, it may include a pruning iteration threshold. When the pruning iteration count exceeds the pruning iteration threshold, the pruning end condition is considered to be satisfied. The current pruning iteration refers to the current pruning iteration, and the pre-pruning weight element refers to the initial weight element of the layer structure before the current pruning iteration.

[0061] Exemplarily, when determining the sparse mask of each layer structure for single-layer pruning, multiple pruning iterations can be performed on the weight elements of the layer structure, and the sparse mask needs to be determined in each pruning iteration. The server can determine the number of pruning iterations for the pruning iteration process. When the number of pruning iterations does not meet the pruning end condition, it indicates that the pruning iteration process has not ended. The server can determine the pre-pruning weight element of this pruning iteration and update the pre-pruning weight element of this pruning iteration based on the weight sparsity rate of the layer structure. Specifically, it can be mapped based on the pre-pruning weight element to obtain the sparse mask of this pruning iteration. If this pruning iteration is the kth pruning iteration process, the pre-pruning weight element can be , the sparse mask M of this pruning iteration can be obtained.

[0062] Furthermore, a single-layer pruning is performed on the weight elements included in the targeted layer structure based on the sparse mask to obtain the sparse weight elements of the targeted layer structure after the single-layer pruning, including: based on the sparse mask of the current pruning iteration of the targeted layer structure, a single-layer pruning is performed on the weight elements included in the targeted layer structure in this pruning iteration to obtain the pruned weight elements of the current pruning iteration; the number of pruning iterations is updated, and when the updated number of pruning iterations does not meet the pruning end condition, the pruned weight elements of the current pruning iteration are used as the pre-pruning weight elements of the next pruning iteration, and the weight sparsity rate corresponding to the targeted layer structure is returned, and the pre-pruning weight elements of the current pruning iteration of the targeted layer structure are updated to obtain the sparse mask of the current pruning iteration of the targeted layer structure, so as to perform the next pruning iteration; when the updated number of pruning iterations meets the pruning end condition, the sparse weight elements of the targeted layer structure after the single-layer pruning are obtained according to the pruned weight elements of the last pruning iteration.

[0063] For example, during this pruning iteration, the server can perform single-layer pruning on the weight elements included in the layer structure based on the sparse mask of this pruning iteration. Specifically, the pruning problem of the layer weight can be regarded as a problem of minimizing the loss between the layer outputs before and after pruning. Based on the alternating direction multiplier method, single-layer pruning is performed on the weight elements included in the layer structure to obtain the pruned weight elements of this pruning iteration. For example, if this pruning iteration is the kth pruning iteration, the weight elements before pruning can be , then the pruned weight element of this pruning iteration can be .

[0064] Furthermore, the server can update the number of pruning iterations, such as adding 1 to the number of pruning iterations to obtain an updated number of pruning iterations. The server can determine whether the updated number of pruning iterations meets the pruning end condition. When the updated number of pruning iterations still does not meet the condition, the next pruning iteration can be performed on the layer structure. The specific server can use the pruned weight element of the pruning iteration as the pre-pruning weight element of the next pruning iteration, and return the weight sparsity rate corresponding to the targeted layer structure, update the pre-pruning weight element of the current pruning iteration of the targeted layer structure, and obtain the sparse mask processing of the current pruning iteration of the targeted layer structure, so that the pruned weight element of the current pruning iteration can be used to perform the next pruning iteration, thereby realizing iterative pruning processing for the layer structure.

[0065] Furthermore, when the updated number of pruning iterations meets the pruning end condition, it indicates that the pruning iteration processing can be ended, and the server can obtain the pruned weight element of the last pruning iteration, that is, the pruned weight element of the last pruning iteration can be obtained, and based on the pruned weight element of the last pruning iteration, the sparse weight element of the layer structure after single-layer pruning is obtained, thereby realizing single-layer pruning processing for the layer structure.

[0066] In this embodiment, the server performs multiple pruning iterations on the layer structure, and the sparse mask can be updated in each pruning iteration, so as to obtain sparse weight elements that meet the pruning requirements and ensure the model accuracy of the network model.

[0067] In an exemplary embodiment, based on the sparse mask of the current pruning iteration of the targeted layer structure, single-layer pruning is performed on the weight elements included in the targeted layer structure of this pruning iteration to obtain the pruned weight elements of this pruning iteration, including: determining the pruning optimization parameter and pruning multiplier parameter of this pruning iteration by the alternating direction multiplier method; performing single-layer pruning calculation based on the sparse mask, pruning optimization parameter and pruning multiplier parameter of the current pruning iteration of the targeted layer structure to obtain the pruned weight elements of this pruning iteration; updating the pruning optimization parameter based on the pruned weight element, pruning multiplier parameter and sparse mask to obtain the pruning optimization parameter of the next pruning iteration; obtaining the pruning multiplier parameter of the next pruning iteration based on the pruned weight element, the pruning optimization parameter and pruning multiplier parameter of the next pruning iteration.

[0068] Among them, the alternating direction multiplier method is usually used in mathematics to solve the optimization problem of convex functions. Assuming that the objective function has two variables, but it is difficult to optimize them at the same time, it may be simpler to fix one of the variables and solve the minimization problem of the other variable. Therefore, it is usually considered to alternately minimize the two variables. In this embodiment, by splitting the network model into independent layer structures, the pruning or quantization of each layer structure can also be processed by the alternating direction multiplier method. The pruning optimization parameter and the pruning multiplier parameter are intermediate parameters in the pruning iterative process determined based on the alternating direction multiplier method, which are used to optimize the weight elements.

[0069] For example, the server may determine the pruning optimization parameter for this pruning iteration based on the alternating direction multiplier method. and the pruning multiplier parameter , the server can optimize the amount of pruning parameters based on , pruning multiplier parameters And the sparse mask of this pruning iteration performs single-layer pruning calculation to calculate the pruned weight elements of this pruning iteration. The server can calculate the pruned weight elements and pruning multiplier parameters based on the pruned weight elements and pruning multiplier parameters. and the sparse mask pair pruning optimization parameters of this pruning iteration The server can further update the pruning multiplier parameter based on the pruned weight element, the pruning optimization parameter of the next pruning iteration, and the sparse mask of this pruning iteration. The pruning optimization amount parameters of the next pruning iteration are updated to obtain the pruning optimization amount parameters of the next pruning iteration, and the pruning optimization amount parameters of the next pruning iteration are used for the single-layer pruning calculation in the next pruning iteration processing.

[0070] Furthermore, when the updated number of pruning iterations meets the pruning end condition, the sparse weight elements of the targeted layer structure after single-layer pruning are obtained based on the pruned weight elements of the last pruning iteration, including: when the updated number of pruning iterations meets the pruning end condition, the sparse weight elements of the targeted layer structure after single-layer pruning are calculated based on the pruned weight elements of the last pruning iteration, the pruning multiplier parameters of the last pruning iteration, and the sparse mask of the last pruning iteration.

[0071] Optionally, when the updated number of pruning iterations meets the pruning end condition, indicating that the pruning iteration process needs to be ended, the server can obtain the last pruning iteration parameters in the last pruning iteration process, specifically including the pruned weight elements, pruning multiplier parameters and sparse masks. The server can calculate the sparse weight elements of the layer structure after single-layer pruning based on the last pruning iteration parameters.

[0072] In this embodiment, the server determines the pruning optimization amount parameter and the pruning multiplier parameter based on the alternating direction multiplier method to perform multiple pruning iterative processes on the layer structure, thereby obtaining sparse weight elements that meet the pruning requirements and ensuring the model accuracy of the network model.

[0073] In an exemplary embodiment, a single-layer quantization is performed on the sparse weight element to obtain the target weight element corresponding to the targeted layer structure, including: determining the number of outer layer quantization iterations for the targeted layer structure; when the number of outer layer quantization iterations does not meet the outer layer iteration end condition, determining the quantization optimization parameter and quantization multiplier parameter of this outer layer quantization iteration by the alternating direction multiplier method; performing inner layer quantization iteration based on the quantization multiplier parameter and the initial sparse weight element of this outer layer quantization iteration to obtain the inner layer quantization iteration variable; performing inner layer quantization iteration based on the inner layer quantization iteration variable, quantization multiplier parameter, quantization optimization parameter and the initial sparse weight element Quantization iteration calculation is performed to obtain the quantization weight of this outer quantization iteration; the number of outer quantization iterations is updated. When the updated number of outer quantization iterations does not meet the outer iteration end condition, the quantization weight of this outer quantization iteration is used as the initial sparse weight element of the next outer quantization iteration, and the step of determining the quantization optimization parameter and quantization multiplier parameter of this outer quantization iteration by the alternating direction multiplication method is returned to perform the next outer quantization iteration; when the updated number of outer quantization iterations meets the outer iteration end condition, the target weight element corresponding to the targeted layer structure is obtained according to the quantization weight of the last outer quantization iteration.

[0074] The outer quantization iteration count is the number of times outer quantization iterations have been performed on the weight elements of the layer structure. The outer iteration termination condition is used to determine whether to continue outer quantization iterations on the weight elements of the layer structure. Specifically, it may include an outer quantization iteration count threshold. When the outer quantization iteration count exceeds the outer quantization iteration count threshold, the outer iteration termination condition is considered to be satisfied. The quantization optimization parameter and the quantization multiplier parameter are intermediate parameters in the outer quantization iteration process determined based on the alternating direction multiplier method and are used to optimize the weight elements.

[0075] Optionally, after completing the single-layer pruning for the weight elements of the layer structure, a single-layer quantization process can be further performed, specifically, multiple outer layer quantization iterations can be performed. The server can determine the number of outer layer quantization iterations for the layer structure, and when the number of outer layer quantization iterations does not meet the outer layer iteration end condition, it indicates that the outer layer quantization iteration process needs to be continued. The server can determine the quantization optimization parameter of this outer layer quantization iteration by the alternating direction multiplication method. and quantized multiplier parameters The server can be based on the quantization multiplier parameters of this outer quantization iteration The initial sparse weight elements of this outer quantization iteration are subjected to inner quantization iteration processing, thereby obtaining inner quantization iteration variables. The inner quantization iteration processing may also include multiple iteration processing based on the alternating direction multiplication method. The initial sparse weight elements are the weight elements of the layer structure before the outer quantization iteration is processed. When the outer quantization iteration is the first outer quantization iteration processing, the initial sparse weight elements may be the sparse elements obtained by single-layer pruning; when the outer quantization iteration is the nth (n>1) outer quantization iteration processing, the initial sparse weight elements may be the quantization weights obtained by the previous outer quantization iteration processing.

[0076] Furthermore, the server can update the number of outer layer quantization iterations. When the updated number of outer layer quantization iterations does not meet the outer layer iteration end condition, it indicates that the outer layer quantization iteration process needs to continue. The server can use the quantization weight of this outer layer quantization iteration as the initial sparse weight element of the next outer layer quantization iteration, and return the process of determining the quantization optimization parameter and quantization multiplier parameter of this outer layer quantization iteration through the alternating direction multiplier method, thereby performing the next outer layer quantization iteration process. When the updated number of outer layer quantization iterations meets the outer layer iteration end condition, it indicates that the outer layer quantization iteration process can be ended. The server can determine the quantization weight of the last outer layer quantization iteration, that is, it can determine the quantization weight obtained by the last outer layer quantization iteration process, and obtain the target weight element corresponding to the layer structure based on the quantization weight obtained by the last outer layer quantization iteration process, thereby realizing single-layer quantization processing for the layer structure.

[0077] In this embodiment, the server determines the quantization optimization parameter and the quantization multiplier parameter through the alternating direction multiplier method, and performs multiple outer-layer quantization iterative processing based on the quantization optimization parameter and the quantization multiplier parameter. The quantization weights that meet the quantization requirements can be obtained through the outer-layer quantization iterative processing, thereby ensuring the model accuracy of the network model.

[0078] In an exemplary embodiment, an inner quantization iteration is performed based on the quantization multiplier parameters and initial sparse weight elements of this outer quantization iteration to obtain an inner quantization iteration variable, including: determining the number of inner quantization iterations corresponding to this outer quantization iteration; when the number of inner quantization iterations does not meet the inner iteration end condition, determining the pre-iteration variable of this inner quantization iteration; calculating the post-iteration variable of this inner quantization iteration based on the initial inner quantization iteration variable, the quantization multiplier parameters and initial sparse weight elements of this outer quantization iteration; updating the number of inner quantization iterations, when the updated number of inner quantization iterations does not meet the inner iteration end condition, using the post-iteration variable of this inner quantization iteration as the pre-iteration variable of the next inner quantization iteration, returning to the step of determining the pre-iteration variable of this inner quantization iteration to perform the next inner quantization iteration; when the updated number of inner quantization iterations meets the inner iteration end condition, obtaining the inner quantization iteration variable according to the post-iteration variable of the last inner quantization iteration.

[0079] Among them, the number of inner-layer quantization iterations is the number of times that inner-layer quantization iteration processing has been performed on the weight elements of the layer structure. The inner-layer iteration end condition is used to determine whether it is necessary to continue the inner-layer quantization iteration processing on the weight elements of the layer structure. Specifically, it may include an inner-layer quantization iteration number threshold. When the number of inner-layer quantization iterations exceeds the inner-layer quantization iteration number threshold, it can be considered that the inner-layer iteration end condition is met.

[0080] Exemplarily, each outer quantization iteration also includes multiple inner quantization iterations. When performing quantization iteration calculations based on the inner quantization iteration variable, quantization multiplier parameter, quantization optimization parameter, and initial sparse weight element, the server can determine the number of inner quantization iterations corresponding to this outer quantization iteration. If the number of inner quantization iterations does not meet the inner iteration termination condition, it indicates that inner quantization iteration processing is required. The server can determine the pre-iteration variable for this inner quantization iteration and calculate based on the initial inner quantization iteration variable, the quantization multiplier parameter for this outer quantization iteration, and the initial sparse weight element to obtain the post-iteration variable for this inner quantization iteration. The server can update the number of inner quantization iterations. If the updated number of inner quantization iterations does not meet the inner iteration termination condition, it indicates that the next inner quantization iteration processing is required. The server can then use the post-iteration variable of this inner quantization iteration as the pre-iteration variable for the next inner quantization iteration and return to the process of determining the pre-iteration variable for this inner quantization iteration to proceed to the next inner quantization iteration.

[0081] Furthermore, when the updated number of inner quantization iterations meets the inner iteration end condition, indicating that the inner quantization iteration processing can be ended, the server can determine the post-iteration variable of the last inner quantization iteration, specifically, it can determine the post-iteration variable of the last inner quantization iteration processing, and obtain the inner quantization iteration variable based on the post-iteration variable of the last inner quantization iteration processing, so as to perform the quantization iteration calculation in the outer quantization iteration processing based on the inner quantization iteration variable.

[0082] In this embodiment, in each outer-layer quantization iteration process, the server performs multiple inner-layer quantization iteration processes to ensure the accuracy of the variables after iteration, thereby performing quantization iterative calculations in the outer-layer quantization iteration process to obtain quantization weights that meet quantization requirements and ensure the model accuracy of the network model.

[0083] This application also provides an application scenario, which applies the above-mentioned network model processing method. Specifically, the application of the network model processing method in this application scenario is as follows:

[0084] In the field of computer vision, existing neural network pruning and quantization technologies have the following problems: ① For weight pruning, existing technologies fail to properly allocate layer sparsity, resulting in severe loss of model accuracy and the need for post-pruning training and fine-tuning. ② For model compression and deployment, traditional approaches often consider designing quantization and pruning modules independently, which complicates the model compression workflow and increases logic design costs. ③ Currently, most pruning and quantization algorithms are designed online, requiring complete training data and backpropagation processes, placing higher demands on computing resources and making them less suitable for model deployment.

[0085] The network model processing method provided in this embodiment relates to an integrated offline framework for weight pruning and quantization based on an alternating optimization strategy: a reasonable weight sensitivity score is proposed to adaptively calculate the sparsity rate of the layer, and an optimization strategy based on the alternating direction multiplier method is used to serially perform pruning and quantization on the weights of each layer of the model, so that good model accuracy can be guaranteed without training and fine-tuning. The network model processing method provided in this embodiment belongs to an offline framework, which only uses a small amount of unlabeled calibration sets to obtain the input of the model offline, which is sufficient for the pruning and quantization process. Moreover, the pruning and quantization of weights are logically designed as one, which reduces the overhead of independent design of training resources and modules during deployment.

[0086] In the field of computer vision, on the one hand, neural network models usually have a large number of redundant weight parameters, which leads to high requirements for the storage space and computing resources of the model when it is actually deployed in edge devices, resulting in high hardware storage and transmission bandwidth delays and slow model inference. As one of the model deployment compression technologies, model pruning is of great significance in reducing model bandwidth requirements and accelerating inference. Existing pruning schemes are generally divided into unstructured pruning (i.e. weight pruning) and structured pruning. The weight pruning scheme evaluates the importance of each weight and prunes the less important elements. This type of method has problems such as unreasonable sparsity rate distribution leading to accuracy loss and expensive training costs to compensate for the accuracy effect; the structured pruning scheme prunes redundant structures according to the importance of neurons or channels, which has a more serious impact on the expressiveness of the model and still relies on training fine-tuning to alleviate the loss of accuracy. Offline pruning schemes that get rid of training have great advantages in saving resources and bandwidth in model deployment. Such as Figure 4 As shown in the figure, in the general process of model deployment and inference on edge devices, after the model structure design is completed, the model needs to be streamlined, including pruning of the pruning module and quantization of the quantization module, so as to perform compression acceleration based on the algorithm layer. After the model file format is converted and optimized through tools such as compilers, it is deployed to the edge device for inference operation.

[0087] On the other hand, quantization is essential for model deployment and accelerated inference. Many quantization schemes still rely on training and computing resources, so maintaining efficient, fast, and precision-friendly quantization is crucial. Traditionally, the pruning and quantization modules are decoupled during deployment, each requiring its own overhead, complicating the entire model workflow. An integrated offline design combining pruning and quantization is highly valuable for practical application deployment. Recently, some approaches have considered the joint optimization effects of quantization and pruning, but these require complex training and fine-tuning, consume more data and computing resources, and have significant disadvantages in deployment.

[0088] In traditional technologies, bandwidth becomes a bottleneck during deployment due to redundant neural network parameters, leading to the proposal of various weight pruning algorithms. However, existing pruning algorithms typically employ unstructured pruning based on the absolute value of weights. Most methods employ uniform sparsity across layers or simply use a global threshold to achieve non-uniform sparsity, without fully considering whether the sparsity distribution across layers is appropriate. This single pruning operation, combined with an irrational sparsity distribution, can easily lead to significant loss of model accuracy, often requiring large amounts of data and lengthy training and fine-tuning to restore some accuracy. Furthermore, due to differences in the underlying principles of pruning and quantization, existing technologies often decouple pruning and quantization during deployment, complicating the model's inference process and increasing the design overhead of logically independent modules. Furthermore, due to their reliance on online training, existing technologies place higher demands on training data and computing resources, hindering scalability.

[0089] Based on this, the network model processing method provided in this embodiment proposes an effective weight sensitivity evaluation score for adaptively calculating the sparsity rate of each layer, alleviating the challenge of accuracy loss caused by unreasonable sparsity rate distribution; moreover, the network model processing method provided in this embodiment uses an optimization strategy based on the alternating direction multiplier method to unify the ideas of pruning and quantization, and integrates pruning and quantization into an integrated design to achieve fast layer-level weight sparsity plus quantization, without the need for costly training and fine-tuning to ensure good model accuracy, and more importantly, it is conducive to saving design overhead during deployment. In addition, the offline framework of the network model processing method provided in this embodiment does not rely on the back propagation of the neural network model, which reduces resource costs and enhances practicality and scalability.

[0090] Specifically, the network model processing method provided in this embodiment can be applied to the field of computer vision. It involves an integrated offline framework for weight pruning and quantization based on an alternating optimization strategy, which can be directly applied to the sparsity and quantization of onnx (Open NeuralNetwork Exchange) models. First, the sparsity rate of each layer is adaptively calculated based on the proposed relative sensitivity score (RSS) of the weights. Then, the optimization strategy based on the alternating direction multiplier method is used to efficiently implement layer-level weight pruning and quantization. There is no need to spend training cost fine-tuning to achieve the accuracy effect that meets the application expectations. Figure 5As shown, for the pre-trained model onnx, the weights of each layer (i.e., layer structure) are decoupled and the non-uniform sparsity rate is adaptively calculated based on RSS, thereby decoupling the sparsity rate of each layer. Furthermore, each layer structure is traversed, and the layers are pruned and quantized independently. Specifically, an integrated offline framework based on an alternating optimization strategy performs weight pruning and quantization in series. After obtaining the quantized weight nodes of each layer, the model is updated, resulting in the compressed and optimized weight quantization model onnx.

[0091] The process of adaptively calculating the non-uniform sparsity rate may include:

[0092] (1) For each pre-trained neural network, the number of weights to be pruned is obtained given the target sparsity ratio of the entire network.

[0093] (2) The proposed weight sensitivity evaluation score is used to process the weight elements of each layer, and the relative sensitivity score (RSS) of each weight value in its layer is obtained.

[0094] The weighted relative sensitivity assessment score is expressed as:

[0095]

[0096] in, and are all weight matrices of a certain layer The element value in , subscript and represents an index sorted by the square of its value, express The absolute value of the index weight is greater than The weight of the index. The numerator of one of the multipliers in the formula is the weight element The absolute value of the denominator is sorted by absolute value. The sum of all other weight elements before, according to the order of magnitude in pruning, the elements with smaller values ​​will be more likely to be pruned, so this ratio measures the importance score of a weight element compared to other surviving weight elements in the layer. Another multiplier Input activation for this layer The L2 norm of the input activation is also of reference value in determining the neuron output, but in order to prevent the activation value from being too large, a balancing factor is used. Therefore, RSS takes into account the impact of weights and activations on the layer output and can be regarded as the relative sensitivity of a weight element in the current layer. The smaller the score, the less important the corresponding weight value.

[0097] (3) Then, the global threshold of the relative sensitivity score RSS is obtained according to the number of weights to be sparse in the entire network. The RSS of the entire network is sorted, and the corresponding weights with RSS smaller than the global threshold will be pruned, thereby realizing the adaptive calculation of non-uniform sparsity rates of each layer.

[0098] In extreme cases, the relative sensitivity score proposed in this invention ensures that at least one weight value survives in each layer of the neural network (when the ratio of the left half of the RSS formula is equal to 1), thereby avoiding the risk of the entire layer being pruned and making the sparsity distribution of each layer more reasonable.

[0099] Specific as Figure 6 As shown in the figure, after flattening and sorting the weight W of a certain layer and the weight U of a certain layer, the relative sensitivity scores of each weight element are calculated respectively, and the relative sensitivity scores are globally sorted, and the sparsity rate of each layer is adaptively calculated to obtain the weight sparsity rate of each layer.

[0100] Furthermore, for the integrated pruning and quantization framework based on the alternating optimization strategy, under the assumption that the weight matrices of each layer of the neural network are independent of each other, the weights of each layer are operated independently, and the pruning and quantization process is performed serially on each layer of weights. This integrated framework aims to jointly achieve layer-level weight sparsity and quantization. The present invention divides the different states of each layer weight during the pruning and quantization process into three types: "initial state, update state, and completed state." The three states and their descriptions are as follows:

[0101] (1) Initial state: The weight state of this layer is the weight of the pre-trained model, and the relevant variables are initialized using the model input obtained from a small number of calibration sets.

[0102] (2) Update state: The weight then enters the "update state", and the weight state of this layer is further divided into the pruned state and the quantized state, with pruning first and then quantization in a serial manner. This scheme adopts the iterative update idea of ​​the alternating direction multiplier method. For the pruning state: each iteration first calculates a more accurate sparse mask based on the proposed mask evolution strategy, and then uses the alternating direction multiplier method to quickly and efficiently update the remaining weights of the layer to ensure that the gap between the weight of the layer after the pruning operation and the original weight is minimized, thereby minimizing the accuracy loss of the entire neural network. For the quantized state: the weight after pruning is further quantized based on the update idea of ​​the alternating direction multiplier method of the inner and outer layers. Although both layers of loops involve optimization variables, the speed is faster.

[0103] (3) Completed state: The weight layer that has completed the "update state" becomes the "completed state", and the pruning and quantization are completed.

[0104] Specifically, regarding the optimization strategy based on the alternating direction multiplier method, which is often used mathematically to solve convex function optimization problems, suppose the objective function has two variables, but it is difficult to optimize them simultaneously. Fixing one variable and solving the minimization problem with respect to the other variable may be simpler. Therefore, it is often considered to minimize the two variables alternately. Splitting the network into independent layers and pruning or quantizing each layer can actually be considered this type of optimization problem. This embodiment simply refers to it as the alternating optimization strategy.

[0105] In the layer pruning process, the pruning problem of layer weights is first regarded as a problem of minimizing the loss between the layer outputs before and after pruning, which is formalized as:

[0106]

[0107] in, Here, it represents the weight of this layer after final pruning. represents the input of this layer, represents the sparse mask of weights, For the XOR operation, is the L2 norm.

[0108] The objective function is expressed as , express the constraints as indicator functions:

[0109]

[0110] The pruning problem is further formalized as follows, which is converted into a constrained optimization problem that can be solved by the alternating direction multiplier method:

[0111]

[0112] According to the analysis of this type of optimization problem using the alternating direction multiplier method, the Lagrange multiplier is introduced. , the following steps will be used to iteratively resolve the variables involved in the above optimization problem :

[0113]

[0114] Where k is the current iteration number, It is a penalty parameter, the default value is 1; Represents the Euclidean projection on the constraint set C, which is determined based on the range of weights. In the pruning weight update problem, this solution uses a mask to calculate the projection and The update of is similar to the main variable update when solving the Lasso problem using the alternating direction multiplier method, so and The optimization can be further simplified. The update of each variable is explicitly expressed as follows:

[0115]

[0116] in, is the identity matrix, and These are all known quantities that can be directly calculated.

[0117] Furthermore, in the hierarchical quantization process, that is, the quantization of the hierarchical weights, it can also be regarded as a problem of minimizing the loss between the hierarchical outputs before and after quantization, which can be formalized as follows:

[0118]

[0119] Similarly, introduce auxiliary variables , In satisfying the constraints It takes 0 when , otherwise it takes infinity. is the scaling factor ( ), is the number of bits. The problem is further formalized as follows:

[0120]

[0121] Introducing Lagrange multipliers , using the alternating direction multiplier method to solve:

[0122]

[0123] The update of the above three variables is located in the outer loop of the iterative process. The key to updating the quantized weights is to analyze , an inner loop needs to be established to solve it.

[0124] make equal , then in the set The Euclidean projection on can be expressed as:

[0125]

[0126] The variable Separate from the constraints and let ,have to:

[0127]

[0128] The minimum value problem can be further solved locally using the alternating direction multiplication method (the following formula represents the number of iterations of the inner loop), and obtains the value of the variable The analytical solution of :

[0129]

[0130] in, and Quantize the iteration variable for the inner layer.

[0131] For the integrated solution of hierarchical weight pruning and quantization, the pruning process is based on mask evolution and alternating direction multiplier method. According to the principle, the entire iterative process of alternating direction multiplier method involves three optimization quantities: Each iteration requires mask calculation, and the weight sparse mask directly affects the update of the aforementioned variables. This scheme proposes to obtain a more accurate sparse mask by setting the weight relative sensitivity score (RSS) to zero for relatively unimportant values. The resulting mask gradually improves with each iteration, and this process is considered a mask evolution strategy.

[0132] More specific steps are: ① Determine the initial value of the variable: Lagrange multiplier is an all-zero variable of the same size as the weight matrix, and the total number of iterations , calculated based on the input activation related variables ② Enter the iterative loop, execute mask evolution, and update variables: On mask selection, calculate The weighted relative sensitivity score RSS is sorted and the position with smaller score is set to zero so that the sparsity reaches the sparse rate of the layer based on RSS, thus obtaining the sparse mask of the weight ; Then update the optimization amount according to the iterative update formula. ③ After the number of iterations is completed, use the mask to update the optimization amount again , as the weight after level sparseness, and end the update.

[0133] For the quantization state process based on alternating updates of inner and outer layers, it can be seen from the principle that this solution uses two layers of loops to quantize and optimize variables. For the outer loop, the optimization variables are involved. , is the sparse weight after pruning. The number of outer iterations is the same as that of pruning state. ,in and Can be updated directly based on known quantities and iterative processes, variables It is necessary to further use the inner loop variable solution update. For the inner loop, the optimization quantity , given the number of local iterations , first initialize the layer using the statistical maximum and minimum weights , and then solve the quantization weight in this case according to the update formula , and then obtain The internal and external parameters are updated alternately in a loop to efficiently quantize the sparse weights of this layer.

[0134] like Figure 7 As shown in the figure, in the process of single-layer pruning and single-layer quantization processing for the layer structure, the original weights of the layer structure go through the initial state, update state and completion state respectively, thereby obtaining the weights after layer pruning and quantization. Among them, in the update state processing process, the initial value of each variable of the layer structure can be determined, and the pruning state processing flow can be realized based on the mask evolution and alternating direction multiplier method. Specifically, the iterative processing is determined by the number of iterations K. When the number of iterations K is not reached, the sparse mask M can be determined based on the mask evolution strategy, and the number of iterations is returned after further updating each variable. After the iteration is completed, the single-layer pruning is completed, and the optimization quantity is finally updated using the sparse mask. , as the weight after hierarchical sparsification. In the quantized state process based on alternating updates of inner and outer layers, the weight obtained based on single-layer pruning is used as the initial weight, and the iterative processing is determined by the outer layer iteration number K and the inner layer iteration number J. In the inner layer iteration process, the inner layer quantized iteration variables are updated. and , update each variable in the outer layer iterative processing, and obtain the hierarchical pruned quantized weight after the single layer quantization is completed, that is, obtain the target weight element corresponding to the layer structure.

[0135] Regarding the strategy for non-uniform sparsity rate distribution, traditional technologies generally use global distribution based on absolute value size, that is, after using the target sparsity rate of the entire network to determine the global threshold, the value with the smaller absolute value of the weight is directly set to zero to achieve non-uniform sparsity in each layer. However, this simple and crude method has problems: ① The sparse rate distribution is unreasonable, which may easily lead to some layers being set to zero, and directly transform unstructured pruning into structured pruning. The rationality of the sparse rate distribution is not as good as the technical solution of the present invention; ② After the pruning operation, the model accuracy is seriously lost, and it usually takes longer to train and fine-tune.

[0136] In some embodiments, the hierarchical weight pruning and updating technology can also be implemented based on the Optimal Brain Compression (OBC) and SparseGPT methods. Both OBC and SparseGPT use the weight importance formula and weight compensation formula based on the Hessian matrix to independently prune and update the weights of each layer of the neural network. Specifically,

[0137] The weight importance formula is used to determine the position of the cut weight as follows:

[0138]

[0139] in, is the p-th weight, and H represents the Hessian matrix of the loss of a given (dense) model. is the p-th diagonal entry of the inverse Hessian matrix.

[0140] The formula for updating the remaining weight according to the weight compensation formula is as follows:

[0141]

[0142] in, is the compensation value for the remaining weight, is the p-th column of the inverse Hessian matrix.

[0143] Hierarchical quantization can also be implemented based on the Optimal Brain Quantization (OBQ) and GPTQ methods. Similar to pruning, both OBQ and GPTQ independently quantize and update weights at each layer based on the Hessian matrix weight importance formula and weight compensation formula. The formula is as follows:

[0144]

[0145] Among them, quant(·) is the quantitative processing, is the pth weight, H F The Hessian matrix representing the loss of a given (dense) model, is the inverse matrix H F The pth diagonal item of ; is the compensation value for the remaining weight, is the inverse matrix H F The qth column of .

[0146] The network model processing method provided in this embodiment proposes a weight relative sensitivity assessment score that more accurately measures the importance of weight elements. This score is used to adaptively calculate the appropriate sparsity rate for each layer, alleviating the loss of pruning accuracy caused by improper sparsity rate distribution. Furthermore, this embodiment proposes an optimization strategy based on the alternating direction multiplier method that unifies the concepts of pruning and quantization, designing an efficient integrated pruning and quantization framework. This allows for offline, serial execution of layer-level pruning and quantization processes, eliminating the need for training and fine-tuning, achieving accuracy that meets application expectations. This not only reduces the demand for training resources but also reduces deployment and design overhead.

[0147] The network model processing method provided in this embodiment alleviates the challenge of unreasonable sparsity rate distribution of each layer in model pruning, realizes efficient weight pruning and quantization of visual neural networks, and ensures good accuracy. Moreover, the network model processing method provided in this embodiment is an offline technology and does not involve the training and back propagation of neural networks, which reduces the demand for training resources and enhances practicality and scalability. In addition, the network model processing method provided in this embodiment logically integrates model pruning and quantization into an integrated design, which is beneficial to reducing deployment design overhead. Moreover, the sparse quantization model of the network model processing method provided in this embodiment effectively reduces the model storage and transmission bandwidth, which is beneficial to inference acceleration.

[0148] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiments of the present application also provide a network model processing device for implementing the network model processing method involved above. The implementation solution provided by the device is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more network model processing device embodiments provided below can be referred to the limitations of the network model processing method above, and will not be repeated here.

[0150] In an exemplary embodiment, Figure 8 As shown, a network model processing device 800 is provided, comprising: a layer sparsity rate determination module 802, a sparse mask determination module 804, a single layer pruning module 806, a single layer quantization module 808 and a model updating module 810, wherein:

[0151] The layer sparsity rate determination module 802 is used to obtain a target network model and determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight element and the weight elements of the same layer, where the weight elements of the same layer belong to the same layer structure;

[0152] The sparse mask determination module 804 is configured to determine, for each layer structure in each layer structure, a sparse mask for the layer structure according to a weight sparsity rate corresponding to the layer structure;

[0153] A single-layer pruning module 806 is configured to perform single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain sparse weight elements of the targeted layer structure after the single-layer pruning;

[0154] A single-layer quantization module 808 is configured to perform single-layer quantization on the sparse weight elements to obtain target weight elements corresponding to the target layer structure;

[0155] The model updating module 810 is used to update the target network model according to the target weight elements corresponding to each layer structure to obtain an optimized target network model; the optimized target network model includes an image processing model, which is used to process the input image to output the image processing result.

[0156] In some embodiments, the layer sparsity rate determination module 802 is also used to determine the target sparsity rate for the target network model as a whole; for each weight element in the weight elements included in each layer structure in the target network model, based on the numerical relationship between the targeted weight element and the weight elements of the same layer, the relative sensitivity parameter of the targeted weight element is calculated; according to the target sparsity rate and the relative sensitivity parameters of each weight element in the target network model, adaptive weight elements are screened from each weight element; based on the adaptive weight elements, the weight sparsity rate corresponding to each layer structure is determined.

[0157] In some embodiments, the sparse mask determination module 804 is further used to determine the number of pruning iterations for the targeted layer structure; when the number of pruning iterations does not meet the pruning end condition, the pre-pruning weight elements of the targeted layer structure are updated based on the weight sparsity rate corresponding to the targeted layer structure to obtain the sparse mask of the targeted layer structure for the current pruning iteration; the single-layer pruning module 806 is further used to perform single-layer pruning of the weight elements included in the targeted layer structure for the current pruning iteration based on the sparse mask of the targeted layer structure for the current pruning iteration to obtain the pruned weight elements of the current pruning iteration; The number of pruning iterations is updated. When the updated number of pruning iterations does not meet the pruning end condition, the pruned weight element of this pruning iteration is used as the pre-pruning weight element of the next pruning iteration, and the weight sparsity rate corresponding to the targeted layer structure is returned. The pre-pruning weight element of the targeted layer structure of this pruning iteration is updated to obtain the sparse mask of the targeted layer structure of this pruning iteration, so as to perform the next pruning iteration; when the updated number of pruning iterations meets the pruning end condition, the sparse weight element of the targeted layer structure after single-layer pruning is obtained according to the pruned weight element of the last pruning iteration.

[0158] In some embodiments, the single-layer pruning module 806 is further used to determine the pruning optimization amount parameter and pruning multiplier parameter of this pruning iteration by the alternating direction multiplier method; perform single-layer pruning calculation based on the sparse mask, pruning optimization amount parameter and pruning multiplier parameter of this pruning iteration of the targeted layer structure to obtain the pruned weight element of this pruning iteration; update the pruning optimization amount parameter based on the pruned weight element, pruning multiplier parameter and sparse mask to obtain the pruning optimization amount parameter of the next pruning iteration; obtain the pruning multiplier parameter of the next pruning iteration based on the pruned weight element, the pruning optimization amount parameter and pruning multiplier parameter of the next pruning iteration; when the updated number of pruning iterations meets the pruning end condition, calculate the sparse weight element of the targeted layer structure after single-layer pruning based on the pruned weight element of the last pruning iteration, the pruning multiplier parameter of the last pruning iteration and the sparse mask of the last pruning iteration.

[0159] In some embodiments, the single-layer quantization module 808 is further configured to determine the number of outer-layer quantization iterations for the targeted layer structure; when the number of outer-layer quantization iterations does not meet the outer-layer iteration termination condition, determine the quantization optimization parameter and quantization multiplier parameter of the current outer-layer quantization iteration by using the alternating direction multiplier method; perform inner-layer quantization iteration based on the quantization multiplier parameter and the initial sparse weight element of the current outer-layer quantization iteration to obtain the inner-layer quantization iteration variable; perform quantization iteration calculation based on the inner-layer quantization iteration variable, the quantization multiplier parameter, the quantization optimization parameter, and the initial sparse weight element to obtain the quantization weight of the current outer-layer quantization iteration; update the number of outer-layer quantization iterations; when the updated number of outer-layer quantization iterations does not meet the outer-layer iteration termination condition, use the quantization weight of the current outer-layer quantization iteration as the initial sparse weight element of the next outer-layer quantization iteration, return to the step of determining the quantization optimization parameter and quantization multiplier parameter of the current outer-layer quantization iteration by using the alternating direction multiplier method, and perform the next outer-layer quantization iteration; when the updated number of outer-layer quantization iterations meets the outer-layer iteration termination condition, obtain the target weight element corresponding to the targeted layer structure based on the quantization weight of the last outer-layer quantization iteration.

[0160] In some embodiments, the single-layer quantization module 808 is also used to determine the number of inner-layer quantization iterations corresponding to the current outer-layer quantization iteration; when the number of inner-layer quantization iterations does not meet the inner-layer iteration end condition, determine the pre-iteration variable of the current inner-layer quantization iteration; based on the initial inner-layer quantization iteration variable, the quantization multiplier parameter of the current outer-layer quantization iteration and the initial sparse weight element, calculate the post-iteration variable of the current inner-layer quantization iteration; update the number of inner-layer quantization iterations, and when the updated number of inner-layer quantization iterations does not meet the inner-layer iteration end condition, use the post-iteration variable of the current inner-layer quantization iteration as the pre-iteration variable of the next inner-layer quantization iteration, and return to the step of determining the pre-iteration variable of the current inner-layer quantization iteration to perform the next inner-layer quantization iteration; when the updated number of inner-layer quantization iterations meets the inner-layer iteration end condition, obtain the inner-layer quantization iteration variable according to the post-iteration variable of the last inner-layer quantization iteration.

[0161] Each module in the above-mentioned network model processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0162] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store various data involved in the network model processing method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a network model processing method is implemented.

[0163] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0164] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0166] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0168] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0169] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A network model processing method, characterized in that: The method comprises: Obtain a target network model, and determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight element and the weight element of the same layer, and the weight element of the same layer belongs to the same layer structure as the weight element; For each layer structure in each of the layer structures, determining a sparse mask of the targeted layer structure according to a weight sparse rate corresponding to the targeted layer structure; Performing single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain sparse weight elements of the targeted layer structure after single-layer pruning; Performing single-layer quantization on the sparse weight elements to obtain target weight elements corresponding to the targeted layer structure; The target network model is updated according to the target weight elements corresponding to each of the layer structures to obtain an optimized target network model; the optimized target network model includes an image processing model, which is used to process the input image to output an image processing result.

2. The method according to claim 1, characterized in that The determining, based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model, the weight sparsity rates corresponding to each layer structure respectively include: Determining a target sparsity rate for the target network model as a whole; For each weight element included in each layer structure in the target network model, based on the numerical relationship between the targeted weight element and the weight element of the same layer, calculate the relative sensitivity parameter of the targeted weight element; According to the target sparsity rate and the relative sensitivity parameters of each of the weight elements in the target network model, an adaptive weight element is obtained by screening out from each of the weight elements; The weight sparsity rates corresponding to the respective layer structures are determined based on the adaptive weight elements.

3. The method according to claim 1, characterized in that The step of determining the sparse mask of the targeted layer structure according to the weight sparse rate corresponding to the targeted layer structure includes: Determine the number of pruning iterations for the targeted layer structure; When the number of pruning iterations does not meet the pruning end condition, based on the weight sparsity rate corresponding to the targeted layer structure, the pre-pruning weight element of the targeted layer structure is updated to obtain the sparse mask of the targeted layer structure; The step of performing single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain the sparse weight elements of the targeted layer structure after single-layer pruning includes: Based on the sparse mask of the targeted layer structure in this pruning iteration, perform single-layer pruning of the targeted layer structure in this pruning iteration to obtain the pruned weight elements of this pruning iteration; The number of pruning iterations is updated. When the updated number of pruning iterations does not meet the pruning end condition, the post-pruning weight element of the current pruning iteration is used as the pre-pruning weight element of the next pruning iteration, the weight sparsity rate corresponding to the targeted layer structure is returned, the pre-pruning weight element of the current pruning iteration of the targeted layer structure is updated, and the sparse mask of the current pruning iteration of the targeted layer structure is obtained, so as to perform the next pruning iteration; When the updated number of pruning iterations satisfies the pruning end condition, the sparse weight elements of the targeted layer structure after single-layer pruning are obtained according to the pruned weight elements of the last pruning iteration.

4. The method according to claim 3, characterized in that The step of performing single-layer pruning of the weight elements included in the targeted layer structure based on the sparse mask of the targeted layer structure in the current pruning iteration to obtain the pruned weight elements of the current pruning iteration includes: The pruning optimization amount parameter and pruning multiplier parameter of this pruning iteration are determined by the alternating direction multiplier method; Performing single-layer pruning calculation based on the sparse mask of the current pruning iteration of the targeted layer structure, the pruning optimization amount parameter and the pruning multiplier parameter to obtain the pruned weight element of the current pruning iteration; Based on the pruned weight element, the pruning multiplier parameter and the sparse mask, the pruning optimization amount parameter is updated to obtain the pruning optimization amount parameter for the next pruning iteration; Obtaining a pruning multiplier parameter for the next pruning iteration based on the pruned weight element, the pruning optimization amount parameter for the next pruning iteration, and the pruning multiplier parameter; When the updated number of pruning iterations satisfies the pruning end condition, obtaining the sparse weight element of the targeted layer structure after single-layer pruning according to the pruned weight element of the last pruning iteration includes: When the updated number of pruning iterations meets the pruning end condition, the sparse weight elements of the targeted layer structure after single-layer pruning are calculated based on the pruned weight elements of the last pruning iteration, the pruning multiplier parameters of the last pruning iteration, and the sparse mask of the last pruning iteration.

5. The method according to any one of claims 1 to 4, characterized in that: The single-layer quantization is performed on the sparse weight element to obtain the target weight element corresponding to the targeted layer structure, including: Determine the number of outer layer quantization iterations for the targeted layer structure; When the number of outer layer quantization iterations does not meet the outer layer iteration termination condition, determining the quantization optimization parameter and the quantization multiplier parameter of this outer layer quantization iteration by an alternating direction multiplier method; Perform inner quantization iteration based on the quantization multiplier parameter of this outer quantization iteration and the initial sparse weight element to obtain an inner quantization iteration variable; Perform quantization iterative calculation based on the inner quantization iteration variable, the quantization multiplier parameter, the quantization optimization parameter and the initial sparse weight element to obtain the quantization weight of this outer quantization iteration; The outer quantization iteration number is updated. When the updated outer quantization iteration number does not meet the outer iteration end condition, the quantization weight of this outer quantization iteration is used as the initial sparse weight element of the next outer quantization iteration, and the step of determining the quantization optimization parameter and the quantization multiplier parameter of this outer quantization iteration by the alternating direction multiplier method is returned to perform the next outer quantization iteration; When the updated number of outer layer quantization iterations meets the outer layer iteration end condition, the target weight element corresponding to the targeted layer structure is obtained according to the quantization weight of the last outer layer quantization iteration.

6. The method according to claim 5, characterized in that The inner quantization iteration is performed based on the quantization multiplier parameter of the outer quantization iteration and the initial sparse weight element to obtain the inner quantization iteration variable, including: Determine the number of inner quantization iterations corresponding to this outer quantization iteration; When the number of inner layer quantization iterations does not satisfy the inner layer iteration end condition, determining the pre-iteration variable of this inner layer quantization iteration; Based on the initial inner quantization iteration variable, the quantization multiplier parameter of the current outer quantization iteration and the initial sparse weight element, the post-iteration variable of the current inner quantization iteration is calculated; The inner quantization iteration number is updated. When the updated inner quantization iteration number does not meet the inner iteration end condition, the post-iteration variable of the current inner quantization iteration is used as the pre-iteration variable of the next inner quantization iteration, and the step of determining the pre-iteration variable of the current inner quantization iteration is returned to perform the next inner quantization iteration; When the updated number of inner layer quantization iterations meets the inner layer iteration end condition, the inner layer quantization iteration variable is obtained according to the post-iteration variable of the last inner layer quantization iteration.

7. A network model processing device, characterized in that: The device comprises: A layer sparsity rate determination module is used to obtain a target network model, and determine the weight sparsity rate corresponding to each layer structure based on the relative sensitivity parameters of the weight elements included in each layer structure in the target network model; the relative sensitivity parameters are determined based on the numerical relationship between the weight elements and the weight elements of the same layer, and the weight elements of the same layer belong to the same layer structure as the weight elements; A sparse mask determination module, configured to determine, for each layer structure in each of the layer structures, a sparse mask of the layer structure according to a weight sparse rate corresponding to the layer structure; A single-layer pruning module, configured to perform single-layer pruning on the weight elements included in the targeted layer structure based on the sparse mask to obtain sparse weight elements of the targeted layer structure after single-layer pruning; A single-layer quantization module, used for performing single-layer quantization on the sparse weight elements to obtain target weight elements corresponding to the targeted layer structure; A model updating module is used to update the target network model according to the target weight elements corresponding to each of the layer structures to obtain an optimized target network model; the optimized target network model includes an image processing model, and the image processing model is used to process the input image to output the image processing result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images