A model processing method and apparatus

Abstract

The embodiment of the present application provides a model processing method and device, the model processing method comprises: obtaining a plurality of training samples, each training sample comprising feature data and a label value; based on the plurality of training samples, the weight of each parameter in a neural network model is calculated, the weight is used to indicate the importance of the parameter; based on the weight of each parameter, part of the parameters in the neural network model are attenuated. The model processing scheme provided by the embodiment of the present application attenuates the parameters in the model based on the importance of each parameter in the neural network model, and improves the model performance.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a model processing method and apparatus. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI is a branch of computer science that attempts to understand the essence of intelligence and enable machines to have the functions of perception, reasoning, and decision-making.

[0003] With the development of AI technology, neural network models, as AI models, have become increasingly complex, making them undeployable in many scenarios. To overcome this problem, numerous neural network compression methods have been proposed, among which neural network pruning is a commonly used approach. In one neural network pruning method, after training the neural network model, the importance of each parameter is estimated, and parameters are pruned based on their importance. However, if the assessment of parameter importance is inaccurate, some important parameters may be pruned, leading to a decrease in the performance of the pruned model. Summary of the Invention

[0004] The embodiments of this application aim to provide a model processing scheme that improves model performance by attenuating the parameters in the model based on the importance of each parameter in the neural network model.

[0005] To achieve the above objectives, the first aspect of this application provides a model processing method, comprising: acquiring multiple training samples, each training sample including feature data and label values; calculating the weights of each parameter in a neural network model based on the multiple training samples, the weights being used to indicate the importance of the parameters; and attenuating some parameters in the neural network model based on the weights of the parameters.

[0006] By attenuating the least important parameters in the neural network model based on their importance, instead of directly pruning them, we can avoid accidentally pruning important parameters, thereby improving the performance of the neural network model.

[0007] In one possible implementation of the first aspect of this application, the method further includes attenuating some parameters in the neural network model and then training the neural network model using the plurality of training samples to obtain a trained neural network model.

[0008] In one possible implementation of the first aspect of this application, the partial parameters include parameters of a first proportion in the neural network model.

[0009] In one possible implementation of the first aspect of this application, the method is used to decay some parameters of the neural network model multiple times, wherein, in the current execution, the first proportion is greater than the proportion of parameters decayed in the previous execution of the method.

[0010] By attenuating a small number of parameters in the neural network model during the initial training phase, and then increasing the number of parameters attenuated as the number of iterations increases, it is possible to avoid mistakenly pruning parameters that appear "unimportant" but are actually important in the early stages of neural network model training when the parameter values ​​in the neural network have not been fully trained. This can improve the performance of the final neural network model.

[0011] In one possible implementation of the first aspect of this application, the first ratio is determined based on the current number of executions, the maximum number of executions, and the target ratio, wherein the target ratio is the proportion of the number of target parameters to be pruned in the neural network model to the total number of parameters in the neural network model.

[0012] In one possible implementation of the first aspect of this application, the first ratio is determined based on a first amount of computation to be reduced in the neural network model, the first amount of computation to be reduced being determined based on the current number of executions of the method, the maximum number of executions, and the target amount of computation to be reduced in the neural network model.

[0013] In one possible implementation of the first aspect of this application, the method further includes: calculating the weights of each parameter in the trained neural network model based on the plurality of training samples; and pruning multiple parameters in the trained neural network model based on the weights of each parameter in the trained neural network model.

[0014] In this way, unimportant parameters in the neural network model are decayed multiple times in multiple iterations, and their values ​​approach 0. Removing these parameters will not affect the prediction loss of the neural network model. At this time, the importance of the parameters is close to their true importance. Therefore, pruning parameters based on their importance can reduce the number of parameters in the neural network model, reduce the computational load of the neural network model, and improve the performance of the neural network model while having a smaller impact on the model's prediction loss.

[0015] In one possible implementation of the first aspect of this application, calculating the weights of each parameter in the neural network model based on the plurality of training samples includes: calculating the change in model prediction loss caused by deleting each parameter from the neural network model as the weight of each parameter based on the plurality of training samples.

[0016] In one possible implementation of the first aspect of this application, calculating the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter includes: calculating a first-order approximation of the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

[0017] In one possible implementation of the first aspect of this application, the neural network model is used to classify images, the training samples include images, the feature data includes image features of the images, and the label values ​​include the classification results of the images.

[0018] In one possible implementation of the first aspect of this application, obtaining multiple training samples includes receiving a training dataset from a user device, obtaining multiple training samples from the training dataset, and the method further includes receiving the neural network model to be compressed from the user device; and after pruning the parameters of the neural network model, returning the pruned neural network model to the user device.

[0019] In one possible implementation of the first aspect of this application, the attenuation of multiple parameters in the neural network model includes attenuating the multiple parameters based on a predetermined coefficient, wherein the predetermined coefficient is greater than zero and less than 1.

[0020] In one possible implementation of the first aspect of this application, the method further includes receiving the target parameter pruning ratio or target computational load of the neural network model from a user device.

[0021] In one possible implementation of the first aspect of this application, the method is executed by any of the following processors: a central processing unit (CPU), a neural network process unit (NPU), or a graphics processing unit (GPU).

[0022] A second aspect of this application provides a model processing apparatus, comprising: an acquisition unit for acquiring multiple training samples, each training sample including feature data and label values; a calculation unit for calculating the weights of various parameters in a neural network model based on the multiple training samples, the weights indicating the importance of the parameters; and a decay unit for decaying some parameters in the neural network model based on the weights of the various parameters.

[0023] In one possible embodiment of the second aspect of this application, the apparatus further includes a training unit for training the neural network model using the plurality of training samples after attenuating some parameters in the neural network model to obtain a trained neural network model.

[0024] In one possible implementation of the second aspect of this application, the partial parameters include parameters of a first proportion in the neural network model.

[0025] In one possible implementation of the second aspect of this application, the apparatus is used to attenuate a portion of the parameters of the neural network model multiple times, wherein, in the current execution, the first proportion is greater than the proportion of parameters attenuated in the previous execution of the method.

[0026] In one possible implementation of the second aspect of this application, the first ratio is determined based on the current number of executions, the maximum number of executions, and the target ratio of the device, wherein the target ratio is the proportion of the number of target parameters to be pruned in the neural network model to the total number of parameters in the neural network model.

[0027] In one possible implementation of the second aspect of this application, the first ratio is determined based on a first amount of computation to be reduced in the neural network model, the first amount of computation to be reduced being determined based on the current number of executions of the device, the maximum number of executions, and the target amount of computation to be reduced in the neural network model.

[0028] In one possible embodiment of the second aspect of this application, the apparatus further includes a pruning unit, configured to: calculate the weights of each parameter in the trained neural network model based on the plurality of training samples; and prune multiple parameters in the trained neural network model based on the weights of each parameter in the trained neural network model.

[0029] In one possible implementation of the second aspect of this application, the computing unit is specifically used to: calculate, based on the plurality of training samples, the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

[0030] In one possible implementation of the second aspect of this application, the computing unit is specifically used to calculate a first-order approximation of the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

[0031] In one possible implementation of the second aspect of this application, the device is deployed on any of the following processors: a central processing unit (CPU), a neural network processor (NPU), or a graphics processing unit (GPU).

[0032] A third aspect of this application provides a computing device including a processing unit and a storage unit, wherein the storage unit stores executable code, and the processing unit executes the executable code to implement the method described in the first aspect of this application.

[0033] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed in a computer or processor, causes the computer or processor to perform the method described in the first aspect of this application.

[0034] The fifth aspect of this application provides a computer program product comprising a computer program that, when run in a computer or processor, causes the computer or processor to perform the method described in the first aspect of this application. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This diagram illustrates a basic framework for artificial intelligence.

[0037] Figure 2 A system architecture provided for embodiments of this application;

[0038] Figure 3 This is a schematic diagram of the CNN model structure;

[0039] Figure 4 This is a schematic diagram illustrating the process of convolving image data using a convolution kernel.

[0040] Figure 5 This is a hardware structure diagram of an NPU chip provided in an embodiment of this application;

[0041] Figure 6 This diagram illustrates the pruning of a neural network model (i.e., a neural network).

[0042] Figure 7 The graph shows the loss function as a function of the parameters.

[0043] Figure 8 This is a schematic diagram illustrating the attenuation of neural network parameters in an embodiment of this application;

[0044] Figure 9A flowchart illustrating a model processing method provided in an embodiment of this application;

[0045] Figure 10 A schematic diagram illustrating the model processing method provided in the embodiments of this application;

[0046] Figure 11 An architectural diagram of a model processing device provided in an embodiment of this application;

[0047] Figure 12 This is a schematic diagram of the structure of the model processing device provided in the embodiments of this application;

[0048] Figure 13 This is an architecture diagram of the cloud service system provided in the embodiments of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0050] Figure 1 A schematic diagram of an artificial intelligence framework is shown, which describes the overall workflow of an artificial intelligence system and is applicable to general artificial intelligence domain needs.

[0051] The above-mentioned artificial intelligence framework will be elaborated from two dimensions: the "intelligent information chain" (horizontal axis) and the "Internet Technology (IT) value chain" (vertical axis).

[0052] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom."

[0053] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of artificial intelligence, information (provided and processed by technology) to the industrial ecosystem of systems.

[0054] The main components of the artificial intelligence framework are as follows:

[0055] (1) Infrastructure

[0056] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This infrastructure includes: sensors for external communication; intelligent chips (hardware acceleration chips such as Central Processing Units (CPUs), Graphics Processing Units (GPUs), Neural-network Processing Units (NPUs), Application Specific Integrated Circuits (ASICs), and Field Programmable Gate Arrays (FPGAs)) for providing computing power; and the basic platform, including distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and then provide this data to the intelligent chips in the distributed computing system provided by the basic platform for computation.

[0057] (2) Data

[0058] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0059] (3) Data processing

[0060] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0061] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0062] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0063] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0064] (4) General ability

[0065] After the data processing mentioned above, some general capabilities can be formed based on the results of the data processing. These capabilities can be algorithms or general systems, such as translation, text analysis, computer vision processing, speech recognition, image recognition, and so on.

[0066] (5) Smart Products and Industry Applications

[0067] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, safe city, and intelligent terminals.

[0068] Figure 2 This application provides a system architecture 200. A data acquisition device 260 is used to collect AI model sample data and store it in a database 230. A training device 220 generates a target model / rule 201 based on the sample data maintained in the database 230.

[0069] The AI ​​model includes, for example, a neural network model.

[0070] The neural network model described is a network structure that mimics the behavioral characteristics of animal neural networks for information processing; it is also simply referred to as an artificial neural network (ANN). Such neural network models include at least one of several types, such as convolutional neural networks (CNNs), deep neural networks (DNNs), and recurrent neural networks (RNNs). The structure of a neural network model consists of a large number of interconnected nodes (or neurons). Based on a specific computational model, it learns and trains from input information to process information. A neural network model includes an input layer, hidden layers, and an output layer. The input layer receives the input signal, the output layer outputs the computational results of the neural network, and the hidden layers are responsible for learning, training, and other computational processes. They are the network's memory units, and the memory function of the hidden layers is represented by a weight matrix; typically, each neuron corresponds to a weight coefficient.

[0071] The function of each layer in a neural network model can be expressed mathematically. To describe, among which, Let y be the input vector of this layer, y be the output value (or output vector) of this layer, and a, W, and b be the model parameters included in this layer. The input vector of the model's input layer is the model's input feature vector, where each element is a feature value of the object to be predicted. The output value of the model's output layer is the model's predicted value, which indicates the prediction result for the object to be predicted. From a physical perspective, the work of each layer in a neural network model can be understood as transforming the input space (the set of input vectors) to the output space through five operations: 1. Dimensionality increase / decrease; 2. Magnification / reduction; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are... The operation 4 is completed using +b, and the operation 5 is implemented using a(). The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within this class of things. Here, W is the weight vector, where each value represents the weight of a neuron in that layer of the neural network. This vector W determines the spatial transformation from the input space to the output space, as described above; that is, the weights W of each layer control how the space is transformed. The purpose of training the neural network model is to ultimately obtain the weight matrix of all layers of the trained neural network (a weight matrix formed by the vectors W from many layers). Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0072] During the training of the neural network model, the training device 220 can compare the current network's predicted value with the desired target value and update the weight vector of each layer of the neural network based on the difference between the two, thereby making the output of the neural network model as close as possible to the true desired predicted value. For example, if the predicted value of the neural network model is larger than the target value, the model's weight vector is adjusted to reduce the predicted value, and vice versa, until a target model / rule 201 that can predict the desired target value (i.e., the true value or label value) is obtained. For this purpose, a loss function or objective function can be predefined, which are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so training the neural network model becomes a process of minimizing this loss as much as possible.

[0073] The target model / rule 201 obtained from training device 220 can be applied to different systems or devices. Figure 2In the process, the execution device 210 is equipped with an I / O interface 212 for data interaction with external devices. The "user" can input data to the I / O interface 212 through the client device 240.

[0074] The execution device 210 can call data, code, etc. in the data storage system 250, and can also store data, instructions, etc. in the data storage system 250.

[0075] The calculation module 211 processes the input data using the target model / rule 201 to output the processing result. Finally, the I / O interface 212 returns the processing result to the client device 240 for the user. The execution device 210 may also include an associated function module (…). Figure 2 The diagram illustrates association function module 213 and association function module 214, which can perform association processing based on the processing result of calculation module 211 to output a result associated with the processing result.

[0076] At a deeper level, the training device 220 can generate corresponding target models / rules 201 based on different data for different objectives, in order to provide users with better results.

[0077] exist Figure 2 In the illustrated scenario, the user can manually specify the data input to execution device 210, for example, by operating through the interface provided by I / O interface 212. Alternatively, client device 240 can automatically input data to I / O interface 212 and obtain results. If automatic data input by client device 240 requires user authorization, the user can set appropriate permissions within client device 240. The user can view the output results of execution device 210 on client device 240, which can be presented through display, sound, animation, or other specific methods. Client device 240 can also act as a data acquisition terminal, storing the collected sample data into database 230.

[0078] It is worth noting that Figure 2 This is merely a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 2 In this context, the data storage system 250 is an external storage device relative to the execution device 210. In other cases, the data storage system 250 can also be placed within the execution device 210. Furthermore, the training device 220 and the execution device 210 can be the same computing device. For example, the training device 220 can be a platform server, which, after training the target model 201, acts as the execution device 210 to provide business processing services to the user.

[0079] Among the various AI models mentioned above, CNN models are commonly used for processing image data. For example, by training a CNN model using training samples, the CNN model can output a semantic classification of the image after inputting image data. If the image shows a cat, the CNN model will classify the image as "cat". The training samples used to train the CNN model include, for example, images (e.g., images containing cats) and the classification label values ​​of the images (e.g., the category "cat" corresponding to images containing cats).

[0080] Figure 3 This is a schematic diagram of the CNN model structure. Figure 3 As shown, the CNN model includes an input layer 31, at least one hidden layer 32, and an output layer 33. Each hidden layer 32 may include a convolutional layer 321, an activation layer 322, and a pooling layer 323. The input layer 31 is used to preprocess the raw image data, such as by removing the mean or by normalization. The convolutional layer 321 includes at least one convolutional kernel, also known as a feature extraction filter. This kernel is equivalent to a neuron in the neural network, corresponding to specific features of the image (such as edge features or embossing features). After performing convolution calculations on the image using this kernel, a feature image of the specific features of the image is output. The activation layer 322 is used to perform non-linear mapping on the output of the convolutional layer 321 using activation functions such as the Rectified Linear Unit (ReLU), the Sigmoid function, and the hyperbolic tangent function (Tanh). Pooling layer 323, also known as a downsampling layer, reduces the amount of data processing while retaining useful information. For example, the pooling layer can select the maximum value from every four neighboring pixels as the new pixel. Output layer 33 is usually a fully connected layer, which acts as a classifier in the convolutional neural network model. It fuses information based on the input of hidden layer 32 and outputs the model's prediction result.

[0081] Figure 4 This is a schematic diagram illustrating the process of convolving image data using a convolution kernel. For example... Figure 4 As shown, the input image 41 on the left is the input image of the convolutional layer 321, that is, the input... Figure 3 The hidden layer 32 receives image data from the input layer 31, where each element in the matrix data represents a pixel of the corresponding image. Figure 4 Convolutional kernel 42 is any convolutional kernel in convolutional layer 321. Figure 4The feature image 43 on the right is the image data output from the convolutional layer 321. The convolutional kernel 42 is a two-dimensional matrix with equal rows and columns (shown as a 3×3 matrix in the figure). During convolution, the kernel 42 slides across the two-dimensional data (i.e., pixel values) of the image and performs inner product calculations on the overlapping local data in the image, such as... Figure 4 As shown, the inner product of convolution kernel 42 and the first local data of input image 41 is 8, and this inner product result is a pixel value of output image 43. The length of one slide of the convolution kernel is called the stride; for example, if the convolution kernel slides one pixel at a time, the stride is 1.

[0082] Figure 5 This is a hardware structure diagram of an NPU chip provided in an embodiment of this application. The model prediction and model training of the AI ​​model (e.g., CNN model) described above can be achieved through... Figure 5 The NPU chip shown is implemented.

[0083] like Figure 5 As shown, the Neural Processing Unit (NPU) is mounted as a coprocessor on the main CPU, and tasks are assigned by the main CPU. The core of the NPU is the arithmetic circuit 503, which is controlled by the controller 504 to retrieve matrix data from the memory and perform multiplication operations.

[0084] In some implementations, the arithmetic circuit 503 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional pulsating array. The arithmetic circuit 503 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0085] The Bus Interface Unit (BIU) 510 is used for the interaction between the bus and the Direct Memory Access Controller (DMAC) 505 and the Instruction Fetch Buffer 509.

[0086] The instruction fetch memory 509 fetches instructions from external memory through the bus interface unit 510, and the DMAC 505 fetches the original data of the input matrix A or the weight matrix B from external memory through the bus interface unit 510.

[0087] DMAC 505 is mainly used to move input data from external memory (such as Double Data Rate Synchronous Dynamic Random Access Memory, DDR Memory) to unified memory 506, or to move weight data (such as weight matrix B) to weight memory 502, or to move input data (such as input matrix A) to input memory 501.

[0088] For example, the arithmetic circuit 503 can read the corresponding data of the weight matrix B from the weight memory 502 and cache it on each PE in the arithmetic circuit 503. The arithmetic circuit 503 reads the input matrix A data from the input memory 501 and performs matrix operations with the weight matrix B, storing part or the final result of the output matrix C in the accumulator 508. The unified memory 506 can be used to store input data and / or output data.

[0089] The vector computation unit 507 includes multiple processing units that further process the output of the computation circuit as needed, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. The vector computation unit 507 is mainly used for computation in non-convolutional / fully connected (FCL) layers of neural networks, such as pooling, batch normalization, and local response normalization.

[0090] In some implementations, vector computation unit 507 can store the processed output vector into a unified buffer 506. In some implementations, vector computation unit 507 can apply a nonlinear function to the output of arithmetic circuit 503 to generate activation values. In some implementations, vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the output vector processed by vector computation unit 507 can be used as activation input to arithmetic circuit 503, for example, for use in subsequent layers of a neural network.

[0091] The instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504. The unified memory 506, input memory 501, weighted memory 502, and instruction fetch buffer 509 are all on-chip memories. The external memories are independent of this NPU hardware architecture.

[0092] in, Figure 3and Figure 4 The operations of each layer in the CNN model shown can be performed by the operation circuit 503 or the vector calculation unit 507.

[0093] As mentioned above, with the development of artificial intelligence, neural network models are becoming increasingly complex while improving performance, meaning the number of parameters and computational load are growing exponentially. This makes neural network models unsuitable for deployment in many scenarios. To overcome this problem, neural network pruning methods are commonly used to remove redundant parameters from the neural network. By removing redundant parameters from the neural network model, the computations involved in those parameters can be saved, thereby compressing the number of neural network parameters and computational load.

[0094] Figure 6 This diagram illustrates the pruning of a neural network model (i.e., a neural network). Figure 6 The diagram illustrates a three-layer fully connected network as an example. The first layer of this network includes two nodes, the second layer includes five nodes, and the third layer includes two nodes. Nodes in different layers are interconnected, and each connection corresponds to a learnable parameter. Figure 6 As shown, after two prunings of the neural network, only three nodes are retained in the second layer of the network. The parameters corresponding to the connections associated with the two removed nodes are removed, thereby achieving compression of the neural network.

[0095] In one related neural network pruning method, the neural network is first trained based on the model prediction loss function (e.g., cross-entropy loss function) mentioned above. The cross-entropy loss function L... CE (x, label) is as shown in formula (1):

[0096] L CE (x, label) = -∑ i y i log(p i (1)

[0097] Where x represents the input data of the neural network model, i.e., the feature data of the object to be predicted, for example, reference... Figure 3 As shown, x is the image data input to the neural network model; label is the category label corresponding to the input data x. As mentioned above, relative to the image to be classified, label is the classification result of the image. x and label constitute a training sample of the neural network model. p i The probability that x belongs to category i, as predicted by the neural network model.

[0098] After the neural network is trained, the importance of each parameter in the neural network is estimated based on a certain rule. The parameters of the neural network are then pruned according to their importance (in ascending order) until the target (computational cost / number of parameters) requirement is met.

[0099] A commonly used metric for evaluating the importance of a parameter is the change in loss (I) caused by removing that parameter. m As shown in formula (2):

[0100] I m =Σ( x,label)∈D (L CE (x,label)-L CE (x,label|w m =0)) 2 (2)

[0101] Among them, I m Indicates parameter w m The importance of I will be discussed further below. m This is called the parameter w. m The importance weights; D represents the dataset, which includes multiple training samples used to train the neural network model; L CE (x, label) represents the model prediction loss corresponding to the input data x and the label; L CE (x,label|w m =0) is to set the parameter w m After setting it to 0, input the data x and the label corresponding to the model prediction loss.

[0102] Because neural networks involve a large number of parameters and the dataset D is often very large, it is usually impossible to directly calculate the importance of each parameter using formula (2). A common approach is to use a first / second-order approximation from the Taylor expansion of formula (2). Formula (3) is a first-order approximation of formula (2):

[0103]

[0104] in, That is, the loss function with respect to parameter w m The partial derivative of .

[0105] However, since using first / second-order approximations to evaluate the importance of each parameter is inaccurate, this inaccurate evaluation leads to the pruning of some important parameters, resulting in poor performance of the pruned model. Suppose that for a certain parameter w in the neural network... m , Figure 7 For the loss function L CE Regarding the parameter w mThe function curve graph. (Reference) Figure 7 ,when At that time, due to the gradient at that point, i.e. therefore Therefore, according to the neural network pruning method described above, the parameter w will be adjusted. m To cut, that is... And from Figure 7 As can be seen, in order Then, the loss function L is made CE Change ΔL CE like Figure 7 The value shown is very large, that is, for w m The clipping causes the loss function L CE The value of changes significantly, thus causing the change in loss I shown in formula (2) to be significantly reduced. m Larger, parameter w m In fact, w is a relatively important parameter for neural network models. According to the importance of the first-order approximation calculation shown in formula (3), w is selected. m Pruning resulted in the pruning of this important parameter.

[0106] In order to avoid pruning these parameters that have a significant impact on the loss function, this application provides a neural network pruning method. In this method, after calculating the importance of each parameter of the neural network as described above by, for example, formula (3), the selected parameters are only attenuated (for example, multiplied by a coefficient less than 1) instead of being pruned directly. Figure 8 This is a schematic diagram illustrating the attenuation of neural network parameters in an embodiment of this application. (Reference) Figure 8 For the parameter w in the neural network m ,when At that time, it can be determined That is, parameter w m Its importance is relatively small, in this case, let To update the parameters w of the neural network m .

[0107] Thus, because hour Therefore, if η is close to 1, Change to The resulting change in loss ΔL CE like Figure 8 The image shown is closer In other words, the decay process does not cause excessive changes in the model's predicted loss. Furthermore, because when When, the gradient of the curve g m ≠0, so in the next time according to When selecting parameters for attenuation, parameter w mThey will not be selected again, thus avoiding the incorrect pruning of important parameters. Only parameters (g) that have no actual impact on the loss are selected. m Always zero, and w m Only when the value is independent of the value will it be decayed multiple times and eventually set to zero.

[0108] Figure 9 A flowchart of a model processing method provided in this application embodiment, the method including the following steps:

[0109] Step S91: Obtain multiple training samples, each training sample including feature data and label values;

[0110] Step S92: Based on the multiple training samples, calculate the weights of each parameter in the neural network model, whereby the weights are used to indicate the importance of the parameter.

[0111] Step S93: Based on the weights of each parameter, some parameters in the neural network model are attenuated.

[0112] The model processing method provided in this application embodiment can be offered to users as a platform service for automatically compressing neural network models. For example, the user can use... Figure 2 The execution device 210 in the platform can predict business through the running model. Figure 2 The training device 220 in the middle is used to perform Figure 9 The method shown is used to ultimately provide a compressed neural network model to the execution device 210. Before the method is executed by the training device 220, the user provides the platform with a dataset for model compression, a target optimization loss function, metric requirements (parameter quantity requirements / model prediction speed / computational requirements, etc.), and the neural network model to be compressed. The dataset includes multiple training samples. The neural network model to be compressed can be an untrained initial model or a trained model. In the case where the neural network model to be compressed is an untrained initial model, the platform can randomly initialize the parameters of the neural network model. It is understood that the above description is merely illustrative and is not intended to limit the scope of the embodiments of this application. For example, the platform can also compress the neural network model based on a predetermined optimization loss function and predetermined metrics that match the neural network model to be compressed.

[0113] As described above, the neural network model to be compressed is, for example, an image classification model. The training samples for this image classification model include image data and the classification result of the image (i.e., image label value). This image classification model is, for example, as shown below. Figure 3The CNN model shown. After the training device 220 returns the compressed image classification model to the execution device 210, the execution device 210 can perform image classification operations using the image classification model. Figure 9 The method shown can be executed by processors such as CPU, NPU, and GPU in the training device 220, but this application embodiment does not limit this.

[0114] Figure 10 This is a schematic diagram of the model processing method provided in the embodiments of this application. The following will be combined with Figure 10 Detailed description Figure 9 The steps of the method shown.

[0115] First, in step S91, multiple training samples are obtained, each of which includes feature data and label values.

[0116] refer to Figure 10 It can be achieved through multiple iterations. Figure 9 The method shown allows for the adjustment of the parameters of the neural network model. In each iteration, a batch of training samples can be obtained from the dataset for that iteration. This acquisition can be, for example, random, and the acquired samples do not need to be returned to the dataset. Here, i can represent the iteration number, starting from 0 and increasing with the iteration number until a preset maximum iteration number T is reached. The number of samples used in one iteration can be preset based on the maximum iteration number T, the model's prediction accuracy, etc. The dataset, for example, is provided by the user to a platform for model compression. This dataset includes multiple training samples, each containing feature data of the object to be predicted and the object's label value. For example, if the neural network model is a CNN model for image classification, each sample includes image data as image feature data and the image's classification result.

[0117] In step S92, based on the plurality of training samples, the weights of each parameter in the neural network model are calculated, and the weights are used to indicate the importance of the parameter.

[0118] Referring to the description above, the weights of each parameter in the neural network model can be calculated using formulas (2), (3), or a second-order approximation of formula (2). The magnitude of these weights corresponds to the importance of the parameter. (See also...) Figure 10 In training device 220, g can be obtained by the NPU running the neural network model to perform forward prediction and backpropagation calculations relative to each training sample. m The value of can be used to calculate the importance of each parameter in the neural network model through formula (3).

[0119] In step S93, based on the weights of each parameter, some parameters in the neural network model are attenuated.

[0120] After obtaining the importance of each parameter in the neural network model, multiple parameters with the lowest importance in the first proportion can be selected as parameters to be attenuated. The first proportion can be a preset proportion or a proportion calculated based on formula (4) or (5) as described below. i , where i represents the number of iterations of the method. The attenuation is, for example, multiplying the parameter w by a parameter η greater than 0 and less than 1, i.e., for the parameter w to be attenuated. m , making w m ←ηw m , where η∈(0,1). It is understood that the attenuation method is not limited to that described above, but can include any method used for parameter attenuation. By attenuating parameters in the neural network model based on the importance of each parameter, rather than directly pruning them, the accidental pruning of important parameters can be avoided, thereby improving the performance of the final neural network model.

[0121] In multiple iterations of this method, the number of parameters decayed in each iteration can be the same or different.

[0122] In one scenario, as mentioned above, the neural network model to be compressed provided by the user to the platform may be an untrained initial model. Therefore, refer to... Figure 10 After parameter decay in a neural network model, a model optimization step can be included to further train the model after parameter decay, resulting in better predictive performance. In the early stages of neural network model training (i.e., the initial few iterations), the parameter values ​​are not fully trained. At this time, "unimportant" parameters may become "important" after sufficient training. As the number of iterations increases, the probability that "unimportant" parameters will eventually become "unimportant" also increases. Therefore, in the early stages of training, a smaller number of parameters in the neural network model can be decayed. As the number of iterations increases, the number of decayed parameters can be increased. By selecting the number of decayed parameters in this way, the performance of the final neural network model can be improved.

[0123] In one implementation, the user provides the platform with a target cropping ratio (Ratio). T The target pruning ratio is the ratio of the number of target pruning parameters to the initial number of parameters in the neural network model. Therefore, in each iteration, the decay ratio Ratio for the i-th iteration can be set. i This makes the Ratio i It increases as i increases, and reaches Ratio in the Tth iteration. TThe attenuation ratio is the ratio of the number of parameters to be attenuated in the i-th iteration to the number of initial parameters in the model. For example, it can be set as shown in formula (4):

[0124]

[0125] It is understandable that formula (4) is just the ratio. i Ratio is an example linear form that varies with i, i The change relative to i can also be in exponential or other linear form, without limitation.

[0126] In another implementation, the user provides the platform with the target computational cost (FLOGs) of the neural network model. target Assuming FLOGs target =2G, assuming the initial computational cost of this neural network model is FLOGs. original =4G, then the amount of computation to be reduced, FLOGs, corresponding to the parameter to be decayed in the current iteration can be calculated using formula (5). i :

[0127]

[0128] Among them, FLOGs original -FLOGs target This represents the target computational cost to be reduced. The computational cost FLOGs to be reduced corresponding to the decay parameter in the current iteration is calculated using formula (5). i Then, based on FLOGs i Determine the attenuation ratio Ratio i This makes the Ratio i Satisfy the ratio of the model to Ratio i After pruning the least important parameters, the computational cost of the neural network model is reduced by FLOGs. i For example, this can be achieved through a binary search based on FLOGs. i Determine the attenuation ratio Ratio i Specifically, you can first set the Ratio i Set it to 0.5, and determine FLOGs based on the reduction in computational cost corresponding to 0.5. i The corresponding ratio should fall within the interval between (0, 0.5) and (0.5, 1). For example, it should fall within (0, 0.5). Then adjust the ratio accordingly. i Set it to 0.25, and similarly determine FLOGs. i The corresponding ratio should fall within the interval between (0, 0.25) and (0.25, 0.5). This process is repeated until the corresponding FLOGs are found. i attenuation ratio i.

[0129] refer to Figure 10 In step S93, the importance of each parameter in the neural network model and the decay ratio (Ratio) for this iteration are determined. i Then, based on the importance of each parameter and the attenuation ratio, i Select the parameters to be decayed in the neural network model, and then decay these selected parameters. For example... Figure 10 As shown in the neural network on the right, after attenuating the selected parameters, the number of nodes (or neurons) in the model does not change, but the parameter values ​​decrease. For example, suppose the connections between nodes 1 and 2 in this neural network and nodes in the previous layer correspond to the attenuated parameters. After attenuation, these connections are shown as gray lines, indicating that the parameter values ​​have decreased. It can be understood that in... Figure 10 Although the diagram shows that all parameters (i.e. all connections) corresponding to nodes 1 and 2 have been attenuated, it is understood that this is merely exemplary and is not intended to limit the scope of the embodiments of this application. In the embodiments of this application, the attenuation of some parameters of the nodes may also be included.

[0130] After performing the above steps S91-S93, optionally, the following can be performed: Figure 10 Step S104 involves training the neural network model. For example, this step S104 can be performed when the user provides an untrained neural network model to the platform, or it can be performed to obtain better model prediction performance. For instance, the neural network model can be trained using common neural network optimization algorithms such as gradient descent (GD) based on the i-th batch of samples and a loss function (e.g., the loss function shown in formula (1) above), thereby optimizing the neural network model and improving its prediction performance. Various specific algorithms within the gradient descent method can be used for model optimization in this embodiment, including, for example, stochastic gradient descent (SGD).

[0131] After step S104, step S105 can be executed to determine whether i has reached the maximum iteration number T, that is, to determine whether i is equal to T. If not, the iteration number i is incremented by 1, and a new round of iteration begins. Through multiple iterations of steps S91-S105, the decay of parameters in each iteration does not cause a significant change in the model's prediction loss. m Parameter w that is not always zero m After one decay, it will not be selected again, thus avoiding the incorrect pruning of important parameters. And for g... mThe parameter w that is always zero (or has a minimum value) m In multiple iterations, it will be selected for decay multiple times until it is finally set to zero. This method can be used to determine the truly unimportant parameters for pruning.

[0132] If it is determined in step S105 that i equals T, then step S106 can be executed to calculate the importance of the parameters in the trained neural network model. The specific implementation of this step can be referred to the description of step S92 above, and will not be repeated here. Since the model parameters are attenuated and the model is optimized after executing step S92, the parameter importance calculated in step S106 has changed compared with the parameter importance in step S92.

[0133] After performing step S106 to calculate the importance of the parameters, step S107 is performed to prune the parameters in the trained neural network model based on the importance of each parameter in the trained neural network model.

[0134] After multiple iterations, in the T-th iteration, the importance of the trained model parameters calculated in step S106 reflects the true importance of the parameters. Unimportant parameters in the neural network model are decayed multiple times in the iterations, their values ​​approaching 0. Therefore, removing these parameters will not affect the predictive loss of the neural network model. Thus, the target pruning ratio (Ratio) can be selected based on the importance of the parameters. T By pruning the parameters, the number of parameters in the neural network model can be reduced, thus decreasing the computational load and improving the performance of the neural network model, while having a smaller impact on the model's prediction loss.

[0135] Figure 11 This is an architectural diagram of a model processing device provided in an embodiment of this application. The model processing device is used to perform... Figure 9 The method shown can be deployed in Figure 2 In the training device 220, the model processing device includes:

[0136] The acquisition unit 111 is used to acquire multiple training samples, each training sample including feature data and label values;

[0137] The calculation unit 112 is used to calculate the weight of each parameter in the neural network model based on the plurality of training samples, wherein the weight is used to indicate the importance of the parameter;

[0138] The attenuation unit 113 is used to attenuate some parameters in the neural network model based on the weights of the various parameters.

[0139] In one possible implementation, the model processing apparatus further includes a training unit 114, configured to train the neural network model using the plurality of training samples after attenuating some parameters in the neural network model to obtain a trained neural network model.

[0140] In one possible implementation, the device is used to decay a portion of the parameters of the neural network model multiple times, wherein, in the current execution, the first proportion is greater than the proportion of parameters decayed in the previous execution of the method.

[0141] In one possible implementation, the first ratio is determined based on the current number of executions, the maximum number of executions, and the target ratio, wherein the target ratio is the proportion of the number of target parameters to be pruned in the neural network model to the total number of parameters in the neural network model.

[0142] In one possible implementation, the first ratio is determined based on a first amount of computation to be reduced in the neural network model, which is determined based on the current number of executions of the device, the maximum number of executions, and the target amount of computation to be reduced in the neural network model.

[0143] In one possible implementation, the model processing device further includes a pruning unit 115, configured to: calculate the weights of each parameter in the trained neural network model based on the plurality of training samples; and prune multiple parameters in the trained neural network model based on the weights of each parameter in the trained neural network model.

[0144] In one possible implementation, the computing unit 112 is specifically used to calculate, based on the plurality of training samples, the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

[0145] In one possible implementation, the computing unit 112 is specifically used to calculate a first-order approximation of the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

[0146] In one possible implementation, the model processing device is deployed on any of the following processors: a central processing unit (CPU), a neural network processor (NPU), or a graphics processing unit (GPU). The various units included in the model processing device can be in hardware, software, or firmware form, and this application embodiment does not limit this.

[0147] This application also provides a computing device, including a processing unit and a storage unit, wherein the storage unit stores executable code, and the processing unit executes the executable code to implement the present application. Figure 9 The method shown.

[0148] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed in a computer or processor, it causes the computer or processor to perform actions as described in this application. Figure 9 The method shown.

[0149] This application also provides a computer program product comprising a computer program that, when run in a computer or processor, causes the computer or processor to perform the actions described in this application. Figure 9 The method shown.

[0150] Figure 12 This is a schematic diagram of the structure of the model processing device provided in an embodiment of this application. The model processing device may include a processor 110, an internal memory 120, a connection module 130, a display 140, and an interface module 150. The model processing device may also include other modules or components, such as an audio module, etc. Figure 12 The components are not shown in the figures. It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the model processing apparatus. In other embodiments of this application, the model processing apparatus may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0151] Processor 110 may include one or more processing units, such as: processor 110 may include at least one of the following: application processor (AP), modem processor, GPU, image signal processor (ISP), CPU, video codec, digital signal processor (DSP), baseband processor, and / or NPU. Figure 12 The processor 110 schematically illustrates a CPU, ISP, NPU, and GPU, which are connected via a bus. The different processing units can be independent devices or integrated into one or more processors. For example, processor 110 can be a single chip or chipset. For example, the application processor can be the CPU.

[0152] The processor 110 may also include a memory for storing instructions and data. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses to external memory, reduces the processor 110's waiting time, and thus improves system efficiency.

[0153] Internal memory 120, also called main memory, can be used to store executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the model processing device by running the instructions stored in internal memory 120. Internal memory 120 may include a program storage area and a data storage area. The program storage area can store the operating system, application code, etc., for example... Figure 12 As shown, the program storage area of ​​the internal memory 120 stores information for execution. Figure 9 The method shown includes multiple code modules such as an acquisition module 121, a calculation module 122, and a decay module 123. The acquisition module 121 acquires multiple training samples, each including feature data and a label value. The calculation module 122 calculates the weights of each parameter in the neural network model based on the multiple training samples; these weights indicate the importance of the parameters. The decay module 123 decays some parameters in the neural network model based on the weights of the parameters.

[0154] In addition, the internal memory 120 may include random access memory (RAM), such as double data rate synchronous dynamic random access memory (DDR memory), and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0155] The connection module 130 can be used for wired connections and wireless local area networks (WLANs), allowing the model processing device to connect to a user device to receive model processing requests from the user device and return a compressed neural network model to the user device. The display screen 140 is used to display text, images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. The interface module 150 includes various interfaces, such as an external memory interface and a USB interface.

[0156] Figure 13 An architecture diagram of the cloud service system provided in this application embodiment. See also... Figure 13 The system includes: a hardware layer 1307 and a virtual machine monitor (VMM) 1301 running on top of the hardware layer 1307, as well as multiple virtual machines (VMs). i 1302. A virtual machine can serve as a virtual server node in the cloud service system 1300. Optionally, a virtual machine can also be designated as a coordinating node. The model processing apparatus described in the above embodiments can be a virtual machine in the cloud service system 1300.

[0157] Specifically, Virtual Machine 1302 is a virtual computer (server) simulated on public hardware resources using virtual machine software. Operating systems and applications can be installed on the virtual machine, and it can also access network resources. For applications running in the virtual machine, it's as if they are working on a real computer.

[0158] The hardware layer 1307 is the hardware platform for running the virtualized environment, which can be abstracted from the hardware resources of one or more physical hosts. The hardware layer may include various hardware components, such as a processor 1304 (e.g., CPU) and memory 1305, as well as network interface cards 1303, high-speed / low-speed input / output (I / O) devices, and other devices with specific processing functions.

[0159] Virtual machine 1302, based on the VMM and the hardware resources provided by hardware layer 1307, runs executable programs to achieve the above. Figure 9 The method steps in the relevant embodiments are omitted here for brevity.

[0160] It should be understood that the descriptions such as "first" and "second" in this article are merely for the sake of simplicity in description and to distinguish similar concepts, and do not have any other limiting function.

[0161] Those skilled in the art will clearly understand that the descriptions of the various embodiments provided in this application can be referenced to each other. For the sake of convenience and brevity, for example, the functions and execution steps of the various devices and equipment provided in the embodiments of this application can be referred to the relevant descriptions of the method embodiments of this application. The method embodiments and the device embodiments can also be referenced to each other.

[0162] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways without exceeding the scope of this application. For example, the embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0164] Furthermore, the described apparatus and methods, as well as the schematic diagrams of different embodiments, can be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of this application. Additionally, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through interfaces, devices, or units, and may be electronic, mechanical, or other forms.

[0165] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model processing method, characterized in that, include: Multiple training samples are obtained, each training sample including feature data and label values. The feature data includes image data, and the label values ​​include the classification results of the image data. Based on the multiple training samples, the weights of each parameter in the neural network model are calculated. The weights are used to indicate the importance of the parameters. The neural network model is used to classify images. Based on the weights of each parameter, the values ​​of some parameters in the neural network model are attenuated. The partial parameters include parameters of a first proportion in the neural network model. The first proportion is determined based on the current number of executions, the maximum number of executions, and the target proportion. The target proportion is the ratio of the number of target pruned parameters in the neural network model to the total number of parameters in the neural network model.

2. The method according to claim 1, characterized in that, It also includes, after attenuating the values ​​of some parameters in the neural network model, using the multiple training samples to train the neural network model to obtain the trained neural network model.

3. The method according to claim 2, characterized in that, The method is used to decay the values ​​of some parameters of the neural network model multiple times, wherein, in the current execution, the first proportion is greater than the proportion of parameters decayed in the previous execution of the method.

4. The method according to claim 1, characterized in that, The first ratio is determined based on the first amount of computation to be reduced in the neural network model, which is determined based on the current number of executions of the method, the maximum number of executions, and the target amount of computation to be reduced in the neural network model.

5. The method according to claim 2, characterized in that, Also includes: Based on the multiple training samples, the weights of each parameter in the trained neural network model are calculated; Based on the weights of each parameter in the trained neural network model, some parameters are pruned in the trained neural network model.

6. The method according to claim 1, characterized in that, Calculating the weights of each parameter in the neural network model based on the multiple training samples includes: calculating the change in model prediction loss caused by deleting each parameter from the neural network model as the weight of each parameter, based on the multiple training samples.

7. The method according to claim 6, characterized in that, Calculating the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter includes: calculating a first-order approximation of the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

8. The method according to any one of claims 1-7, characterized in that, The method is executed by any of the following processors: CPU, NPU, or GPU.

9. A model processing device, characterized in that, include: An acquisition unit is used to acquire multiple training samples, each training sample including feature data and label values, wherein the feature data includes image data and the label values ​​include the classification results of the image data; A computing unit is used to calculate the weights of each parameter in a neural network model based on the plurality of training samples. The weights are used to indicate the importance of the parameters. The neural network model is used to classify images. The attenuation unit is used to attenuate the values ​​of some parameters in the neural network model based on the weights of the various parameters. The partial parameters include parameters of a first proportion in the neural network model. The first proportion is determined based on the current number of executions, the maximum number of executions, and the target proportion of the device. The target proportion is the proportion of the number of target pruned parameters of the neural network model to the total number of parameters of the neural network model.

10. The apparatus according to claim 9, characterized in that, Also includes: The training unit is used to train the neural network model using the multiple training samples after attenuating the values ​​of some parameters in the neural network model to obtain the trained neural network model.

11. The apparatus according to claim 9, characterized in that, The device is used to decay the values ​​of some parameters in the neural network model multiple times, wherein, in the current execution, the first proportion is greater than the proportion of parameters decayed in the previous execution of the device.

12. The apparatus according to claim 9, characterized in that, The first ratio is determined based on the first amount of computation to be reduced in the neural network model, which is determined based on the current number of executions of the device, the maximum number of executions, and the target amount of computation to be reduced in the neural network model.

13. The apparatus according to claim 10, characterized in that, It also includes a pruning unit, used to: calculate the weights of each parameter in the trained neural network model based on the plurality of training samples; Based on the weights of each parameter in the trained neural network model, some parameters are pruned in the trained neural network model.

14. The apparatus according to claim 13, characterized in that, The computing unit is specifically used to: calculate the change in model prediction loss caused by deleting each parameter in the neural network model based on the multiple training samples, and use this change as the weight of each parameter.

15. The apparatus according to claim 14, characterized in that, The computing unit is specifically used to: calculate a first-order approximation of the change in model prediction loss caused by deleting each parameter in the neural network model as the weight of each parameter.

16. The apparatus according to any one of claims 9-15, characterized in that, The device is deployed on any of the following processors: a central processing unit (CPU), a neural network processor (NPU), or a graphics processing unit (GPU).

17. A computing device comprising a processing unit and a storage unit, wherein the storage unit stores executable code, and the processing unit executes the executable code to implement the method of any one of claims 1-8.

18. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed in a computer or processor, it causes the computer or processor to perform the method of any one of claims 1-8.

19. A computer program product comprising a computer program that, when run in a computer or processor, causes the computer or processor to perform the method of any one of claims 1-8.

Images