A model pruning method and device, task execution method and device, storage medium and equipment

Abstract

The specification discloses a model pruning, task execution method, device, storage medium and equipment. In the method, a to-be-processed model can be acquired, and for each intermediate layer included in the to-be-processed model, the importance of the intermediate layer is quantitatively evaluated according to input data input to the intermediate layer in the process of executing an inference task by the to-be-processed model and output results obtained by the intermediate layer, so as to determine a mutual information quantitative contribution value of the intermediate layer. Then, the mutual information quantitative contribution value of each intermediate layer can be used to select redundant intermediate layers from the intermediate layers, and the to-be-processed model is pruned according to the selection result to obtain a processed model, so as to reduce the parameter quantity of the processed model while avoiding large-scale retraining of the processed model, thereby reducing the cost required for adjusting a large-scale deep learning model.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of artificial intelligence, and particularly relates to a model pruning method, a task execution method, a device, a storage medium and equipment. BACKGROUND

[0002] With the rapid development of deep learning technology, large-scale deep learning models (for example: pre-training language models, convolutional neural network models, etc.) have been widely applied in key fields such as natural language processing and computer vision, and have greatly promoted the progress of artificial intelligence technology. However, since large-scale deep learning models usually have tens of billions or even more parameters, the inference process (i.e., the process of predicting or classifying new data by a trained model) of large-scale deep learning models needs to consume a large amount of computing resources and time, resulting in low inference efficiency and difficulty in meeting the needs of real-time scenarios.

[0003] In order to improve the inference efficiency of large-scale deep learning models, methods such as model compression (i.e., reducing the number of model parameters or reducing the precision of parameters to reduce computational complexity) and quantization (i.e., converting floating-point parameters in the model to low-bit integers to reduce computing resource consumption and improve computing speed) can be used to process large-scale deep learning models. However, since the model compression and quantization methods process large-scale deep learning models, large-scale retraining of the model is required to adapt to the model structure after parameter changes, resulting in high cost for adjusting large-scale deep learning models. SUMMARY

[0004] The present specification provides a model pruning method, a task execution method, a device, a storage medium and equipment to partially solve the above problems existing in the prior art.

[0005] The present specification adopts the following technical solutions:

[0006] The present specification provides a model pruning method, which comprises:

[0007] Obtaining mutual information quantization contribution values of each intermediate layer in the process of executing an inference task by a to-be-processed model, the mutual information quantization contribution value being used to represent the proportion of contribution information relative to redundant information for reducing the uncertainty of the output result of the intermediate layer in the case of known input data input to the intermediate layer, and the inference task comprising a visual processing task;

[0008] According to the mutual information quantization contribution values of the intermediate layers, the intermediate layers are sorted, and redundant intermediate layers are selected from the intermediate layers according to the sorting result;

[0009] According to the selection result, the redundant intermediate layer included in the to-be-processed model is pruned to obtain a processed model.

[0010] Optionally, the mutual information quantization contribution value of each intermediate layer in the process of the to-be-processed model performing an inference task is obtained, and specifically includes:

[0011] For each intermediate layer included in the to-be-processed model, the mutual information of the intermediate layer is determined according to input data and output results of the intermediate layer.

[0012] According to the mutual information, the entropy of the input data of the intermediate layer, and the entropy of the output results of the intermediate layer, the redundant information value between the input data and the output results of the intermediate layer is determined.

[0013] According to the ratio of the mutual information and the redundant information value, the mutual information quantization contribution value of the intermediate layer is determined.

[0014] Optionally, the mutual information of the intermediate layer is determined according to the input data and the output results of the intermediate layer, and specifically includes:

[0015] The entropy value of the input data of the intermediate layer and the conditional entropy value of the input data of the intermediate layer in the case of knowing the output results of the intermediate layer are determined.

[0016] According to the entropy value and the conditional entropy value, the mutual information of the intermediate layer is determined.

[0017] Optionally, the mutual information quantization contribution value of each intermediate layer in the process of the to-be-processed model performing an inference task is obtained, and specifically includes:

[0018] According to the order of the intermediate layers included in the to-be-processed model, the input data and the output results of each intermediate layer are sequentially input into a preset evaluation model, so that the evaluation model determines the mutual information quantization contribution value of each intermediate layer according to the input data and the output results of the intermediate layer and the correlation between the intermediate layers before the intermediate layer.

[0019] Optionally, the method further includes:

[0020] The preset first sample data is input into the processed model to determine the performance parameter of the processed model when performing an inference task corresponding to the first sample data.

[0021] According to a deviation between the performance parameter and a preset original performance parameter, a detection result for the processed model is obtained, and when it is determined according to the detection result that the processed model is abnormal, pruning processing is performed again on the to-be-processed model, and the original performance parameter is a performance parameter when the to-be-processed model performs an inference task corresponding to the first sample data.

[0022] Optionally, the method further includes:

[0023] The preset second sample data is input into the processed model to obtain output information.

[0024] According to a deviation between the output information and actual output information of the second sample data, a target loss value is determined, and the target loss value is positively correlated with the deviation.

[0025] The model parameters of the processed model are fine-tuned to obtain an adjusted model, with the optimization goal being to minimize the target loss value.

[0026] The present specification provides a task execution method, including:

[0027] Obtaining to-be-executed task data, the to-be-executed task data including at least one of text data, picture data, and video data;

[0028] The to-be-executed task data is input into a processed model to obtain an output result, and a task is executed according to the output result, and the processed model is obtained through the above model pruning method.

[0029] The present specification provides a model pruning device, including:

[0030] An obtaining module is configured to obtain mutual information quantization contribution values of each intermediate layer in a process in which a to-be-processed model performs an inference task, the mutual information quantization contribution value being used to represent a proportion of contribution information relative to redundant information for reducing uncertainty of an output result of the intermediate layer in a case where input data input into the intermediate layer is known, and the inference task including a visual processing task.

[0031] A selecting module is configured to sort the intermediate layers according to the mutual information quantization contribution values of the intermediate layers, and select redundant intermediate layers from the intermediate layers according to a sorting result.

[0032] A processing module is configured to perform pruning processing on the redundant intermediate layers included in the to-be-processed model according to a selection result, to obtain a processed model.

[0033] The specification provides a computer readable storage medium, the storage medium stores a computer program, the computer program is executed by a processor to realize the above-mentioned model pruning method.

[0034] The above-mentioned at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0035] In the model pruning method provided in the specification, first, the mutual information quantization contribution value of each intermediate layer in the process of executing the inference task of the to-be-processed model is obtained, wherein the mutual information quantization contribution value is used to represent the proportion of the contribution information to reducing the uncertainty of the output result of the intermediate layer relative to the redundant information under the condition that the input data input into the intermediate layer is known, the inference task includes a visual processing task, then the mutual information quantization contribution value of each intermediate layer is sorted, and according to the sorting result, the redundant intermediate layer is selected from each intermediate layer, and according to the selection result, the redundant intermediate layer contained in the to-be-processed model is pruned to obtain the processed model.

[0036] As can be seen from the above method, the mutual information quantization contribution value of each intermediate layer can be determined according to the input data and output result of each intermediate layer contained in the to-be-processed model, so that the to-be-processed model can be pruned according to the mutual information quantization contribution value of each intermediate layer to remove the intermediate layer containing more redundant information while ensuring that the performance loss of the to-be-processed model is within a controllable range, so that the parameter amount of the processed model can be reduced to reduce the multiplication and addition operations in matrix operation, and large-scale retraining of the processed model can be avoided to reduce the cost required for adjusting the large-scale deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0037] The drawings described herein are used to provide further understanding of the specification, and form a part of the specification. The illustrative embodiments of the specification and their descriptions are used to explain the specification, and do not constitute an improper limitation on the specification. In the drawings:

[0038] Figure 1 A flowchart of a model pruning method provided in the specification;

[0039] Figure 2 A schematic diagram of the process of obtaining the mutual information quantization contribution value provided in the specification;

[0040] Figure 3 A flowchart of a task execution method provided in the specification;

[0041] Figure 4 A schematic diagram of a model pruning device provided in the specification;

[0042] Figure 5 A task execution device is provided for the present specification;

[0043] Figure 6 An electronic device schematic diagram is provided for the present specification. Figure 1 An electronic device schematic diagram is provided for the present specification. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present specification clearer, the technical scheme of the present specification will be described in detail below in combination with specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present specification, not all. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present specification.

[0045] The technical scheme provided by each embodiment of the present specification will be described in detail below in combination with the drawings.

[0046] At present, due to the large parameter size of large-scale deep learning models, the efficiency of performing inference tasks through large-scale deep learning models is often low, therefore, in order to improve the inference efficiency of large-scale deep learning models, it is usually necessary to perform model compression and quantization on large-scale deep learning models, and after model compression and quantization of large-scale deep learning models, large-scale retraining of large-scale deep learning models is required, and the performance of the model may also be reduced, therefore, how to improve the inference efficiency of large-scale deep learning models without large-scale retraining is particularly important.

[0047] Figure 1 A model pruning method provided in the present specification is shown in the flowchart, including the following steps:

[0048] S101: Obtain mutual information quantization contribution values of each intermediate layer in the process of executing an inference task by a to-be-processed model, the mutual information quantization contribution values are used to represent the proportion of contribution information relative to redundant information for reducing the uncertainty of the output results of the intermediate layer under the condition that the input data input to the intermediate layer is known, and the inference task includes a visual processing task.

[0049] In the present specification, the business platform can obtain a to-be-processed model, and determine a mutual information quantization contribution value of each intermediate layer in the to-be-processed model according to a ratio of contribution information and redundancy information obtained by the intermediate layer in each layer of the to-be-processed model under the condition that input data known to be input into the intermediate layer is obtained to reduce uncertainty of an output result of the intermediate layer, and further can prune the to-be-processed model according to the mutual information quantization contribution value of each intermediate layer, so as to remove the intermediate layer containing more redundancy information, thereby reducing the parameter amount of the processed model, reducing the multiply-add operation in matrix operation, and further reducing the demand for computing resources when performing an inference task by the processed model, and reducing the occupation of GPU memory (GPU Memory) when performing an inference task by the processed model, so as to allow the graphics processor (Graphics Processing Unit, GPU) to use a larger batch size for data processing, thereby improving the utilization of the GPU.

[0050] The to-be-processed model described above can be a model adopting a Transformer architecture, such as a Bidirectional Encoder Representations from Transformers (BERT) model, a Generative Pre-trained Transformer (GPT) model, a Qwen model, etc.

[0051] The intermediate layer described above can include an attention mechanism layer and a feedforward neural network layer.

[0052] In the present specification, the execution subject for implementing the model pruning method can be a specified device such as a server provided in the business platform, or a terminal device such as a desktop computer or a notebook computer. For the convenience of description, the server is taken as an example of the execution subject, and the model pruning method provided in the present specification is described below.

[0053] In the present specification, after obtaining the to-be-processed model, the server can determine the input data input into each intermediate layer in the to-be-processed model and the output result obtained by the intermediate layer in the process of performing an inference task by the to-be-processed model, and further can determine the mutual information quantization contribution value of the intermediate layer according to the input data and the output result of the intermediate layer, as shown in Figure 2

[0054] Figure 2 The present specification provides a schematic diagram of the process of obtaining the mutual information quantization contribution value.

[0055] In combination with​Figure 2 It can be seen that the server can first determine the mutual information between the input data and the output result of the intermediate layer, where the mutual information is used to represent the amount of information that contributes to reducing the uncertainty of the output result of the intermediate layer when the input data input to the intermediate layer is known, which can be specifically referred to as the following formula:

[0056] I(X; Y) = H(X) - H(X|Y)

[0057] In the above formula, I(X; Y) is the mutual information between the input data X and the output result Y of the intermediate layer, H(X) is the entropy of the input data of the intermediate layer, and H(X|Y) is the conditional entropy of the input data of the intermediate layer when the output result of the intermediate layer is known.

[0058] Further, the server can determine the redundant information value between the input data and the output result according to the mutual information between the input data and the output result of the intermediate layer, the entropy of the input data of the intermediate layer, and the entropy of the output result of the intermediate layer, where the redundant information value is used to represent the amount of redundant information, which can be specifically referred to as the following formula:

[0059] Redundant information value = H(X) - I(X; Y) - H(Y)

[0060] In the above formula, H(Y) is the entropy of the output result Y of the intermediate layer.

[0061] Further, the server can determine the mutual information quantification contribution value of the intermediate layer according to the ratio between the mutual information and the redundant information value of the intermediate layer.

[0062] In actual application scenarios, the server can also input the input data and the output result of each intermediate layer of the to-be-processed model in the process of executing the inference task through the to-be-processed model into the preset evaluation model in sequence according to the order of the intermediate layers contained in the to-be-processed model, so that the evaluation model determines the mutual information quantification contribution value of each intermediate layer according to the input data input to the intermediate layer, the output result obtained through the intermediate layer, and the association relationship between the intermediate layers located before the intermediate layer.

[0063] The above evaluation model can be a long short-term memory neural network model (LSTM).

[0064] In the above content, the association relationship between the intermediate layers located before the intermediate layer can be a long-term dependency relationship between the intermediate layers.

[0065] S102: According to the mutual information quantitative contribution value of each intermediate layer, the intermediate layers are sorted, and a redundant intermediate layer is selected from the intermediate layers according to the sorting result.

[0066] S103: According to the selection result, the redundant intermediate layer contained in the to-be-processed model is pruned to obtain a processed model.

[0067] After the server determines the mutual information quantitative contribution value of each intermediate layer contained in the to-be-processed model, the server can sort the intermediate layers according to the mutual information quantitative contribution value of each intermediate layer, and select a redundant intermediate layer from the intermediate layers according to the order of the intermediate layers after priority sorting, and prune the to-be-processed model according to the selection result to obtain a processed model.

[0068] In addition, the server can determine whether to select each intermediate layer contained in the to-be-processed model as a redundant intermediate layer according to whether the mutual information quantitative contribution value of the intermediate layer is higher than a preset threshold.

[0069] In this specification, after the server selects a redundant intermediate layer, the server can set the weight of the neuron node contained in the selected intermediate layer to 0 to prune the to-be-processed model to obtain a processed model. In order to facilitate understanding, the following will take the example of selecting a redundant intermediate layer from the intermediate layers according to whether the mutual information quantitative contribution value of the intermediate layer is higher than a preset threshold and pruning the model as an example to explain the model pruning method in detail. For details, please refer to the following formula:

[0070]

[0071] In the above formula, W i is the weight matrix contained in the i-th intermediate layer, w i is the mutual information quantitative contribution value of the i-th intermediate layer, and ∈ is the preset threshold of the mutual information quantitative contribution value.

[0072] In actual application scenarios, the model after pruning may have performance loss. Therefore, in order to reduce the performance loss of the processed model, the server can also obtain first sample data after the server obtains the processed model, and input the first sample data into the processed model to determine the performance parameter of the processed model when performing an inference task corresponding to the first sample data. Then, according to the deviation between the performance parameter of the processed model and the preset original performance parameter, a detection result for the processed model is obtained, and when it is determined according to the detection result of the processed model that the processed model is abnormal, the to-be-processed model is pruned again.

[0073] The performance parameter is used to reflect the performance of the processed model in performing the inference task for the first sample data, such as accuracy, recall, mean square error, memory occupation, BLEU score, etc.

[0074] The original performance parameter is the performance parameter of the to-be-processed model in performing the inference task corresponding to the first sample data.

[0075] In the above, the server can determine whether the deviation between the performance parameter of the processed model and the preset original performance parameter exceeds the preset difference threshold, and if so, obtain the detection result that the processed model is abnormal.

[0076] In addition, the server can input the performance parameter of the processed model into the preset detection model to obtain the detection result of whether the processed model is abnormal through the preset detection model.

[0077] In addition, the server can obtain second sample data and input the second sample data into the processed model to obtain output information, and then determine a target loss value according to the deviation between the output information and actual output information corresponding to the second sample data, and fine-tune the model parameters of the processed model to obtain an adjusted model, with the minimization of the target loss value as the optimization objective.

[0078] The target loss value is positively correlated with the deviation between the output information and the real output information corresponding to the sample data.

[0079] In addition, in order to avoid increasing too many parameters or adjusting the values of part of the model parameters too much in the process of fine-tuning the model parameters of the processed model, so that the process of performing the task through the adjusted model greatly depends on these model parameters, thereby causing the generalization ability of the model to be reduced, the server can also determine a supplementary loss value according to the model parameters of the processed model, and then determine a fusion loss value of the processed model according to the target loss value and the supplementary loss value of the processed model, and fine-tune the model parameters of the processed model to obtain an adjusted model, with the minimization of the fusion loss value of the processed model as the optimization objective.

[0080] The supplementary loss value increases with the increase of the complexity of the processed model, for example, the more the model parameters of the processed model, the larger the supplementary loss value, and the larger the sum of squares of the model parameters of the processed model, the larger the supplementary loss value.

[0081] In the foregoing, the server can determine the fusion loss value of the processed model according to the target loss value and the supplementary loss value of the processed model, and the method can refer to the following formula:

[0082]

[0083] In the foregoing formula, is the fusion loss value, is the target loss value, is the supplementary loss value, and λ is a hyperparameter and θ is a model parameter.

[0084] The performance parameter is used to reflect the performance of the processed model when performing the inference task on the sample data, such as accuracy, recall, mean square error, memory occupation, BLEU score, etc.

[0085] In the foregoing, the server can determine the fusion loss value of the processed model according to the target loss value and the supplementary loss value of the processed model, and the method can refer to the following formula:

[0086] In addition, the server can input the performance parameter of the processed model into a preset detection model to obtain a detection result of whether the processed model is abnormal through the preset detection model. It should be noted that the inference task can include visual processing tasks such as image recognition, object detection, and semantic segmentation.

[0087] It should be noted that the inference task in the present specification can include visual processing tasks such as image recognition, object detection, and semantic segmentation.

[0088] When the inference task is an image recognition task, the server can input sample image data into the to-be-processed model to obtain the mutual information quantization contribution value of each intermediate layer of the to-be-processed model in the process of performing the inference task corresponding to the sample image data, and perform pruning processing on the to-be-processed model based on the mutual information quantization contribution value of each intermediate layer of the to-be-processed model in the process of performing the inference task corresponding to the sample image data to obtain a processed model. Then, the processed model can be used to perform image recognition on the to-be-recognized image data to obtain a recognition result.

[0089] When the inference task is an object detection task, the server can input the sample image data into the to-be-processed model to obtain the mutual information quantization contribution value of each intermediate layer of the to-be-processed model in the process of executing the inference task corresponding to the sample image data, perform pruning processing on the to-be-processed model based on the mutual information quantization contribution value of each intermediate layer of the to-be-processed model in the process of executing the inference task corresponding to the sample image data to obtain a processed model, and then perform object detection on the to-be-detected image data through the processed model to determine the category and position of each specific object (for example, a road sign, a pedestrian, a vehicle, etc.) contained in the to-be-detected image data.

[0090] Of course, the inference task described above can also include other deep learning tasks such as natural language processing (for example, machine translation, dialogue generation, sentiment analysis), speech processing (for example, speech recognition, speech synthesis), etc.

[0091] As can be seen from the above method, the server can determine the mutual information quantization contribution value of each intermediate layer in the to-be-processed model according to the input data and output result of each intermediate layer, so as to prune the to-be-processed model according to the mutual information quantization contribution value of each intermediate layer to remove intermediate layers containing more redundant information, thereby reducing the parameter amount of the processed model to reduce the multiplication and addition operations in matrix operation, and avoiding large-scale retraining of the processed model to reduce the cost required for adjusting large-scale deep learning models.

[0092] For ease of understanding, the process of performing a task by the processed model obtained by the above model pruning method is described in detail below, as shown in Figure 3

[0093] Figure 3 A flowchart of a task execution method provided in the present specification includes the following steps:

[0094] S301: Obtain to-be-executed task data, the to-be-executed task data including at least one of text data, picture data, and video data.

[0095] S302: Input the to-be-executed task data into the processed model to obtain an output result, and perform a task according to the output result, the processed model being obtained by pruning processing through the above model pruning method.

[0096] In the present specification, the server can obtain to-be-executed task data, input the to-be-executed task data into a processed model to obtain an output result, and perform a task according to the output result. ​

[0097] The to-be-executed task data can include at least one of text data, picture data, video data, and other multi-modal data.

[0098] The processed model can be obtained by pruning the model by using the model pruning method.

[0099] As can be seen from the above, the server can apply the processed model to deep learning tasks such as natural language processing and computer vision without the need for large-scale retraining of the processed model, which can effectively improve the inference efficiency and reduce the consumption of hardware resources.

[0100] The above is one or more embodiments of the model pruning and business execution method of the present specification. Based on the same idea, the present specification also provides a corresponding model pruning and business execution device, as shown in Figure 4 , Figure 5

[0101] Figure 4 A schematic diagram of a model pruning device provided by the present specification includes:

[0102] The acquisition module 401 is configured to acquire mutual information quantization contribution values of each intermediate layer in a process of executing an inference task by a to-be-processed model, the mutual information quantization contribution value being used to represent a proportion of contribution information relative to redundant information for reducing uncertainty of an output result of the intermediate layer in a case where input data input to the intermediate layer is known, and the inference task including a visual processing task.

[0103] The selection module 402 is configured to sort the intermediate layers according to the mutual information quantization contribution values of the intermediate layers, and select redundant intermediate layers from the intermediate layers according to a sorting result.

[0104] The processing module 403 is configured to perform pruning processing on the redundant intermediate layers included in the to-be-processed model according to a selection result, to obtain a processed model.

[0105] Optionally, the acquisition module 401 is specifically configured to, for each intermediate layer included in the to-be-processed model, determine mutual information of the intermediate layer according to input data and an output result of the intermediate layer, determine a redundant information value between the input data and the output result of the intermediate layer according to the mutual information, an entropy of the input data of the intermediate layer, and an entropy of the output result of the intermediate layer, and determine the mutual information quantization contribution value of the intermediate layer according to a ratio of the mutual information and the redundant information value.

[0106] ​Optionally, the obtaining module 401 is specifically configured to determine an entropy value of input data of the intermediate layer and a conditional entropy value of the input data of the intermediate layer in a case where an output result of the intermediate layer is known; and determine the mutual information amount of the intermediate layer according to the entropy value and the conditional entropy value.

[0107] Optionally, the obtaining module 401 is specifically configured to input the input data and the output result of each intermediate layer into a preset evaluation model in a sequence according to the sequence of the intermediate layers included in the to-be-processed model, so that the evaluation model determines the mutual information quantization contribution value of each intermediate layer according to the input data and the output result of the intermediate layer and the association relationship between the intermediate layers located before the intermediate layer.

[0108] Optionally, the apparatus further includes a detection module 404.

[0109] The detection module 404 is configured to input preset first sample data into the processed model to determine a performance parameter of the processed model when the processed model performs an inference task corresponding to the first sample data; obtain a detection result of the processed model according to a deviation between the performance parameter and a preset original performance parameter, the original performance parameter being a performance parameter of the to-be-processed model when the to-be-processed model performs the inference task corresponding to the first sample data; and when it is determined according to the detection result that the processed model is abnormal, re-perform the pruning processing on the to-be-processed model.

[0110] Optionally, the apparatus further includes an adjustment module 405.

[0111] The adjustment module 405 is configured to input preset second sample data into the processed model to obtain output information; determine a target loss value according to a deviation between the output information and actual output information of the second sample data, the target loss value being positively correlated with the deviation; and fine-tune model parameters of the processed model to obtain an adjusted model, with minimizing the target loss value as an optimization goal.

[0112] Figure 5 A task execution apparatus provided in the present specification includes:

[0113] An obtaining module 501 is configured to obtain to-be-executed task data, the to-be-executed task data including at least one of text data, picture data, and video data.

[0114] An execution module 502 is configured to input the to-be-executed task data into a processed model to obtain an output result, and perform a task according to the output result, the processed model being obtained by pruning the to-be-processed model through the model pruning method.

[0115] The specification also provides a computer readable storage medium storing a computer program, which can be used to execute the above Figure 1 The provided model pruning method.

[0116] The specification also provides a computer readable storage medium storing a computer program, which can be used to execute the above Figure 6 The provided model pruning method. Figure 1 The provided model pruning method. Figure 6 As described above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The provided model pruning method. Of course, in addition to the software implementation, the specification does not exclude other implementation manners, such as logic devices or a combination of software and hardware, and so on, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0117] For a technical improvement, it can be obvious whether the improvement is in hardware (e.g., improvement of circuit structures of diodes, transistors, switches, etc.) or in software (e.g., improvement of method flow). However, with the development of technology, many improvements of method flow today can be considered as direct improvements of hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement of method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A designer programs a digital system "integrated" on a PLD by himself / herself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually manufacturing an integrated circuit chip, such programming is now mostly implemented by "logic compiler" software, which is similar to a software compiler used when developing programs, and the original code before compilation also needs to be written in a specific programming language, which is called a hardware description language (HDL), and there are many kinds of HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that only a little logical programming of the method flow in the above-mentioned hardware description languages and programming into an integrated circuit can easily obtain a hardware circuit that implements the logical method flow.

[0118] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0119] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0120] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the specification.

[0121] Those skilled in the art will understand that the embodiments of the specification can be provided as a method, a system or a computer program product. Therefore, the specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0122] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1 The flow diagrams and / or block diagrams in this specification can present a process and / or system and components of a process and / or system as boxes of the flow diagrams and / or block diagrams. Alternatively, and / or additionally, the flow diagrams and / or block diagrams can represent concurrent processes and / or systems, and / or concurrent processes and / or systems may

[0123] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1 The flow diagrams and / or block diagrams in this specification can present a process and / or system and components of a process and / or system as boxes of the flow diagrams and / or block diagrams. Alternatively, and / or additionally, the flow diagrams and / or block diagrams can represent concurrent processes and / or systems, and / or concurrent processes and / or systems may

[0124] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 ​ The flow diagrams and / or block diagrams in this specification can present a process and / or system and components of a process and / or system as boxes of the flow diagrams and / or block diagrams. Alternatively, and / or additionally, the flow diagrams and / or block diagrams can represent concurrent processes and / or systems, and / or concurrent processes and / or systems may

[0125] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0126] The memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, such as Read Only Memory (ROM) or flash memory, among others in a computer readable medium. The memory is an example of computer readable media.

[0127] ​​​Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0128] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0129] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0131] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different aspects of the description. For each embodiment, the description focuses on the differences from the other embodiments. In particular, the description of the system embodiments is relatively brief, as the system embodiments are largely analogous to the method embodiments. The relevant parts of the description of the method embodiments are therefore referred to.

[0132] The above description is embodied in the form of examples of the description, and is not intended to limit the description. For those skilled in the art, the description can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the description shall be included in the scope of claims of the description.

Claims

1. A model pruning method, characterized in that, The method comprises the following steps: acquiring mutual information quantization contribution values of each intermediate layer in a process in which a to-be-processed model executes an inference task, wherein the inference task is used for processing at least one of text data, picture data and video data; for each intermediate layer contained in the to-be-processed model, mutual information of the intermediate layer is determined according to input data and output results of the intermediate layer; according to the mutual information, an entropy of the input data of the intermediate layer and an entropy of the output results of the intermediate layer, a redundant information value between the input data and the output results of the intermediate layer is determined; according to a ratio of the mutual information and the redundant information value, a mutual information quantization contribution value of the intermediate layer is determined; sorting the intermediate layers according to the mutual information quantization contribution values of the intermediate layers, and selecting redundant intermediate layers from the intermediate layers according to a sorting result; performing pruning processing on the redundant intermediate layers contained in the to-be-processed model according to the selection result, to obtain a processed model.

2. The method of claim 1, wherein, The mutual information of the intermediate layer is determined according to the input data and the output results of the intermediate layer, and specifically comprises the following steps: determining an entropy value of the input data of the intermediate layer and a conditional entropy value of the input data of the intermediate layer in a case where the output results of the intermediate layer are known; determining the mutual information of the intermediate layer according to the entropy value and the conditional entropy value.

3. The method of claim 1, wherein, The mutual information quantization contribution values of each intermediate layer in a process in which a to-be-processed model executes an inference task are acquired, and specifically comprises the following steps: inputting the input data and the output results of each intermediate layer into a preset evaluation model in a sequence according to an order of the intermediate layers contained in the to-be-processed model, so that the evaluation model determines a mutual information quantization contribution value of each intermediate layer according to the input data and the output results of the intermediate layer and an association relationship between the intermediate layers located before the intermediate layer.

4. The method of claim 1, wherein, The method further comprises the following steps: inputting preset first sample data into the processed model to determine a performance parameter of the processed model when the processed model executes an inference task corresponding to the first sample data; obtaining a detection result of the processed model according to a deviation between the performance parameter and a preset original performance parameter, wherein the original performance parameter is a performance parameter of the to-be-processed model when the to-be-processed model executes the inference task corresponding to the first sample data, and when it is determined according to the detection result that the processed model is abnormal, pruning processing is performed again on the to-be-processed model.

5. The method of claim 1, wherein, The method further comprises the following steps: inputting preset second sample data into the processed model to obtain output information; determining a target loss value according to a deviation between the output information and actual output information of the second sample data, wherein the target loss value is positively correlated with the deviation; fine-tuning model parameters of the processed model to obtain an adjusted model, with minimizing the target loss value as an optimization target.

6. A task execution method characterized by, The method comprises the following steps: acquiring to-be-executed task data, wherein the to-be-executed task data comprises at least one of text data, picture data and video data; The task data to be executed is input into the processed model to obtain an output result, and a task is executed according to the output result, wherein the processed model is obtained by pruning the model according to the method in any one of claims 1-5.

7. A model pruning apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire mutual information quantization contribution values of each intermediate layer in a process of executing an inference task by a to-be-processed model, wherein the inference task is configured to process at least one of text data, picture data, and video data; for each intermediate layer included in the to-be-processed model, mutual information of the intermediate layer is determined according to input data and an output result of the intermediate layer; a redundant information value between the input data and the output result of the intermediate layer is determined according to an entropy of the input data, the mutual information, and an entropy of the output result; and a mutual information quantization contribution value of the intermediate layer is determined according to a ratio of the mutual information and the redundant information value; A selection module is configured to sort the intermediate layers according to the mutual information quantization contribution values of the intermediate layers, and select redundant intermediate layers from the intermediate layers according to a sorting result. A processing module is configured to prune the redundant intermediate layers included in the to-be-processed model according to a selection result to obtain a processed model.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-6.

Images