Neural network model pruning method and apparatus, device, and medium

Abstract

The present disclosure relates to the technical field of neural network compression, in particular to a neural network model pruning method, device, equipment and medium. The neural network model pruning method comprises: obtaining environment state information of a neural network model to be pruned through reinforcement learning; obtaining a target pruning rate of each layer of the neural network model to be pruned according to the environment state information; obtaining a target reward value according to the target pruning rate and a reward function; in response to the target reward value being greater than or equal to a preset threshold, pruning the neural network model to be pruned according to the target pruning rate; and performing image processing on a target image according to the target neural network model. This method not only reduces the calculation amount of the neural network model, but also reduces the influence of errors that may occur during the operation of the pruned neural network model on the entire system, thereby ensuring that the reliability and energy consumption of the pruned neural network model meet the requirements.

Description

Technical Field

[0001] This disclosure relates to the field of neural network compression technology, specifically to a method, apparatus, device, and medium for pruning neural network models. Background Technology

[0002] With the rapid development of neural network algorithms and hardware chips, applying neural networks to industries such as image processing and power systems has become a promising solution. Neural network models possess advantages such as powerful and fast parallel computing capabilities, high fault tolerance, and strong learning ability. Typically, neural network models require very large computational costs and storage space; however, neural network compression can reduce the number of parameters or storage space required.

[0003] Among related technologies, model pruning is a relatively mainstream compression scheme. Specifically, model pruning removes unimportant weights (connections) or neurons from a neural network model, significantly reducing the model size and computational cost without sacrificing model accuracy, thereby reducing energy consumption.

[0004] However, even if the neural network model is pruned according to the above scheme to reduce the computational load, the pruned neural network model may still have a higher probability of errors in some cases. These errors can propagate through the neural network model and affect the output of the entire system, resulting in a significant decrease in the reliability of the system. Consequently, the accuracy of the pruned neural network model in image processing is low. Therefore, ensuring that the reliability and energy consumption of the pruned neural network model can meet the requirements has become an urgent problem to be solved. Summary of the Invention

[0005] To address the problems in the related technologies, embodiments of this disclosure provide a method, apparatus, device, and medium for pruning neural network models.

[0006] Firstly, this disclosure provides a method for pruning a neural network model.

[0007] Specifically, the neural network model pruning method includes:

[0008] Through reinforcement learning, environmental state information of the neural network model to be pruned is obtained, including basic feature information and enhanced feature information of the neural network model to be pruned;

[0009] Based on the environmental state information, the target pruning rate of each layer in the neural network model to be pruned is obtained;

[0010] Based on the target pruning rate and the reward function, a target reward value is obtained. The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate.

[0011] In response to the target reward value being greater than or equal to a preset threshold, the neural network model to be pruned is pruned according to the target pruning rate;

[0012] Image processing of the target object is performed based on the target neural network model obtained after cropping.

[0013] In conjunction with the first aspect, in a first implementation of the first aspect of this disclosure, before obtaining the target reward value based on the target pruning rate and the reward function, the method further includes:

[0014] The reliability of the pruned neural network model is obtained based on the reliability evaluation function.

[0015] Based on the energy consumption evaluation function, obtain the energy consumption of the pruned neural network model;

[0016] The reward function is obtained based on the reliability and energy consumption of the pruned neural network model.

[0017] In conjunction with the first aspect and the first implementation of the first aspect, in the second implementation of the first aspect of this disclosure, before obtaining the reliability of the pruned neural network model according to the reliability evaluation function, the method further includes:

[0018] Obtain system architecture reliability parameters and neuron sensitivity parameters;

[0019] Based on the system architecture reliability parameters and the neuron sensitivity parameters, the reliability evaluation function is constructed.

[0020] The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

[0021] In conjunction with the first aspect, the first implementation of the first aspect, and the second implementation of the first aspect, in the third implementation of the first aspect of this disclosure, the acquisition of system architecture reliability parameters includes:

[0022] according to Obtain the system architecture reliability parameter ARFi ;

[0023] Among them, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i represents the memory access amount of the i-th layer in the pruned neural network model, and All Params represents the memory access amount of all layers in the pruned neural network model.

[0024] In conjunction with the first aspect, the first implementation of the first aspect, and the second implementation of the first aspect, in the fourth implementation of the first aspect of this disclosure, the step of obtaining the reliability of the pruned neural network model according to the reliability evaluation function includes:

[0025] according to The reliability of the pruned neural network model (NRF) i ;

[0026] Among them, ARF i Let ΔE(h) be the reliability parameter of the system architecture. i ) represents the sensitivity parameter of the neuron, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i Let $\frac{1}{i}$ be the multiply-accumulate computation cost of the $i$-th layer in the pruned neural network model, and $\frac{AllMAC}{\frac{1}{2}$ be the multiply-accumulate computation cost of all layers in the pruned neural network model. $Params$ i Let represent the memory access amount of the i-th layer in the pruned neural network model, and AllParams represent the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

[0027] In conjunction with the first aspect and the first implementation of the first aspect, in the fifth implementation of the first aspect of this disclosure, before obtaining the energy consumption of the pruned neural network model according to the energy consumption evaluation function, the method further includes:

[0028] Obtain the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model;

[0029] The energy consumption evaluation function is constructed based on the full-load operation energy consumption and the bottleneck operation energy consumption.

[0030] In conjunction with the first aspect, the first implementation of the first aspect, and the fifth implementation of the first aspect, in the sixth implementation of the first aspect of this disclosure, obtaining the full-load operating energy consumption of the pruned neural network model includes:

[0031] Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation;

[0032] Based on the first computational load and the first memory access load, the first estimated power is obtained;

[0033] The full-load operating energy consumption is obtained based on the first estimated power and the first running time of the pruned neural network model under full-load operation.

[0034] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, the fourth implementation of the first aspect, the fifth implementation of the first aspect, and the sixth implementation of the first aspect, in the seventh implementation of the first aspect of this disclosure, the basic feature information includes at least one of the following:

[0035] The number of layers in the neural network model, the number of channels in the input feature map, the length of the input feature map, the width of the input feature map, the number of convolutional kernels, the length of the convolutional kernel, the width of the convolutional kernel, and the stride of the convolutional kernel.

[0036] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, the fourth implementation of the first aspect, the fifth implementation of the first aspect, and the sixth implementation of the first aspect, in the eighth implementation of the first aspect of this disclosure, the enhanced feature information includes at least one of the following:

[0037] Energy consumption of the pruned neural network model, reliability of the pruned neural network model, and distribution parameters of each layer in the pruned neural network model.

[0038] Secondly, this disclosure provides a neural network model pruning device.

[0039] Specifically, the neural network model pruning device includes:

[0040] The first acquisition module is configured to acquire environmental state information of the neural network model to be pruned through reinforcement learning. The environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned.

[0041] The second acquisition module is configured to acquire the target pruning rate of each layer in the neural network model to be pruned based on the environmental state information.

[0042] The third acquisition module is configured to obtain a target reward value based on the target pruning rate and the reward function. The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate.

[0043] The first execution module is configured to prune the neural network model to be pruned according to the target pruning rate in response to the target reward value being greater than or equal to a preset threshold.

[0044] The processing module is configured to perform image processing on the target image based on the target neural network model obtained after cropping.

[0045] In conjunction with the second aspect, in the first implementation of the second aspect of this disclosure, the neural network model pruning device further includes:

[0046] The fourth acquisition module is configured to obtain the reliability of the pruned neural network model based on the reliability evaluation function;

[0047] The fourth acquisition module is also configured to acquire the energy consumption of the pruned neural network model based on the energy consumption evaluation function;

[0048] The fifth acquisition module is configured to acquire the reward function based on the reliability of the pruned neural network model and the energy consumption of the pruned neural network model.

[0049] In conjunction with the second aspect and the first implementation of the second aspect, in the second implementation of the present disclosure, the neural network model pruning device further includes:

[0050] The sixth acquisition module is configured to acquire system architecture reliability parameters and neuron sensitivity parameters;

[0051] The second execution module is configured to construct the reliability evaluation function based on the architecture reliability parameters and the neuron sensitivity parameters;

[0052] The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

[0053] In conjunction with the second aspect, the first implementation of the second aspect, and the second implementation of the second aspect, in the third implementation of the second aspect of this disclosure, the sixth acquisition module is configured as follows:

[0054] according to Obtain the system architecture reliability parameter ARF i ;

[0055] Among them, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i represents the memory access amount of the i-th layer in the pruned neural network model, and All Params represents the memory access amount of all layers in the pruned neural network model.

[0056] In conjunction with the second aspect, the first implementation of the second aspect, and the second implementation of the second aspect, in the fourth implementation of the second aspect of this disclosure, the fourth acquisition module is configured as follows:

[0057] according to The reliability of the pruned neural network model (NRF) i ;

[0058] Among them, ARF i Let ΔE(h) be the reliability parameter of the system architecture. i ) represents the sensitivity parameter of the neuron, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i Let $\frac{1}{i}$ be the multiply-accumulate computation cost of the $i$-th layer in the pruned neural network model, and $\frac{AllMAC}{\frac{1}{2}$ be the multiply-accumulate computation cost of all layers in the pruned neural network model. $Params$ i Let represent the memory access amount of the i-th layer in the pruned neural network model, and AllParams represent the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

[0059] In conjunction with the second aspect and the first implementation of the second aspect, in the fifth implementation of the second aspect of this disclosure, the neural network model pruning device further includes:

[0060] The seventh acquisition module is configured to acquire the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model;

[0061] The third execution module is configured to construct the energy consumption evaluation function based on the full-load operation energy consumption and the bottleneck operation energy consumption.

[0062] In conjunction with the second aspect, the first implementation of the second aspect, and the fifth implementation of the second aspect, in the sixth implementation of the second aspect of this disclosure, the seventh acquisition module is configured as follows:

[0063] Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation;

[0064] Based on the first computational load and the first memory access load, the first estimated power is obtained;

[0065] The full-load operating energy consumption is obtained based on the first estimated power and the first running time of the pruned neural network model under full-load operation.

[0066] In conjunction with the second aspect, the first implementation of the second aspect, the second implementation of the second aspect, the third implementation of the second aspect, the fourth implementation of the second aspect, the fifth implementation of the second aspect, and the sixth implementation of the second aspect, in the seventh implementation of the second aspect of this disclosure, the basic feature information includes at least one of the following:

[0067] The number of layers in the neural network model to be pruned, the number of channels in the input feature map, the length of the input feature map, the width of the input feature map, the number of convolutional kernels, the length of the convolutional kernels, the width of the convolutional kernels, and the stride of the convolutional kernels.

[0068] In conjunction with the second aspect, the first implementation of the second aspect, the second implementation of the second aspect, the third implementation of the second aspect, the fourth implementation of the second aspect, the fifth implementation of the second aspect, and the sixth implementation of the second aspect, in the eighth implementation of the second aspect of this disclosure, the enhanced feature information includes at least one of the following:

[0069] Energy consumption of the pruned neural network model, reliability of the pruned neural network model, and distribution parameters of each layer in the pruned neural network model.

[0070] Thirdly, embodiments of this disclosure provide an electronic device including a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method as described in any one of 1 to 9.

[0071] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the method as described in any one of 1 to 9.

[0072] According to the technical solution provided in the embodiments of this disclosure, pruning the neural network model can not only reduce the computational load of the neural network model, but also reduce the impact of errors that may occur during the operation of the pruned neural network model on the entire system. This ensures that the reliability and energy consumption of the pruned neural network model meet the requirements, thereby improving the accuracy of image processing by the pruned neural network model.

[0073] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0074] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0075] Figure 1 A flowchart illustrating a neural network model pruning method according to an embodiment of the present disclosure is shown.

[0076] Figure 2 A structural block diagram of a neural network model pruning apparatus according to an embodiment of the present disclosure is shown;

[0077] Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0078] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the method according to embodiments of the present disclosure is shown. Detailed Implementation

[0079] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0080] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0081] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0082] In this disclosure, any operation involving the acquisition of user information or user data, or the display of user information or user data to others, is an operation authorized or confirmed by the user, or actively selected by the user.

[0083] As mentioned above, with the rapid development of neural network algorithms and hardware chips, applying neural networks to power systems has become a promising solution. Neural network models possess advantages such as powerful and fast parallel computing capabilities, high fault tolerance, and strong learning ability. Typically, neural network models require very high computational costs and storage space; however, neural network compression can reduce the number of parameters or storage space required.

[0084] Among related technologies, model pruning is a relatively mainstream compression scheme. Specifically, model pruning removes unimportant weights (connections) or neurons from a neural network model, significantly reducing the model size and computational cost without sacrificing model accuracy, thereby reducing energy consumption.

[0085] However, even if the neural network model is pruned according to the above scheme to reduce the computational load, the pruned neural network model may have a higher probability of errors in some cases. These errors will propagate layer by layer in the neural network model, affecting the output of the entire system and causing a significant decrease in the reliability of the system. Therefore, how to ensure that the reliability and energy consumption of the pruned neural network model can meet the requirements has become an urgent problem to be solved.

[0086] In view of the above-mentioned technical deficiencies, this disclosure provides a neural network model pruning method. Through reinforcement learning, environmental state information of the neural network model to be pruned is obtained. This environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned. Based on the environmental state information, a target pruning ratio is obtained for each layer of the neural network model to be pruned. Based on the target pruning ratio and a reward function, a target reward value is obtained. The reward function indicates the correspondence between the pruning ratio and the reliability and energy consumption of the pruned neural network model. The target reward value indicates the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning ratio. In response to the target reward value being greater than or equal to a preset threshold, the neural network model to be pruned is pruned according to the target pruning ratio. Finally, image processing is performed on the target image based on the pruned target neural network model. By pruning the neural network model using the above technical solution, not only can the computational load of the neural network model be reduced, but the impact of errors that may occur during the operation of the pruned neural network model on the entire system can also be reduced. This ensures that the reliability and energy consumption of the pruned neural network model meet the requirements, thereby improving the accuracy of image processing using the pruned neural network model.

[0087] Figure 1 A flowchart illustrating a neural network model pruning method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the neural network model pruning method may include the following steps S101-S105:

[0088] In step S101, the environmental state information of the neural network model to be pruned is obtained through reinforcement learning.

[0089] The environmental state information includes the basic feature information and enhanced feature information of the neural network model to be pruned.

[0090] In step S102, the target pruning rate of each layer in the neural network model to be pruned is obtained based on the environmental state information.

[0091] In step S103, the target reward value is obtained based on the target pruning rate and the reward function.

[0092] The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model, and the target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate.

[0093] In step S104, in response to the target reward value being greater than or equal to a preset threshold, the neural network model to be pruned is pruned according to the target pruning rate.

[0094] In step S105, the target image is processed according to the target neural network model obtained after cropping.

[0095] In one embodiment of this disclosure, the neural network model pruning method can be applied to computers, electronic devices, etc., for pruning neural network models.

[0096] Reinforcement learning, a branch of machine learning, can be viewed as a method of learning through exploration. In reinforcement learning, the learning agent is the primary learner, and the designer does not provide supervisory signals. Instead, the agent predicts its next action at each moment and receives a reward signal for each action in its interaction with the environment. By varying the levels of these reward signals, the agent gradually changes its behavior prediction rules to maximize the accumulated reward from a series of actions, thereby autonomously exploring the optimal solution to the target problem. For details, please refer to the detailed descriptions in related technologies; the embodiments disclosed herein will not be elaborated upon further.

[0097] In one possible application scenario within a power system, a series of image data collected from power equipment, along with a large amount of historical operational data and fault log data, can be used as a training dataset. This dataset can then be input into a neural network and trained using pre-defined target values ​​to obtain a trained neural network model, i.e., a neural network model to be pruned. The target values ​​can be manually set by technicians or derived from historical data. For example, the neural network model can be applied to power system transmission stability analysis, load detection, static and dynamic stability analysis, and fault prediction.

[0098] In another possible application scenario, neural network models can also be applied to the field of image processing, such as image recognition, edge detection, image segmentation, image compression, and image restoration.

[0099] In one embodiment of this disclosure, the neural network model to be pruned may include a deep neural network model, a convolutional neural network model, a recurrent neural network model, etc.

[0100] In one embodiment of this disclosure, the basic feature information can be understood as inherent to the network model to be pruned, and this information can be estimated offline.

[0101] In one embodiment of this disclosure, the basic feature information may include at least one of the following:

[0102] The number of layers in the neural network model to be pruned, the number of channels in the input feature map, the length of the input feature map, the width of the input feature map, the number of convolutional kernels, the length of the convolutional kernels, the width of the convolutional kernels, and the stride of the convolutional kernels.

[0103] In one embodiment of this disclosure, the enhanced feature information can represent the amount of change that is likely to occur in the neural network model. This amount of change will change with each iteration of the pruning rate and can be obtained online after each pruning of the neural network model to be pruned.

[0104] In one embodiment of this disclosure, the enhanced feature information may include at least one of the following:

[0105] Energy consumption of the pruned neural network model, reliability of the pruned neural network model, and distribution parameters of each layer in the pruned neural network model.

[0106] In one embodiment of this disclosure, the energy consumption and reliability of the pruned neural network model can both be understood as the energy consumption and reliability obtained through evaluation, as detailed in the following embodiments.

[0107] In one embodiment of this disclosure, the distribution parameters of each layer in the pruned neural network model can be understood as statistical parameters, which may include any of the following: maximum value, minimum value, median value, mean value, and variance.

[0108] In one embodiment of this disclosure, the enhanced feature information may further include information such as historical cropping rate.

[0109] In this implementation, after obtaining the environmental state information of the neural network model to be pruned, reinforcement learning can identify the proportion of unimportant convolutional kernels in each layer of the neural network model and generate the target pruning rate for each layer. Furthermore, the target pruning rate for each layer varies depending on the number of convolutional kernels.

[0110] In one embodiment of this disclosure, after obtaining the target pruning rate for each layer in the neural network model to be pruned, the number of convolutional kernels to be pruned in each layer can be determined based on the target pruning rate of each layer and the total number of convolutional kernels in the current layer. That is, the number of convolutional kernels to be pruned in each layer is determined under the current circumstances. The reliability of neurons in each convolutional layer is obtained and used as a score. Then, based on the target pruning rate, the convolutional kernels that should be pruned in each layer can be determined.

[0111] In this implementation, convolutional kernels with high fault tolerance in each layer of the neural network model to be pruned can be deleted sequentially, while convolutional kernels with low fault tolerance will be retained.

[0112] Optionally, the preset threshold can be understood as the maximum reward value, which can be user-defined or not derived from historical experience data.

[0113] In one embodiment of this disclosure, the reward function differs accordingly for the two different requirements of high reliability and high energy efficiency.

[0114] For example, to meet the high reliability requirement, the reliability of the neural network model should be improved as much as possible while satisfying the qualitative requirement of energy consumption. In this case, the reward function can be as shown in formula (1) below, thus obtaining the target reward value Reward1:

[0115]

[0116] Here, Etotal represents the energy consumed by the target neural network model under the target pruning rate, Etarget is the required target energy consumption value, and Ra represents the reliability assessment result of the target neural network model under the target pruning rate.

[0117] It should be noted that the above formula (1) uses squares to increase the penalty when energy consumption does not meet the conditions. At this time, reducing energy consumption can bring a large return. However, when energy consumption meets the conditions, the energy consumption condition term is set to 1. At this time, reducing energy consumption will not bring any benefit, and the benefit will come entirely from reliability.

[0118] For example, for the requirement of high energy efficiency (i.e., low energy consumption), while meeting the qualitative requirements of reliability, the energy efficiency of the neural network model should be improved as much as possible, that is, the energy consumption of the neural network model should be saved. At this time, the reward function can be as shown in the following formula (2), so that the target reward value Reward2 can be obtained:

[0119]

[0120] Among them, R total R represents the evaluation reliability of the target neural network model obtained under the target pruning rate. target This represents the target reliability of the requirement, E. a This represents the energy efficiency evaluation result of the target neural network model obtained after pruning at the target pruning rate. Similarly, in formula (2), the reliability is weighted by square to represent a higher optimization priority.

[0121] In one embodiment of this disclosure, the target image may be one or more images of any type acquired by a computing device.

[0122] Specifically, the image processing of the target image based on the target neural network model obtained after cropping may include image recognition, image segmentation, image restoration, and other processing methods, which are not limited in this embodiment.

[0123] In this embodiment, pruning the neural network model to be pruned according to the target pruning rate can be understood as compressing the computational load and storage space occupied by the neural network model to be pruned. This results in a significantly reduced computational load and storage space for the pruned target neural network model, enabling cost savings on the computing devices deploying the target neural network model, while also allowing the target neural network model to run faster and with higher accuracy. This ensures the normal operation of the neural network model deployed on the hardware platform and meets the requirements for reliability and energy consumption.

[0124] This disclosure provides a method for pruning a neural network model. Through reinforcement learning, it acquires environmental state information of the neural network model to be pruned, including basic and enhanced feature information. Based on this environmental state information, it obtains a target pruning rate for each layer of the model. A target reward value is obtained based on the target pruning rate and a reward function, where the reward function indicates the relationship between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value indicates the benefit of pruning the target neural network model based on the target pruning rate. In response to the target reward value being greater than or equal to a preset threshold, the neural network model is pruned according to the target pruning rate. Image processing is then performed on the target image based on the pruned target neural network model. This method not only reduces the computational load of the neural network model but also mitigates the impact of errors that may occur during the pruned model's operation on the entire system, ensuring that the reliability and energy consumption of the pruned neural network model meet requirements, thereby improving the accuracy of image processing using the pruned neural network model.

[0125] In one embodiment of this disclosure, before step S103, i.e., before obtaining the target reward value based on the target pruning rate and the reward function, the method may further include the following steps:

[0126] The reliability of the pruned neural network model is obtained based on the reliability evaluation function.

[0127] The energy consumption of the pruned neural network model is obtained based on the energy consumption evaluation function.

[0128] The reward function is obtained based on the reliability and energy consumption of the pruned neural network model.

[0129] In one embodiment of this disclosure, the reliability evaluation function is used to evaluate the reliability of the entire neural network model, and the energy consumption evaluation function is used to evaluate the energy consumption of the entire neural network model.

[0130] In one embodiment of this disclosure, energy consumption and reliability can be considered as two optimization objectives of the neural network model to be pruned. A balance between these two objectives is achieved by pruning the neural network model. The reliability and energy consumption of the neural network model to be pruned vary with the pruning rate. Thus, by pruning the neural network model according to the pruning rate of each layer, the energy consumption and reliability of the resulting pruned neural network model are minimized.

[0131] In one embodiment of this disclosure, both the reliability evaluation function and the energy consumption evaluation function are related to the pruning rate of each layer in the pruned neural network model.

[0132] In this disclosed embodiment, since the reliability and energy consumption of the pruned neural network model can be obtained according to the reliability evaluation function and the energy consumption evaluation function respectively, the reliability and energy consumption of the pruned neural network model can be evaluated by means of these two evaluation models, thereby achieving the optimization goal of the neural network model to be pruned.

[0133] In one embodiment of this disclosure, prior to the step of obtaining the reliability of the pruned neural network model based on the reliability evaluation function, the method may further include the following steps:

[0134] Obtain system architecture reliability parameters and neuron sensitivity parameters;

[0135] Based on the system architecture reliability parameters and the neuron sensitivity parameters, a reliability evaluation function is constructed.

[0136] The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

[0137] For example, consider a pruned neural network model called a DNN. Each layer in a DNN involves a huge amount of computation and memory access, which is a characteristic of hardware-based computation. When soft errors occur in the neural network at a certain failure rate, it will inevitably cause a large number of parameter errors. The percentage of time certain key neurons in a neural network remain in the network affects the reliability of the entire compressed model. Considering these characteristics, when an error occurs in a layer of the neural network, there is sufficient reason to believe that the reliability of the corresponding layer is closely related to the memory access, computation, and on-chip residence time of the current layer. Therefore, a parameter is proposed to evaluate the impact of soft errors in each layer on the reliability of the neural network model, namely the Architecture Reliability Factor (ARF).

[0138] In one embodiment of this disclosure, the step of obtaining system architecture reliability parameters may specifically include the following steps:

[0139] according to Obtaining the architecture reliability parameter ARF i .

[0140] Among them, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i represents the memory access amount of the i-th layer in the pruned neural network model, and All Params represents the memory access amount of all layers in the pruned neural network model.

[0141] It should be noted that BER in the above formula iThe larger the value of P, the more parameters in the pruned neural network model are prone to errors. P represents the computational density of the i-th layer in the pruned neural network model, and according to Amdell's Law, it is also closely related to hardware performance. For simplicity, in actual calculations, we use the ratio of the number of computation cycles to the number of memory access cycles to represent p. The larger p is, the more execution time the computation takes, and the more cycles the data resides during the computation process. In this case, the computational stage of the entire system is more prone to errors, and the overall system reliability deteriorates. Conversely, the same applies during memory access. Since each layer has different computational and memory access volumes, the magnitude of errors between different layers also varies. Layers with higher computational and memory access volumes have a much higher probability of failure than layers with lower computational and memory access volumes. Therefore, we use the proportion of memory access volume and computational volume in the entire model to represent its probability. In addition, MAC in the above formula... i and Params i The type of the i-th layer in the pruned neural network model is determined by the layer type of the i-th layer.

[0142] In this disclosed embodiment, computational load, memory access load, and runtime are incorporated into the factors affecting reliability, thereby evaluating the reliability of the pruned neural network model based on the system architecture reliability parameters.

[0143] In one embodiment of this disclosure, since different neurons in a neural network model have different vulnerabilities, the neuron sensitivity parameter ΔE(h) is obtained by analyzing the different vulnerabilities of different neurons in the pruned neural network model. i Specifically, ΔE(h) i It can be obtained through the following formula:

[0144]

[0145] in, The loss function in neuron h i The first-order partial derivative at Δh i This represents the magnitude of the change in a neuron before and after it was pruned.

[0146] It should be noted that ΔE(h) i The value () represents the sensitivity of a neuron. The sensitivity of a neural network mainly depends on the first-order partial derivative of the neuron and the magnitude of the corresponding neuron changes. The greater the sensitivity, the greater the change in the loss function caused by the neuron's change, and the worse the accuracy, and vice versa.

[0147] In this disclosed embodiment, a reliability evaluation function is constructed by combining system architecture reliability parameters and neuron sensitivity parameters, thereby facilitating the evaluation of the reliability of the pruned neural network model from multiple perspectives, including both software and system architecture levels.

[0148] In one embodiment of this disclosure, the step of obtaining the reliability of the pruned neural network model based on the reliability evaluation function may specifically include the following steps:

[0149] according to The reliability of the pruned neural network model (NRF) i .

[0150] Among them, ARF i Let ΔE(h) be the reliability parameter of the system architecture. i ) represents the sensitivity parameter of the neuron, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i Let represent the memory access amount of the i-th layer in the pruned neural network model, and All Params represent the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

[0151] In one embodiment disclosed, the NRF i This can be understood as an evaluation and analysis of the reliability of a single convolutional kernel in a pruned neural network model. If the reliability of a certain layer of the neural network model needs to be obtained, it can be obtained by the ratio of the sum of the reliability of all convolutional kernels in that layer to the total number of convolutional kernels in that layer.

[0152] In this disclosed embodiment, the reliability of the pruned neural network model can be obtained based on information such as the failure rate of each layer of the pruned neural network model, the computation-to-memory ratio in hardware operations, and the amount of multiplication-accumulation computation. This is a reliability result obtained through comprehensive analysis at both the software and system architecture levels, thereby improving the accuracy of reliability analysis.

[0153] In one embodiment of this disclosure, before the step of obtaining the energy consumption of the pruned neural network model based on the energy consumption evaluation function, the method may further include the following steps:

[0154] Obtain the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model;

[0155] The energy consumption evaluation function is constructed based on the full-load operation energy consumption and the bottleneck operation energy consumption.

[0156] In one embodiment of this disclosure, since neural network models often run on different hardware platforms in actual operation, and these hardware platforms have significantly different computing power, storage size and bandwidth resources, the same neural network model will also have significant energy consumption differences when running on different platforms. Therefore, it is necessary to obtain the energy consumption of the trimmed neural network model when running on the hardware system, that is, the energy consumption of full-load operation and the energy consumption of bottleneck operation.

[0157] In this disclosed embodiment, an energy consumption evaluation function is constructed by obtaining the full-load operation energy consumption and bottleneck operation energy consumption of the pruned neural network model, thereby facilitating a multi-faceted evaluation of the energy consumption of the pruned neural network model by combining the model scale level and hardware feature level.

[0158] In one embodiment of this disclosure, the step of obtaining the full-load operating energy consumption of the pruned neural network model may specifically include the following steps:

[0159] Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation;

[0160] Based on the first computational load and the first memory access load, the first estimated power is obtained;

[0161] Based on the first estimated power and the first running time of the pruned neural network model under full load operation, the full load operation energy consumption is obtained.

[0162] In one embodiment of this disclosure, different types of network layers in the neural network model correspond to different computational and memory access requirements, thereby allowing the computational and memory access requirements of all layers in the pruned neural network model to be obtained. Table 1 below shows the number of multiply-accumulate operations and memory access requirements corresponding to several common layer types. Based on this table, the computational and memory access requirements of each layer in the pruned neural network model can be estimated, and thus the computational and memory access requirements of the entire neural network model can be calculated.

[0163] Table 1. Number of multiply-accumulate operations and number of memory accesses for different types of network layers.

[0164] Layer type Multiplication and addition computational complexity Number of memory accesses Convolutional layer <![CDATA[O w *O h *O c *K w *K h *I c ]]> <![CDATA[O c *(K w *K h *I c +1)+(O w *O h *O c )]]> Fully connected layer <![CDATA[2*I n *THE m ]]> <![CDATA[I n *O m +2*O m ]]> Pooling sampling layer <![CDATA[O c *O h *O w *(K w *K h -1)]]> <![CDATA[O c *O h *O w ]]> Standardization layer <![CDATA[O c *O h *O w *(n+3)]]> <![CDATA[O c *O h *O w ]]> Activation layer <![CDATA[O m *(THE m +1)]]> <![CDATA[O m ]]> Nonlinear layer <![CDATA[O m ]]> <![CDATA[O m ]]>

[0165] In one embodiment of this disclosure, a Gaussian regression model can be used to fit the first computational load and the first memory access load to obtain the first estimated power.

[0166] In one embodiment of this disclosure, the full-load operating energy consumption is obtained by multiplying the first estimated power and the first operating time. The first operating time can be understood as the shortest time consumed in calculation and computation.

[0167] In one embodiment of this disclosure, when the hardware platform experiences a memory bottleneck, the bottleneck operating energy consumption can be understood as memory access energy consumption; when the hardware platform experiences a computing bottleneck, the bottleneck operating energy consumption can be understood as computing energy consumption.

[0168] It is understandable that different hardware platforms have different hardware characteristics, and consequently, different performance bottlenecks, resulting in different bottleneck operating power consumption.

[0169] For example, the energy consumption evaluation function can be expressed by the following formula, thereby obtaining the energy consumption y of the pruned neural network model. energy .

[0170] y energy =y common * y bound =P common *T common +P bound *T bound

[0171] Among them, y common The energy consumption during full-load operation, y bound P represents the energy consumption during bottleneck operation. common This represents the power under full-load operation (i.e., the first estimated function), T common This represents the runtime under full load (i.e., the first runtime), P bound T represents the power under bottleneck operating conditions. bound This represents the runtime of the bottleneck operation.

[0172] For the power under bottleneck operation conditions, please refer to the specific description of the first estimated power in the above embodiments, which will not be repeated in this disclosure.

[0173] In this disclosed embodiment, the computational and memory access costs of each layer in the pruned neural network model are calculated at the model scale level, and the power of the pruned neural network is predicted based on a Gaussian regression model, thereby improving the accuracy of energy consumption prediction.

[0174] Figure 2 A structural block diagram of a neural network model pruning device according to an embodiment of the present disclosure is shown. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0175] like Figure 2 As shown, the neural network model pruning device includes a first acquisition module 201, a second acquisition module 202, a third acquisition module 203, a first execution module 204, and a processing module 205.

[0176] The first acquisition module 201 is configured to acquire environmental state information of the neural network model to be pruned through reinforcement learning. The environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned.

[0177] The second acquisition module 202 is configured to acquire the target pruning rate of each layer in the neural network model to be pruned based on the environmental state information.

[0178] The third acquisition module 203 is configured to obtain a target reward value based on the target pruning rate and the reward function. The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate.

[0179] The first execution module 204 is configured to prune the neural network model to be pruned according to the target pruning rate in response to the target reward value being greater than or equal to a preset threshold.

[0180] The processing module 205 is configured to perform image processing on the target image based on the target neural network model obtained after cropping.

[0181] In one embodiment of this disclosure, the neural network model pruning device further includes:

[0182] The fourth acquisition module is configured to obtain the reliability of the pruned neural network model based on the reliability evaluation function;

[0183] The fourth acquisition module is also configured to acquire the energy consumption of the pruned neural network model based on the energy consumption evaluation function;

[0184] The fifth acquisition module is configured to acquire the reward function based on the reliability of the pruned neural network model and the energy consumption of the pruned neural network model.

[0185] In one embodiment of this disclosure, the neural network model pruning device further includes:

[0186] The sixth acquisition module is configured to acquire system architecture reliability parameters and neuron sensitivity parameters;

[0187] The second execution module is configured to construct the reliability evaluation function based on the architecture reliability parameters and the neuron sensitivity parameters;

[0188] The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

[0189] In one embodiment of this disclosure, the sixth acquisition module is configured as follows:

[0190] according to Obtain the system architecture reliability parameter ARF i ;

[0191] Among them, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i represents the memory access amount of the i-th layer in the pruned neural network model, and All Params represents the memory access amount of all layers in the pruned neural network model.

[0192] In one embodiment of this disclosure, the fourth acquisition module is configured as follows:

[0193] according to The reliability of the pruned neural network model (NRF) i ;

[0194] Among them, ARF i Let ΔE(h) be the reliability parameter of the system architecture. i ) represents the sensitivity parameter of the neuron, BER i Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations, and MAC is the maximum number of operations per second. i denoted as , where is the multiply-accumulate computation cost of the i-th layer in the pruned neural network model, and All MAC is the multiply-accumulate computation cost of all layers in the pruned neural network model. Params i Let represent the memory access amount of the i-th layer in the pruned neural network model, and All Params represent the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

[0195] In one embodiment of this disclosure, the neural network model pruning device further includes:

[0196] The seventh acquisition module is configured to acquire the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model;

[0197] The third execution module is configured to construct the energy consumption evaluation function based on the full-load operation energy consumption and the bottleneck operation energy consumption.

[0198] In one embodiment of this disclosure, the seventh acquisition module is configured as follows:

[0199] Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation;

[0200] Based on the first computational load and the first memory access load, the first estimated power is obtained;

[0201] The full-load operating energy consumption is obtained based on the first estimated power and the first running time of the pruned neural network model under full-load operation.

[0202] In one embodiment of this disclosure, the basic feature information includes at least one of the following:

[0203] The number of layers in the neural network model to be pruned, the number of channels in the input feature map, the length and width of the input feature map, the number of convolutional kernels, the length and width of the convolutional kernels, and the stride of the convolutional kernels.

[0204] In one embodiment of this disclosure, the enhanced feature information includes at least one of the following:

[0205] Energy consumption of the pruned neural network model, reliability of the pruned neural network model, and distribution parameters of each layer in the pruned neural network model.

[0206] This disclosure provides a neural network model pruning device that can acquire environmental state information of a neural network model to be pruned through reinforcement learning. The environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned. Based on the environmental state information, a target pruning rate is obtained for each layer of the neural network model to be pruned. A target reward value is obtained based on the target pruning rate and a reward function, where the reward function indicates the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value indicates the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate. In response to the target reward value being greater than or equal to a preset threshold, the neural network model to be pruned is pruned according to the target pruning rate. Image processing is performed on the target image based on the pruned target neural network model. Pruning the neural network model using this device not only reduces the computational load of the neural network model but also reduces the impact of errors that may occur during the operation of the pruned neural network model on the entire system, thereby ensuring that the reliability and energy consumption of the pruned neural network model meet the requirements, and improving the accuracy of image processing using the pruned neural network model.

[0207] This disclosure also discloses an electronic device. Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0208] like Figure 3 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to embodiments of the present disclosure.

[0209] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing the method according to embodiments of the present disclosure is shown.

[0210] like Figure 4 As shown, the computer system includes a processing unit that can execute various methods described above based on a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer system. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0211] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks, etc.; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processes via a network such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as needed. The processing unit can be implemented as a CPU, GPU, TPU, FPGA, NPU, etc.

[0212] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0213] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0214] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0215] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in this disclosure.

[0216] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for pruning a neural network model, characterized in that, The method includes: By using reinforcement learning, environmental state information of the neural network model to be pruned is obtained. This environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned. The basic feature information includes at least one of the following: the number of layers in the neural network model to be pruned, the number of channels in the input feature map, the length of the input feature map, the width of the input feature map, the number of convolutional kernels, the length of the convolutional kernels, the width of the convolutional kernels, and the stride of the convolutional kernels. The enhanced feature information includes at least one of the following: the energy consumption of the pruned neural network model, the reliability of the pruned neural network model, and the distribution parameters of each layer in the pruned neural network model. Based on the environmental state information, the target pruning rate of each layer in the neural network model to be pruned is obtained; Based on the target pruning rate and the reward function, a target reward value is obtained. The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate. In response to the target reward value being greater than or equal to a preset threshold, the neural network model to be pruned is pruned according to the target pruning rate; Image processing is performed on the target image based on the target neural network model obtained after cropping; Before obtaining the target reward value based on the target pruning rate and the reward function, the method further includes: Obtain system architecture reliability parameters and neuron sensitivity parameters; Based on the system architecture reliability parameters and the neuron sensitivity parameters, the reliability evaluation function is constructed. The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

2. The method according to claim 1, characterized in that, Before obtaining the target reward value based on the target pruning rate and the reward function, the method further includes: The reliability of the pruned neural network model is obtained based on the reliability evaluation function. Based on the energy consumption evaluation function, obtain the energy consumption of the pruned neural network model; The reward function is obtained based on the reliability and energy consumption of the pruned neural network model.

3. The method according to claim 1, characterized in that, The acquisition of system architecture reliability parameters includes: according to Obtain the system architecture reliability parameters ; in, Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations. The computational cost of multiplying and adding in the i-th layer of the pruned neural network model. This represents the computational cost of multiplying and adding all layers in the pruned neural network model. This represents the memory access amount of the i-th layer in the pruned neural network model. This represents the memory access amount for all layers in the pruned neural network model.

4. The method according to claim 1, characterized in that, The step of obtaining the reliability of the pruned neural network model based on the reliability evaluation function includes: according to The reliability of the pruned neural network model is obtained. ; in, These are the reliability parameters of the system architecture. The sensitivity parameter of the neuron, Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations. The computational cost of multiplying and adding in the i-th layer of the pruned neural network model. This represents the computational cost of multiplying and adding all layers in the pruned neural network model. This represents the memory access amount of the i-th layer in the pruned neural network model. This represents the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

5. The method according to claim 2, characterized in that, Before obtaining the energy consumption of the pruned neural network model based on the energy consumption evaluation function, the method further includes: Obtain the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model; The energy consumption evaluation function is constructed based on the full-load operation energy consumption and the bottleneck operation energy consumption.

6. The method according to claim 5, characterized in that, The process of obtaining the full-load operating energy consumption of the pruned neural network model includes: Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation; Based on the first computational load and the first memory access load, the first estimated power is obtained; The full-load operating energy consumption is obtained based on the first estimated power and the first running time of the pruned neural network model under full-load operation.

7. A neural network model pruning device, characterized in that, The neural network model pruning device includes: The first acquisition module is configured to acquire environmental state information of the neural network model to be pruned through reinforcement learning. The environmental state information includes basic feature information and enhanced feature information of the neural network model to be pruned. The basic feature information includes at least one of the following: the number of layers of the neural network model to be pruned, the number of channels of the input feature map, the length of the input feature map, the width of the input feature map, the number of convolutional kernels, the length of the convolutional kernel, the width of the convolutional kernel, and the stride of the convolutional kernel. The enhanced feature information includes at least one of the following: the energy consumption of the pruned neural network model, the reliability of the pruned neural network model, and the distribution parameters of each layer in the pruned neural network model. The second acquisition module is configured to acquire the target pruning rate of each layer in the neural network model to be pruned based on the environmental state information. The third acquisition module is configured to obtain a target reward value based on the target pruning rate and the reward function. The reward function is used to indicate the correspondence between the pruning rate and the reliability and energy consumption of the pruned neural network model. The target reward value is used to indicate the benefit of the target neural network model obtained after pruning the neural network model to be pruned according to the target pruning rate. The first execution module is configured to prune the neural network model to be pruned according to the target pruning rate in response to the target reward value being greater than or equal to a preset threshold. The processing module is configured to perform image processing on the target image based on the target neural network model obtained after cropping; The neural network model pruning device also includes: The sixth acquisition module is configured to acquire system architecture reliability parameters and neuron sensitivity parameters; The second execution module is configured to construct the reliability evaluation function based on the architecture reliability parameters and the neuron sensitivity parameters; The system architecture reliability parameter is used to evaluate the impact of each layer failure in the pruned neural network model on the reliability of the pruned neural network model, and the neuron sensitivity parameter is used to evaluate the impact of neuron failure in the pruned neural network model on the reliability of the entire pruned neural network model.

8. The apparatus according to claim 7, characterized in that, The neural network model pruning device also includes: The fourth acquisition module is configured to obtain the reliability of the pruned neural network model based on the reliability evaluation function; The fourth acquisition module is also configured to acquire the energy consumption of the pruned neural network model based on the energy consumption evaluation function; The fifth acquisition module is configured to acquire the reward function based on the reliability of the pruned neural network model and the energy consumption of the pruned neural network model.

9. The apparatus according to claim 8, characterized in that, The sixth acquisition module is configured as follows: according to Obtain the system architecture reliability parameters ; in, Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations. The computational cost of multiplying and adding in the i-th layer of the pruned neural network model. This represents the computational cost of multiplying and adding all layers in the pruned neural network model. This represents the memory access amount of the i-th layer in the pruned neural network model. This represents the memory access amount for all layers in the pruned neural network model.

10. The apparatus according to claim 8, characterized in that, The fourth acquisition module is configured as follows: according to The reliability of the pruned neural network model is obtained. ; in, These are the reliability parameters of the system architecture. The sensitivity parameter of the neuron, Let p represent the failure rate of the pruned neural network model at layer i, where p represents the computation-to-memory ratio of the pruned neural network model at layer i in hardware operations. The computational cost of multiplying and adding in the i-th layer of the pruned neural network model. This represents the computational cost of multiplying and adding all layers in the pruned neural network model. This represents the memory access amount of the i-th layer in the pruned neural network model. This represents the memory access amount of all layers in the pruned neural network model. This refers to the sensitivity parameter of the neuron.

11. The apparatus according to claim 9, characterized in that, The neural network model pruning device also includes: The seventh acquisition module is configured to acquire the full-load running energy consumption and bottleneck running energy consumption of the pruned neural network model; The third execution module is configured to construct the energy consumption evaluation function based on the full-load operation energy consumption and the bottleneck operation energy consumption.

12. The apparatus according to claim 11, characterized in that, The seventh acquisition module is configured as follows: Obtain the first computational cost and first memory access cost of the pruned neural network model under full load operation; Based on the first computational load and the first memory access load, the first estimated power is obtained; The full-load operating energy consumption is obtained based on the first estimated power and the first running time of the pruned neural network model under full-load operation.

13. An electronic device, characterized in that, It includes a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any one of claims 1 to 6.

14. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by a processor, the computer instructions implement the method steps of any one of claims 1 to 6.

Images