Neural network pruning method for pattern recognition based on compressed excitation attention

Abstract

The application discloses a neural network pruning method for mode recognition based on compressed excitation attention, and comprises the following steps: acquiring an EfficientNet for image processing, inserting a soft threshold mask into each compressed excitation attention module, setting a regularization coefficient, and calculating a weighted regularization coefficient; training the soft threshold mask, calculating a loss function, and updating the soft threshold mask; binarizing the trained soft threshold mask, pruning inverted residual bottleneck modules in the EfficientNet network through the mask, and obtaining a pruned EfficientNet network model; pruning non-inverted residual bottleneck modules in the EfficientNet network model through a method based on a batch normalization scale factor and an indirect pruning mode, obtaining a pruning model, and processing images through the pruning model. Through the pruning operation on the EfficientNet network, the pruned EfficientNet network model is obtained, so that the pruned model can be used to detect and process images more conveniently and quickly.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and particularly relates to a neural network pruning method, device and equipment for pattern recognition based on compressed excitation attention and a storage medium. BACKGROUND

[0002] In recent years, deep learning technology has developed rapidly, and the parameter quantity of neural networks used in the field of computer vision and the like is increasing. These huge models need to occupy a large amount of computing resources, and are difficult to deploy on mobile edge devices, so pruning operation needs to be performed on the model to reduce the parameter quantity and the computation of the model and reduce the training and inference cost of the model.

[0003] EfficientNet is a multi-dimensional mixed scaling network combined with neural network search technology, and the EfficientNet series network is obtained by adjusting the depth, width, resolution and loss rate on the basis of EfficientNet-B0. Although the performance of EfficientNet in the field of computer vision has surpassed many hand-designed neural networks, there is still a problem of large computational overhead, and the lightest EfficientNet-B0 in some pattern recognition fields also has the problem of over-parameterization. Most of the current pruning methods are mainly for ResNet and VGG and the like, and few of them are for pruning EfficientNet and the like. The existing pruning algorithm needs to be changed a lot when applied to EfficientNet, and it is not convenient for pruning EfficientNet.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The technical problem to be solved by the present application is that, in the field of computer vision, EfficientNet has the problem of over-parameterization when performing image processing, which leads to large computational overhead. The present application provides a neural network pruning method, device and equipment for pattern recognition based on compressed excitation attention and a storage medium. The present application can analyze the importance of channels by training a mask and compressed excitation attention in EfficientNet and perform soft pruning. The present application does not need to introduce an additional attention module, and adjusts the value of the regularization coefficient according to the parameter quantity of different modules, so as to more finely control the mask sparsification, so that the present application can conveniently and quickly obtain an EfficientNet network model with higher compression rate. The higher compression rate of the EfficientNet network model after pruning is used to process images to achieve the effect of smaller computational overhead.

[0006] To solve the above problems in the prior art, an embodiment of the present application provides a neural network pruning method for pattern recognition based on compressed excitation attention, which comprises the following steps:

[0007] obtaining a first EfficientNet network for image processing, inserting a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network, setting a regularization coefficient, and calculating a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module;

[0008] training the soft threshold mask, calculating a loss function based on the training result, the regularization coefficient and the weighted regularization coefficient, updating the soft threshold mask through the loss function, and stopping the training when the number of training rounds reaches a specified number of rounds;

[0009] binarizing the trained soft threshold mask, performing a modular pruning strategy on the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, and obtaining a third EfficientNet network model after pruning;

[0010] pruning the non-inverted residual bottleneck module in the third EfficientNet network model through a method based on a batch normalization scale factor and an indirect pruning manner to obtain a pruning model;

[0011] controlling the pruning model to process an input image.

[0012] When the first EfficientNet network for image processing is obtained, the weight of a second EfficientNet model with the same structure as the first EfficientNet network is obtained, which is trained on a large-scale image classification dataset ImageNet, and the first EfficientNet network loads the weight as the initial weight of the first EfficientNet network.

[0013] The trainable soft threshold mask inserted into each compressed excitation attention module in the first EfficientNet network, the setting of the regularization coefficient, and the calculation of the weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module specifically comprise the following steps:

[0014] The inserted trainable soft threshold mask is initialized as a full 1 vector;

[0015] The regularization coefficient is a pre-set value;

[0016] The weighted regularization coefficient is calculated through the parameter quantity of the inverted residual bottleneck module in the first EfficientNet network;

[0017] The regularization coefficient and the weighted regularization coefficient do not change in the soft threshold mask training process.

[0018] The soft threshold mask is trained, and a loss function is calculated based on the training result, the regularization coefficient and the weighted regularization coefficient. The soft threshold mask is updated through the loss function, and the training is stopped when the number of training rounds reaches a specified number of rounds, specifically including:

[0019] The first EfficientNet network inserted with the trainable soft threshold mask is trained on a target image classification dataset, and a loss function is calculated based on the training result, the regularization coefficient and the weighted regularization coefficient. The soft threshold mask is updated through the loss function and backpropagated, and the target image classification dataset is trained again;

[0020] When the number of training rounds reaches a specified number of rounds, the training is stopped.

[0021] The trained soft threshold mask is binarized, and a modular pruning strategy is performed on the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, to obtain a third EfficientNet network model after pruning, specifically including:

[0022] A threshold is set in advance according to the pruning requirement, and the soft threshold mask is binarized through the threshold to obtain a binarized soft threshold mask. The binarized soft threshold mask is used to perform a modular pruning strategy on each inverted residual bottleneck module in the first EfficientNet network. The compression excitation attention module, the channel-wise convolution, the point-wise convolution and the dilated convolution in the inverted residual bottleneck module are sequentially pruned to obtain a third EfficientNet network model after pruning.

[0023] The non-inverted residual bottleneck module in the third EfficientNet network model is pruned by a method based on a batch normalization scale factor and an indirect pruning method to obtain a pruning model, specifically including:

[0024] The backbone convolution layer in the third EfficientNet network model is pruned by an indirect pruning method;

[0025] The feature integration convolution layer and the fully connected layer in the third EfficientNet network model are pruned by a method based on a batch normalization scale factor.

[0026] The non-inverted residual bottleneck module in the third EfficientNet network model is pruned by a method based on a batch normalization scale factor and an indirect pruning method to obtain a pruning model, specifically including:

[0027] The loss function is optimized on the target image classification dataset, the pruned model is fine-tuned, and when the number of training rounds reaches the manually specified number of rounds, the fine-tuning is stopped, and the fine-tuned pruned model is output.

[0028] The second aspect of the embodiment of the application provides a device for pruning a neural network based on compressed excitation attention for pattern recognition, and the device comprises:

[0029] The pruning pre-module obtains a first EfficientNet network used for image processing, inserts a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network, sets a regularization coefficient, and calculates a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module;

[0030] The training module trains the soft threshold mask, and calculates a loss function based on the training result, the regularization coefficient and the weighted regularization coefficient, updates the soft threshold mask through the loss function, and stops training when the number of training rounds reaches a specified number of rounds;

[0031] The inverted residual bottleneck module pruning module binarizes the trained soft threshold mask, prunes the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, and obtains a third EfficientNet network model after pruning;

[0032] The non-inverted residual bottleneck module pruning module prunes the non-inverted residual bottleneck module in the third EfficientNet network model through a method based on a batch normalization scale factor and an indirect pruning manner, obtains a pruning model, and controls the input image to be processed through the pruning model.

[0033] The third aspect of the embodiment of the application provides a terminal device, characterized in that the terminal device comprises a memory, a processor and a neural network pruning method for pattern recognition based on compressed excitation attention stored on the memory and executable on the processor, and the processor executes the neural network pruning method for pattern recognition based on compressed excitation attention, and the steps of the neural network pruning method for pattern recognition based on compressed excitation attention are realized.

[0034] The fourth aspect of the embodiment of the application provides a computer readable storage medium, characterized in that a neural network pruning method for pattern recognition based on compressed excitation attention is stored on the computer readable storage medium, and when the neural network pruning method for pattern recognition based on compressed excitation attention is executed by a processor, the steps of the neural network pruning method for pattern recognition based on compressed excitation attention are realized.

[0035] Beneficial effects: compared with the prior art, the present application provides a neural network pruning method, device, equipment and storage medium for pattern recognition based on compression excitation attention, the method comprising: acquiring a first EfficientNet network for image processing, inserting a trainable soft threshold mask into each compression excitation attention module in the first EfficientNet network, setting a regularization coefficient, and calculating a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module; training the soft threshold mask, and calculating a loss function based on the training result, the regularization coefficient and the weighted regularization coefficient; updating the soft threshold mask through the loss function, and stopping training when the training round number reaches a specified round number; binarizing the trained soft threshold mask, and performing a modular pruning strategy on the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, to obtain a third EfficientNet network model after pruning; and pruning the non-inverted residual bottleneck module in the third EfficientNet network model through a method based on a batch normalization scale factor and an indirect pruning manner, to obtain a pruning model, and processing an image through the pruning model. In the field of computer vision, for EfficientNet, the present application can combine the advantages of the mask-based and attention-based pruning algorithms, use the soft threshold mask and the attention mechanism to guide pruning, fully utilize the compression excitation attention in EfficientNet, enable the present application to analyze channel importance through attention, and enable the present application to perform soft pruning after training through the mask, without the need to introduce an additional attention module, adjust the value of the regularization coefficient according to the parameter quantity of different modules, control the mask sparsification in a more fine-grained manner, adopt a modular pruning strategy, avoid indirect pruning between most modules, and enable the present application to conveniently and quickly obtain an EfficientNet network model with a higher compression ratio, so as to realize the effect of smaller calculation overhead and less memory occupation in image processing through the pruning model after pruning. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flowchart of the neural network pruning method for pattern recognition based on compression excitation attention provided by the present application;

[0038] Figure 2 The pruning algorithm workflow provided by the embodiment of the present application;

[0039] Figure 3 A structure diagram of the inverted residual bottleneck module in EfficientNet provided for an embodiment of the present application is shown in FIG. 1.

[0040] Figure 4 A structure diagram of the inverted residual bottleneck module in the present application after inserting a soft threshold mask provided for an embodiment of the present application is shown in FIG. 2.

[0041] Figure 5 A modular pruning workflow diagram provided for an embodiment of the present application is shown in FIG. 3.

[0042] Figure 6 A design diagram of pruning different modules of EfficientNet provided for an embodiment of the present application is shown in FIG. 4.

[0043] Figure 7 A principle block diagram of a neural network pruning device for pattern recognition based on compression excitation attention provided for an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0044] The present application provides a neural network pruning method, device, equipment and storage medium for pattern recognition based on compression excitation attention. In order to make the purpose, technical solution and effect of the present application more clear and explicit, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0045] It can be understood by those skilled in the art that, unless specifically stated, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the use of the phrase "comprising" in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, the "connection" or "coupling" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0046] As used herein, including the claims, the singular forms "a," "an," and "the" include plural reference unless the context clearly dictates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0047] In addition, if the description of the embodiments of the present application involves "first", "second", etc., the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of the ordinary skilled in the art, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0048] In recent years, the deep learning technology has developed rapidly, and the neural network parameter quantity used in the field of computer vision and other fields is increasing. These huge models need to occupy a large amount of computing resources, and it is difficult to deploy on mobile edge devices, so it is necessary to perform pruning operation on the model to reduce the model parameter quantity and calculation quantity, and reduce the model training and inference cost. In addition, in some simple pattern recognition tasks, the commonly used computer vision model has the problem of excessive network parameter quantity, and it is time-consuming and laborious to design a special model, so it is necessary to use a pruning algorithm to remove redundant parameters, and generate a lightweight special model for the task.

[0049] The neural network pruning algorithm refers to analyzing the importance of neurons of a neural network, statically or dynamically removing neurons with less contribution, and achieving the effect of reducing the parameter quantity of the entire network. Among them, the importance-based pruning algorithm establishes some metrics that reflect the relative importance between weights as much as possible, prunes the trained model according to the metrics, and the metrics usually do not participate in training, such as the absolute value of the weight, the 1-norm of the weight, etc.; the feature map-based pruning algorithm collects feature maps generated by data passing through the network layer in a data-driven manner, and then calculates relevant statistical information as a judgment index for pruning; the automatic machine learning-based pruning algorithm uses reinforcement learning and the like to automatically give the network pruning scheme, without manually setting the pruning proportion of each layer; the pruning algorithm based on the attention mechanism inserts an attention module into the network to be pruned, the attention module is trained with the network, and after training, the network is pruned according to the attention module, and the attention module is removed for fine tuning to restore the performance; the pruning algorithm based on the mask uses a mask to measure the importance between the weights in the network layer, the mask is optimized with the network to dynamically adjust the output of the network layer, and after training, the channels or filters of the specified threshold are pruned according to the mask, which is better than the importance-based pruning algorithm. In addition, some pruning algorithms use dynamic pruning, different pruning networks are used for different inputs, and dynamic pruning obtains a subnet set of the original network. During inference, different subnets are selected according to the input data to reduce the floating point operations per second (FLOPs).

[0050] EfficientNet is a multi-dimensional mixed scaling network combined with neural network search technology. The EfficientNet series network is obtained by adjusting the depth, width, resolution and loss rate on the basis of EfficientNet-B0.

[0051] EfficientNet is composed of multiple inverted residual bottleneck modules (Mobile Inverted Residual Bottleneck Block, abbreviated as MB Conv, hereinafter referred to as MB Conv) modules and other separate convolution layers and full connection layers. A MB Conv module contains multiple convolution layers. In order to simplify, BN (Batch Normalization), activation function and the like network layers and feature map size are ignored, and the input and output are used to express the effect of the MB Conv module. Specifically, as shown in Figure 3 , it is an inverted residual bottleneck module structure diagram in EfficientNet provided by the embodiment of the application. The module structure includes five parts: expansion convolution, channel-wise convolution, dimension reduction convolution, dimension increase convolution and point-wise convolution. Assuming that the channel of the feature map input into the MB Conv module is C in , the channel of the feature map output by the MB Conv module is Cout The parameters of each convolution kernel can be represented in the form of a graph, where C E represents the number of channels after expansion, C E The width of the network is adjusted by the scale factor ws of EfficientNet, and the value of ws is determined by the network structure. The calculation formula is C E = C in × ws; and C R represents the intermediate conversion channel in the excitation operation of the compressed excitation attention module, C R is controlled by SERatio, and SERatio is a fixed value of 0.25. The calculation formula is C R = C E × SERatio. The parameters of the 5 convolution layers of the MB Conv module depend on the input and output channels of the MB Conv module. From the code level, the internal implementation of the module is controlled by specifying the input and output channels of the MB Conv module. From the macroscopic point of view, the MB Conv module shields the details of the convolution operation in the network; EfficientNet is stacked by a MB Conv module, and the MB Conv module can be regarded as a fusion layer of “convolution layer + BN + activation function”. Let W i represent the parameter amount of the i-th MB Conv module, W j i represent the parameter amount of the j-th convolution of the i-th MB Conv module. Because the dilated convolution does not necessarily appear in each MB Conv module, the parameters of the i-th MB Conv can be represented by the following formula:

[0052]

[0053] Although the performance of EfficientNet has surpassed many hand-designed neural networks in the field of computer vision, it still has the problem of large computational overhead. In some pattern recognition fields, the lightest EfficientNet-B0 also has the problem of over-parameterization. Therefore, model pruning is needed to reduce the overhead. Most of the current pruning methods are mainly aimed at ResNet and VGG and other hand-designed models, and few are aimed at pruning EfficientNet and other lightweight networks. The existing pruning algorithm will have a lot of changes when applied to EfficientNet, such as special processing of shortcut connections and compressed excitation layers, and a lot of time is needed to search for the pruning rate of each layer.

[0054] In order to solve the problem of over-parameterization of EfficientNet in image processing in the field of computer vision at present, the present application provides a neural network pruning method for pattern recognition based on compressed excitation attention, which can be executed by a neural network pruning device for pattern recognition based on compressed excitation attention. The device can be realized by software or hardware, and can be applied to intelligent terminal equipment such as tablet computers, computers and the like installed with an operating system. In the embodiment of the application, the importance of the channel is analyzed and soft pruning is performed through the trained mask and the compressed excitation attention in the EfficientNet, which does not need to introduce additional attention modules, and the value of the regularization coefficient is adjusted according to the parameter size of different modules, so as to control the mask sparsification more finely, so that the present application can conveniently and quickly obtain an EfficientNet network model with higher compression ratio.

[0055] Example method

[0056] As shown in Figure 1 The embodiment of the present application provides a flowchart of the neural network pruning method for pattern recognition based on compressed excitation attention, which can be applied to intelligent terminal equipment. In the embodiment of the present application, the method comprises the following steps:

[0057] Step S10, obtaining a first EfficientNet network for image processing, inserting a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network, setting a regularization coefficient, and calculating a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module;

[0058] Specifically, the structured pruning of the present application is performed after the neural network training is completed, and a modular pruning strategy is adopted, so that each MB Conv module has a soft threshold mask for representing the pruning of the module, so that after obtaining the first EfficientNet network, for example, EfficientNet-B0, a trainable soft threshold mask is inserted into each compressed excitation attention module in the first EfficientNet network, which is used to realize the pruning effect after training, and a regularization coefficient is set, and a weighted regularization coefficient is calculated according to the parameter quantity of each inverted residual bottleneck module; wherein the first EfficientNet network refers to the EfficientNet network for pattern recognition.

[0059] Further, the first EfficientNet network is obtained by loading the weights of a second EfficientNet model trained on a large-scale image classification dataset ImageNet, the second EfficientNet model having the same structure as the first EfficientNet network.

[0060] Specifically, the first EfficientNet network is obtained by loading the weights of a second EfficientNet model trained on a large-scale image classification dataset ImageNet, the second EfficientNet model having the same structure as the first EfficientNet network.

[0061] By loading the weights, the first EfficientNet network adopts the weights of the second EfficientNet model trained on the large-scale image classification dataset ImageNet, and the first and second EfficientNet networks have the same structure, which ensures that the first EfficientNet network can successfully load the weights of the second EfficientNet network.

[0062] Further, the first EfficientNet network is obtained by loading the weights of a second EfficientNet model trained on a large-scale image classification dataset ImageNet, the second EfficientNet model having the same structure as the first EfficientNet network.

[0063] The inserted trainable soft threshold mask is initialized as a full 1 vector.

[0064] The regular coefficient is a preset value.

[0065] The weighted regular coefficient is calculated based on the parameter amount of the inverted residual bottleneck module in the first EfficientNet network.

[0066] The regular coefficient and the weighted regular coefficient do not change during the training of the soft threshold mask.

[0067] Specifically, the compression excitation attention module models the dependency relationship between channels and can adaptively adjust the feature response value of each channel. The compression excitation attention module uses a Sigmoid activation function, and the activation value is in the range of 0-1. When the activation value acts on the output of the network layer before the compression excitation attention module, some output channels with an activation value close to 0 are equivalent to being disabled, and the contribution to the subsequent network is reduced, so it can be regarded as a kind of pruning. Therefore, the compression excitation attention module can be used to guide pruning, and in order to obtain sparse activation values, the present application uses L1 regularization to constrain the compression excitation attention module, and at the same time, the present application inserts a trainable soft threshold mask into the compression excitation attention module. The soft threshold mask is initialized as a full 1 vector and is updated by gradient descent method during the training process. Assuming that the expansion convolution output of the compression excitation attention module is O se , then the output O of the pruned compression excitation attention module can be expressed as:

[0068]

[0069] , wherein, represents a trainable soft threshold mask, i represents the position of the corresponding convolution channel, Sigmoid is an activation function, and ||*||1 is an L1 regularization calculation.

[0070] After inserting the trainable soft threshold mask into each compression excitation attention module in the first EfficientNet network, the position of the mask insertion is as shown in Figure 4 , and the soft threshold mask is added in the channel-by-channel convolution, dimension increasing convolution and point-by-point convolution. After adding the soft threshold mask, the loss function of the network is

[0071]

[0072] , wherein a is an l1 regularization coefficient, which controls the sparsification intensity, and is usually taken as 10 -4 to 10 -5 , and a does not change in the subsequent soft threshold mask training process; loss is usually a cross-entropy loss function, x represents the input data of the network, y represents the true label of the input data, f(W, x) represents the prediction result obtained by calculating the input data through the neural network, and W is the model parameter.

[0073] The current design of neural networks tends to gradually increase the number of channels from shallow to deep, and intuitively, deep networks should have more redundant parts than shallow networks, and the results of some pruning algorithms also show that deep networks have a higher pruning rate. Therefore, in addition to using the L1 regularization coefficient alpha to control the strength of regularization, the present application also weights the parameter amount of the different compression excitation attention modules to obtain beta and alpha to jointly control the strength of regularization, and beta is calculated as follows:

[0074]

[0075] Where beta i is the weighted regularization coefficient of the i-th MB Conv module, which does not change during the soft threshold mask training process, W i represents the i-th MB Conv module, in order to reduce the calculation, only the dimensionality increasing convolution of the compression excitation attention module is counted, which is also the fourth convolution layer in the MB Conv module, because the two convolutions of the compression excitation attention module are symmetric structures, so the parameter amount of the dilated convolution can be used to replace the parameter amount of the entire compression excitation attention module. The beta parameter is calculated before training and remains unchanged during training. After adding the beta parameter, the loss is represented by the following formula:

[0076]

[0077] Step S20, training the soft threshold mask, and calculating the loss function based on the training result, the regularization coefficient and the weighted regularization coefficient, updating the soft threshold mask through the loss function, and stopping training when the training round number reaches the specified round number.

[0078] Specifically, in the training phase, the loss function is obtained by training, the soft threshold mask is updated by the loss function, and the soft threshold mask of the compression excitation attention module is updated by back propagation, and then the soft threshold mask is synchronized to the mask of the channel-by-channel convolution and the point-by-point convolution. Among them, since the filter mask of the compression excitation attention module also limits the input and output of the channel-by-channel convolution and the point-by-point convolution, the mask of the channel-by-channel convolution and the point-by-point convolution cannot be trained during training, but the mask is updated by back propagation after the mask in the dimensionality increasing convolution is trained. In this way, the compression excitation attention module will not act on the output of the channel-by-channel convolution and the point-by-point convolution, so that the update of the pruning soft threshold mask of the channel-by-channel convolution and the point-by-point convolution will not cause the loss of some important channels, and it also ensures that the channel-by-channel convolution and the point-by-point convolution can follow the compression excitation attention module to gradually sparsify, in addition, the values of the soft threshold masks in the same MB Conv module are kept consistent.

[0079] Further, the soft threshold mask is trained, and a loss function is calculated based on a training result, a regularization coefficient and a weighted regularization coefficient, the soft threshold mask is updated through the loss function, and the training is stopped when a training round number reaches a specified round number, and the specific method comprises the following steps:

[0080] The first EfficientNet network with the trainable soft threshold mask is trained on a target image classification data set, a loss function is calculated based on a training result, a regularization coefficient and a weighted regularization coefficient, the soft threshold mask is updated through the loss function and the soft threshold mask is back propagated, and the first EfficientNet network is trained again on the target image classification data set;

[0081] The training is stopped when the training round number reaches the specified round number.

[0082] Specifically, as shown in the figure, the modular pruning workflow provided by the embodiment of the application is shown, and the specific process of the training stage is as follows: Figure 5

[0083] Step S511, back propagation;

[0084] Step S512, updating the soft threshold mask;

[0085] Step S513, copying the soft threshold mask;

[0086] Specifically, the first EfficientNet network with the trainable soft threshold mask is trained on a target image classification data set, a loss function is optimized on the target image classification data set based on a gradient descent method, and the loss function is calculated based on a training result, a regularization coefficient and a weighted regularization coefficient, that is,

[0087]

[0088] The soft threshold mask in the dimension-up convolution is optimized through the loss function, the mask is back propagated and the model weight is updated, so that the effect of updating the mask of other convolutions is realized, and then the mask is copied for the next training, and the training is stopped when the training round number reaches the manually specified round number.

[0089] Step S30, binarizing the trained soft threshold mask, and performing a modular pruning strategy on the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, to obtain a third EfficientNet network model after pruning;

[0090] ​Specifically, the soft threshold mask is binarized according to a threshold value θ set in advance according to pruning requirements after the soft threshold mask training is completed, wherein a value greater than or equal to the threshold value is set to 1, and a value less than the threshold value is set to 0, the inverted residual bottleneck module in the first EfficientNet network is subjected to a modular pruning strategy through the binarized soft threshold mask, and a third EfficientNet network model is obtained after pruning.

[0091] Specifically as Figure 5 shown, a modular pruning workflow diagram provided by the embodiment of the application is provided, and the specific process of the pruning stage is:

[0092] Step S521, obtaining a soft threshold mask;

[0093] Step S522, binarizing the mask;

[0094] Step S523, pruning the compression excitation module;

[0095] Step S524, pruning the MB Conv module.

[0096] Specifically, the trained soft threshold mask is obtained, the soft threshold mask is binarized through a threshold value θ, and the compression excitation module is pruned through the binarized soft threshold mask, and then the MB Conv module is pruned.

[0097] Further, the trained soft threshold mask is binarized, the inverted residual bottleneck module in the first EfficientNet network is subjected to a modular pruning strategy through the binarized soft threshold mask, and a third EfficientNet network model after pruning is obtained, and specifically includes:

[0098] The threshold value is set in advance according to the pruning requirements, the soft threshold mask is binarized through the threshold value, the binarized soft threshold mask is obtained, each inverted residual bottleneck module in the first EfficientNet network is subjected to a modular pruning strategy through the binarized soft threshold mask, the compression excitation attention module, the channel-wise convolution, the point-wise convolution and the dilated convolution in the inverted residual bottleneck module are sequentially pruned, and a third EfficientNet network model after pruning is obtained.

[0099] Specifically, the design of modern neural network architecture is continuously deepening in the direction of modularization, and many channel pruning algorithms or filter pruning algorithms adopt an indirect pruning method to process the network layers before and after the pruning layer, without considering the influence of the pruning layer on the module. EfficientNet is composed of multiple MB Conv modules and other separate convolutional layers and full connection layers, and this pruning method is not suitable for the MB Conv module because the MB Conv module contains a compressed excitation attention module, and the compressed excitation attention mechanism will act on the output of the channel-wise convolution. Pruning the channel-wise convolution cannot guarantee that the pruned channels are the part with lower activation values in the compressed excitation attention module. Pruning the channel-wise convolution and point-wise convolution to indirectly prune the compressed excitation attention module may cause some important channels to be lost, so the present application selects the MB Conv as a whole for pruning, and prunes the MB Conv through the mask trained on the compressed excitation attention module, that is, first, the soft threshold mask of the compressed excitation attention module is converted into a binary mask according to a global threshold, and then the compressed excitation attention module, the channel-wise convolution, the point-wise convolution and the dilated convolution are pruned by the binary mask, wherein the global threshold makes the obtained binary mask be able to act on all layers in the MB Conv.

[0100] According to the pruning requirement, a threshold is set in advance, and the soft threshold mask is binarized by the threshold to obtain a binarized soft threshold mask, denoted as Where i is used to indicate a specific module, and the parameters of the i-th MB Conv module after pruning are which can be expressed as the following formula, and “[]” represents the slicing operation:

[0101]

[0102] Suppose the parameter amount of other separate network layers of the EfficientNet is W other , and the parameter amount of the other separate network layers after pruning is Wotherpruning, then the parameter amount of the EfficientNet with N layers of MB Conv modules before and after pruning can be expressed as:

[0103]

[0104]

[0105] Where W represents the parameter amount before pruning, and W pruning represents the parameter amount after pruning.

[0106] The application prunes the MB Conv module in the EfficientNet through a trainable soft threshold mask, and compared with the pruning algorithm according to importance, the soft threshold mask has the following advantages: the pruning scheme of each layer, that is, the soft threshold mask, is obtained through network training, and then the pruning intensity is controlled through manual setting of a threshold θ, which is easy to adjust parameters and does not need to set a pruning rate for each layer; the importance of neurons is not static, and the importance of neurons depends on input data to a great extent, the filter mask is trained with the network, and the output of the compression excitation attention module is dynamically adjusted, so that the performance after pruning is recovered.

[0107] In step S40, the non-inverted residual bottleneck module in the third EfficientNet network model is pruned through a method based on a batch normalization scale factor and an indirect pruning manner to obtain a pruning model, and the input image is processed through the pruning model.

[0108] In the application, the pruning of the EfficientNet is divided into two parts, that is, the pruning of the MB Conv module and the pruning of the non-MBConv module. The pruning of the MB Conv module is performed by using a soft threshold mask obtained through a compression excitation attention module for pruning, and the pruning of the non-MBConv module is performed by using a method based on a batch normalization scale factor or an indirect pruning manner for pruning. Through the pruning of the non-MBConv module in the EfficientNet network, the pruning of the entire EfficientNet is more complete, and the pruning model EfficientNetSEAP (EfficientNetSEAP represents a network model generated after pruning, and in the following, EfficientNetSEAP represents the same meaning) is more lightweight.

[0109] Further, the non-inverted residual bottleneck module in the third EfficientNet network model is pruned through a method based on a batch normalization scale factor and an indirect pruning manner to obtain a pruning model, and the pruning model is pruned through a method based on a batch normalization scale factor and an indirect pruning manner.

[0110] The main convolutional layer in the third EfficientNet network model is pruned through an indirect pruning manner.

[0111] The feature integration convolutional layer and the fully connected layer in the third EfficientNet network model are pruned through a method based on a batch normalization scale factor.

[0112] Specifically, taking EfficientNet-B0 as an example, there is a backbone convolutional layer (Stem Conv layer) before the first MB Conv module. Because the first MB Conv module does not contain Expand Conv, pruning the MB Conv module will affect the output of the Stem Conv layer. Therefore, when indirectly pruning the output channels of the Stem Conv layer using the binary mask of the first MB Conv module, the Expand Conv in the Stem Conv must also be pruned; otherwise, it will lead to abnormal network structure, i.e., the output channels of the Stem Conv layer correspond one-to-one with the channel-by-channel convolutional output channels of the first MB Conv module. After the last MB Conv module, there is a feature integration convolutional layer (Head Conv layer) and a fully connected layer (FC fully connected layer), and the Head Conv layer and FC layer cannot be indirectly pruned. Figure 6 As shown, the Head Conv and FC layers have a large number of parameters, and neglecting pruning significantly impacts the overall pruning rate of the network. Therefore, a batch normalized scaling factor (BMS) approach is introduced to prune the Head Conv and FC layers. This method applies an LI penalty to the BMS and then prunes based on a threshold. The implementation also uses a binary mask; that is, the mask is binarized based on a pre-set threshold θ according to the pruning requirements after applying the BMS penalty. Values ​​greater than or equal to the threshold are set to 1, and values ​​less than the threshold are set to 0. The corresponding network layer channels are then pruned based on the binary mask. The regularization coefficient for the BMS method is set to 5 times α because the Head Conv is the last convolutional layer, and according to the analysis that deeper networks should have more redundant parts than shallower networks, a stronger penalty is required.

[0113] After pruning the non-MB Conv modules, the number of parameters W in EfficientNet with N layers of MB Conv modules after pruning is as follows: pruning It can be represented as:

[0114]

[0115] Where M head M represents the binary mask of Head Conv. fc The binary mask representing FC. This indicates the binary mask used by StemConv, W stem W head and W fc These represent the parameter quantities of the Stem Conv, Head Conv, and FC layers, respectively. W head [M head ]、Wfc [M fc ] are the parameter quantities of Stem Conv, Head Conv and FC layer respectively after pruning.

[0116] Further, the non-inverted residual bottleneck module in the third EfficientNet network model is pruned by the method based on the batch normalization scale factor and the indirect pruning manner, and after the pruned model is obtained, the method further comprises:

[0117] The loss function is optimized on the target image classification data set, the pruned model is fine-tuned, the fine-tuned pruned model is output when the training round number reaches the manually specified round number, and the fine-tuned pruned model is output.

[0118] Specifically, in the present application, after obtaining the pruned model EfficientNetSEAP, if the user wants to retain the current trained weight and further fine-tune the model, the loss function of the pruned model EfficientNetSEAP is optimized on the target image classification data set using the gradient descent method, the EfficientNetSEAP model is fine-tuned through the optimized loss function, and the fine-tuned pruned model EfficientNetSEAP is output when the training round number of the target image classification data set reaches the manually specified round number.

[0119] In another embodiment of the present application, after pruning the non-inverted residual bottleneck module in the third EfficientNet network model, if the current weight is not retained, the weight of the pruned model is initialized and retrained.

[0120] In addition, the pruning algorithm should avoid pruning shortcut connections, because pruning shortcut connections is not reliable. The present application does not prune shortcut connections by design, because only the internal part of the MB Conv module is pruned, which does not affect the relationship between MB Convs, and shortcut connections are made between MB Conv modules, but this only applies to EfficientNet-B0, which has only one MB Conv module without expansion convolution, and the previous layer of this MB Conv module is Stem Conv, so the pruning method described in the present application does not need to process the shortcut connection of the MB Conv module when applied to EfficientNet-B0. For deeper EfficientNet, they will have multiple MB Conv modules without expansion convolution, which will cause the pruning of the MB Conv module without expansion convolution to indirectly cut the pointwise convolution of the previous MB Conv module; in addition, there will be a situation that the MB Conv module without expansion convolution produces shortcut connection; for the two situations of the MB Conv module without expansion convolution affecting the previous MB Conv module and the MB Conv module without expansion convolution producing shortcut connection, the pruning of the shortcut connection is solved in an indirect pruning manner, guided by the mask of the previous module and ignoring the mask of the current module.

[0121] When the pruned model EfficientNetSEAP obtained by pruning is used to process images, the pruned model removes redundant channels, neuron nodes, network layers, etc. due to pruning, thereby occupying less storage space and computational overhead; and important channels, network layers, etc. in the pruned network model are still retained, so the network performance is not affected, and the pruned model can detect images faster due to the lighter network in the speed of detecting images; in addition, the pruned network has smaller storage space and computational overhead, so it can be easily deployed on mobile edge devices such as mobile phones and single-chip computers.

[0122] The pruning method for pattern recognition based on compressed excitation attention of the neural network can be applied to pruning of an EfficientNet network structure. Specifically, in the application process, a first EfficientNet network is obtained, a trainable soft threshold mask is inserted into each compressed excitation attention module in the first EfficientNet network, a regularization coefficient is set, and a weighted regularization coefficient is calculated according to the parameter quantity of each inverted residual bottleneck module; the soft threshold mask is trained, and a loss function is calculated based on the training result, the regularization coefficient and the weighted regularization coefficient; the soft threshold mask is updated through the loss function, and the training is stopped when the training round number reaches a specified round number; the trained soft threshold mask is binarized, and the inverted residual bottleneck module in the first EfficientNet network is subjected to a modular pruning strategy through the binarized soft threshold mask, to obtain a third EfficientNet network model after pruning; the non-inverted residual bottleneck module in the third EfficientNet network model is pruned through a method based on a batch normalization scale factor and an indirect pruning manner, to obtain a pruning model EfficientNetSEAP, and the pruning model EfficientNetSEAP is used to process an image.

[0123] It can be understood that in the application of the present application, the steps involved in the above application process can be implemented by a server and a terminal. The present application is not limited in the execution subject as long as the actions disclosed in the embodiments of the present application are executed.

[0124] Specifically, the present application includes another embodiment, the present application prunes EfficientNet-B0, and calculates the performance of the pruned network model. In this embodiment, training and testing are performed on the ALASKA2 dataset. In the result comparison of the application, Acc (accuracy), wAUC (area under the weighted ROC curve), Params (parameter quantity) and FLOPs are used as test indicators, wherein the higher the values of Acc and wAUC, the better the model performance, and the lower the values of Params and FLOPs, the lighter the model. In this embodiment, all the models used in the application are implemented using the deep learning framework PyTorch, trained on an NVIDIA Tesla A100 GPU, with a batch size (the number of images input to the neural network each time in the training process) of 32 during training; the random number seed is 2021, and the training is performed for 100 rounds; Adamax is used as the optimizer, and the initial learning rate is 0.001; the optimizer update strategy uses CosineAnnealingLR (cosine annealing), and the minimum learning rate is 0.00001; horizontal and vertical flipping are used as data augmentation; the first 10 rounds of steganographic images and cover images are trained in pairs.

[0125] In this embodiment, EfficientNet-B0 and its pruned version EfficientNet SEAP-B0 are used to train and test on ALASKA2 JUNIWARD QF75 dataset, and different pruning hyperparameters are selected to obtain different pruned networks for comparison. The results are shown in Table 1, where θ represents the pruning threshold below which the mask corresponding to the filter can be pruned; and α represents the l1 regularization coefficient, which affects the sparsity of the mask.

[0126]

[0127]

[0128]

[0129] Table 1: Comparison of pruning performance under different hyperparameter settings

[0130] From Table 1, it can be seen that the pruning algorithm proposed in the present application can effectively reduce the parameter amount of the model while maintaining the accuracy. Among them, the value of θ has little effect on pruning when the value of α is kept the same. The value of α has a greater impact on model compression, and a higher α value can allow the model to retain fewer parameters, but it will also cause the model accuracy to be lost. Considering comprehensively, θ = 0.01 and α = 0.00002 are selected as the hyperparameters of the pruning algorithm, which not only ensures that the model does not lose accuracy, but also greatly compresses the model.

[0131] In this embodiment, EfficientNet-B0 and its pruned version EfficientNet SEAP-B0 are used to train and test on ALASKA2 JUNIWARD QF75 dataset, and the pruning hyperparameters are set to the values selected in the foregoing, i.e. θ = 0.01 and α = 0.00002, and different versions of pruning algorithms are selected for comparison. The results are shown in Table 2, where v1 only sparsifies the compressed excitation attention module for training; v2 adds a modular pruning strategy based on v1; and v3 introduces pruning based on the batch normalization scale factor based on v2.

[0132]

[0133] Table 2: Comparison of different versions of pruning algorithms

[0134] From Table 2, it can be seen that the v2 version has a lower model compression rate than the v1 version, indicating that the modular pruning strategy is more suitable for pruning MB Conv modules than indirect pruning; and the v3 version, which adds pruning of non-MB Conv modules based on the batch normalization scale factor, further improves the model compression rate while not losing accuracy.

[0135] In this embodiment, the comparison results of EfficientNet-B0 and its pruned version EfficientNetSEAP-B0 in the training and testing experiments on the ALASKA2 Kaggle dataset. Among them, uerd, jmipod and juniward belong to different steganography algorithms, and QF represents the quality factor of the image.

[0136]

[0137]

[0138]

[0139] Table 3: Comparison of model performance before and after pruning on the ALASKA2 dataset

[0140] From Table 3, it can be seen that the pruned model EfficientNetSEAP has almost no loss in accuracy after fine-tuning, except that the accuracy of uerd QF90, jmipod QF75 and jmipod QF90 decreases by 0.044, 0.146 and 0.173 respectively; while the accuracy of jmipod QF95, juniward QF75 and juniward QF90 is improved after pruning, because optimizing the soft threshold mask during training may reduce the model performance, and the accuracy after fine-tuning will be improved; for steganography algorithms such as juniward, the compression ratio increases with the increase of the compression factor. In summary, the present application maintains a compression ratio of 90%-95% on EfficientNet-B0, while the accuracy remains the same as before pruning.

[0141] Therefore, from the above, after pruning the EfficientNet network by using the neural network pruning method for pattern recognition based on compressed excitation attention, the network obtained by pruning removes redundant channels, neuron nodes, network layers, etc., thereby occupying less storage space and computational overhead; and important channels, network layers, etc. in the pruned network model are still retained, so it does not affect the network performance, and the pruned model can detect images faster due to the lighter network in detecting images. The reduction of storage space and computational overhead occupied by the pruned network model makes it easy to deploy high-precision network models on mobile edge devices such as mobile phones and single-chip microcomputers.

[0142] It should be noted that the above application scenarios are only shown for the convenience of understanding the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0143] The application summarizes the overall process of the neural network pruning method for pattern recognition based on compression excitation attention, and specifically as shown in the figure, the pruning algorithm workflow provided by the embodiment of the application, and the specific method process is to sequentially obtain an EfficientNet model, insert a soft threshold mask, calculate a weighted regularization coefficient, train the model, binarize the mask for pruning, fine-tune the retrained model, and obtain an EfficientNet SEAP pruning model. Figure 2

[0144] Specifically, the neural network pruning method for pattern recognition based on compression excitation attention of the application prunes the compression excitation attention module in the EfficientNet, combines the output of the compression excitation attention module and the soft threshold mask as the pruning condition, and prunes unimportant filters or channels; a modular pruning strategy is adopted, that is, the pruning condition of the compression excitation attention module is applied to the entire inverted residual bottleneck layer, instead of pruning a layer of convolution and then indirectly pruning the network layer before or after; based on the premise that the redundancy of a deep neural network is greater than that of a shallow neural network, the regularization coefficient is weighted according to the parameter amount of the compression excitation attention module, a stronger constraint is applied to the deep module, and a higher compression rate is obtained; the application dynamically prunes in training, gradually sparsifies the output of the network module, and achieves the purpose of soft pruning, that is, in the training, it is in an unstructured pruning state, and the channels are not really deleted, and after the training is completed, structured pruning is performed according to the threshold set by hand; the application can effectively reduce the video memory overhead and speed up the inference speed while ensuring the accuracy, and reduce the deployment cost; at the same time, it does not need to manually set the pruning rate of each layer, can achieve a high pruning rate, and the accuracy remains almost unchanged.

[0145] In addition, the application combines the advantages of the mask-based and attention-based pruning algorithms, uses a soft threshold mask and an attention mechanism to guide pruning, and fully utilizes the compression excitation attention in the EfficientNet. Both the importance of the channel through attention analysis and the soft pruning in the training process through the mask can be achieved, and no additional attention module needs to be introduced; unlike common mask-based pruning algorithms, the application adjusts the value of the regularization coefficient according to the parameter amount of different modules, more finely controls the mask sparsification, and enables the deep network to have a higher compression rate; most pruning algorithms use indirect pruning to process the relationship between layers, but we use a modular pruning strategy, and for the EfficientNet, the mask of the dimension-increasing convolution of the compression excitation attention module is selected as the pruning scheme of the entire MB Conv module, thereby avoiding most indirect pruning between modules.

[0146] Exemplary apparatus

[0147] As Figure 7 ​As shown in the above method for pruning a neural network for pattern recognition based on compression excitation attention, the embodiment of the present application provides a device for pruning a neural network for pattern recognition based on compression excitation attention, which comprises:

[0148] The pruning pre-module 71 acquires a first EfficientNet network for image processing, inserts a trainable soft threshold mask into each compression excitation attention module in the first EfficientNet network, sets a regularization coefficient, and calculates a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module;

[0149] The training module 72 trains the soft threshold mask, calculates a loss function based on the training result, the regularization coefficient and the weighted regularization coefficient, updates the soft threshold mask through the loss function, and stops training when the number of training rounds reaches a specified number of rounds;

[0150] The inverted residual bottleneck module pruning module 73 binarizes the trained soft threshold mask, prunes the inverted residual bottleneck module in the first EfficientNet network through the binarized soft threshold mask, and obtains a third EfficientNet network model after pruning;

[0151] The non-inverted residual bottleneck module pruning module 74 prunes the non-inverted residual bottleneck module in the third EfficientNet network model through a method based on a batch normalization scale factor and an indirect pruning manner, obtains a pruning model, and controls the input image to be processed through the pruning model.

[0152] Based on the above embodiment, the present application further provides a terminal device. The terminal device comprises a memory, a processor and a method for pruning a neural network for pattern recognition based on compression excitation attention stored on the memory and executable on the processor. When the processor executes the method for pruning a neural network for pattern recognition based on compression excitation attention, the steps of the method for pruning a neural network for pattern recognition based on compression excitation attention are implemented.

[0153] Based on the above embodiment, the present application further provides a computer readable storage medium, characterized in that it stores a method for pruning a neural network for pattern recognition based on compression excitation attention. When the processor executes the method for pruning a neural network for pattern recognition based on compression excitation attention, the steps of the method for pruning a neural network for pattern recognition based on compression excitation attention according to any one of the above are implemented.

[0154] In summary, the present application provides a neural network pruning method, device and equipment for pattern recognition based on compressed excitation attention, and a storage medium. The method comprises the following steps: obtaining a first EfficientNet network for image processing; inserting a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network; setting a regularization coefficient; and calculating a weighted regularization coefficient according to the parameter quantity of each inverted residual bottleneck module. The soft threshold mask is trained, and a loss function is calculated based on the training result, the regularization coefficient and the weighted regularization coefficient. The soft threshold mask is updated through the loss function, and the training is stopped when the number of training rounds reaches a specified number. The trained soft threshold mask is binarized, and the inverted residual bottleneck module in the first EfficientNet network is pruned through a modular pruning strategy based on the binarized soft threshold mask, to obtain a third EfficientNet network model after pruning. The non-inverted residual bottleneck module in the third EfficientNet network model is pruned through a method based on a batch normalization scale factor and an indirect pruning manner, to obtain a pruning model EfficientNetSEAP. The image is processed through the pruning model EfficientNetSEAP. In the field of computer vision, for EfficientNet, the present application can combine the advantages of the mask-based and attention-based pruning algorithms, use the soft threshold mask and the attention mechanism to guide pruning, fully utilize the compressed excitation attention in the EfficientNet, enable the present application to analyze the channel importance through attention, and enable the present application to perform soft pruning after training through the mask, without the need to introduce an additional attention module. At the same time, according to the parameter quantity of different modules, the value of the regularization coefficient is adjusted, the mask sparsification is controlled in a more fine-grained manner, a modular pruning strategy is adopted, indirect pruning between most modules is avoided, and the present application can conveniently and quickly obtain an EfficientNet network model with a higher compression ratio, so that the EfficientNetSEAP after pruning can achieve the effect of smaller calculation overhead and less memory occupation when processing images.

[0155] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0156] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RD RAM), etc.

[0157] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes are within the scope of the appended claims of the present application.

[0158] Specifically, replacing the mask with other indicators is within the scope of the appended claims of the present application; the present application uses attention combined with mask pruning, which can replace the mask with other importance measurement indicators, such as absolute value, and can directly prune the model without training; or remove the mask, collect high-order statistical information of filter feature maps on the training set for pruning. Adjusting the position of the mask is within the scope of the appended claims of the present application, the present application sets the mask at the dimension-up convolution, which is convenient for using attention information, and similarly the mask can be placed after point-by-point convolution, and modular pruning strategy can also be used. Using other pruning algorithms to process non-MB Conv parts is within the scope of the appended claims of the present application, and the processing of non-MB Conv parts can be replaced by importance-based pruning algorithm, feature map-based pruning algorithm and reinforcement learning-based pruning algorithm.

[0159] These modifications and variations are within the scope of the appended claims, and the application is not limited to the examples described above, but can be modified or adapted in various ways by those skilled in the art, all such modifications and variations being within the scope of the appended claims.

Claims

1. A neural network pruning method based on compressed excitation attention for pattern recognition, characterized in that, The method includes: Obtain a first EfficientNet network for image processing, insert a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network, set regularization coefficients, and calculate weighted regularization coefficients based on the number of parameters of each inverse residual bottleneck module. The soft threshold mask is trained, and a loss function is calculated based on the training results, regularization coefficients, and weighted regularization coefficients. The soft threshold mask is updated using the loss function, and training stops when the number of training rounds reaches a specified number of rounds. The trained soft threshold mask is binarized, and the inverse residual bottleneck module in the first EfficientNet network is pruned using a modular pruning strategy through the binarized soft threshold mask to obtain the pruned third EfficientNet network model. The non-reverse residual bottleneck module in the third EfficientNet network model is pruned using a batch normalization scaling factor method and indirect pruning to obtain a pruning model. Indirect pruning is achieved by adjusting the parameters of adjacent layers in a corresponding manner when directly removing or pruning the parameters of a certain network layer during the structured pruning process, thereby indirectly achieving the pruning effect on the adjacent layers. The control processes the input image through the pruning model.

2. The neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, When acquiring the first EfficientNet network for image processing, the weights of a second EfficientNet model with the same structure as the first EfficientNet network, trained on the large-scale image classification dataset ImageNet, are acquired, and the first EfficientNet network loads these weights as its initial weights.

3. The neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, The step of inserting a trainable soft threshold mask into each compressed stimulated attention module in the first EfficientNet network, setting regularization coefficients, and calculating weighted regularization coefficients based on the number of parameters in each inverse residual bottleneck module specifically includes: The inserted trainable soft threshold mask is initialized as an all-1 vector; The regularization coefficient is a preset value; The weighted regularization coefficients are calculated using the parameter count of the inverse residual bottleneck module in the first EfficientNet network; The regularization coefficients and the weighted regularization coefficients do not change during the soft threshold mask training process.

4. The neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, The process of training the soft threshold mask, calculating a loss function based on the training results, regularization coefficients, and weighted regularization coefficients, updating the soft threshold mask using the loss function, and stopping training when the training epochs reach a specified number of epochs specifically includes: The first EfficientNet network with a trainable soft threshold mask inserted is trained on the target image classification dataset. The loss function is calculated based on the training results, regularization coefficients and weighted regularization coefficients. The soft threshold mask is updated by the loss function and backpropagated. The network is then trained again on the target image classification dataset. Training stops when the specified number of training rounds is reached.

5. A neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, The process of binarizing the trained soft threshold mask and then using the binarized soft threshold mask to perform a modular pruning strategy on the inverse residual bottleneck module in the first EfficientNet network to obtain the pruned third EfficientNet network model specifically includes: A threshold is pre-set according to the pruning requirements. The soft threshold mask is binarized using this threshold to obtain a binarized soft threshold mask. A modular pruning strategy is then applied to each inverted residual bottleneck module in the first EfficientNet network using the binarized soft threshold mask. The compressed excitation attention module, channel-wise convolution, pointwise convolution, and dilated convolution in the inverted residual bottleneck module are pruned sequentially to obtain the pruned third EfficientNet network model.

6. A neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, The non-reverse residual bottleneck module in the third EfficientNet network model is pruned using a batch normalization scaling factor-based method and indirect pruning to obtain a pruned model, specifically including: Indirect pruning is used to prune the backbone convolutional layers in the third EfficientNet network model. The feature integration convolutional layer and fully connected layer in the third EfficientNet network model are pruned using a batch normalization scaling factor-based method.

7. A neural network pruning method for pattern recognition based on compressed excitation attention according to claim 1, characterized in that, The process of pruning the non-reverse residual bottleneck module in the third EfficientNet network model using a batch normalization scaling factor method and indirect pruning, resulting in a pruned model, further includes: Optimize the loss function on the target image classification dataset, fine-tune the pruning model, and stop fine-tuning when the number of training rounds reaches the manually specified number of rounds, and output the fine-tuned pruning model.

8. A neural network pruning device for pattern recognition based on compressed excitation attention, wherein the neural network pruning device for pattern recognition based on compressed excitation attention is applied to the neural network pruning method for pattern recognition based on compressed excitation attention according to any one of claims 1-7, characterized in that, The device includes: The pre-pruning module obtains a first EfficientNet network for image processing, inserts a trainable soft threshold mask into each compressed excitation attention module in the first EfficientNet network, sets regularization coefficients, and calculates weighted regularization coefficients based on the number of parameters of each inverse residual bottleneck module. The training module trains the soft threshold mask and calculates a loss function based on the training results, regularization coefficients, and weighted regularization coefficients. The soft threshold mask is updated using the loss function, and training stops when the number of training rounds reaches a specified number of rounds. For the inverse residual bottleneck module pruning module, the trained soft threshold mask is binarized, and the inverse residual bottleneck module in the first EfficientNet network is pruned through the binarized soft threshold mask to obtain the pruned third EfficientNet network model. The non-reverse residual bottleneck module pruning module in the third EfficientNet network model is pruned using a batch normalization scaling factor-based method and indirect pruning to obtain a pruning model, which controls the processing of the input image through the pruning model.

9. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a neural network pruning method for pattern recognition based on compressed excitation attention, stored in the memory and executable on the processor. When the processor executes the neural network pruning method for pattern recognition based on compressed excitation attention, it implements the steps of the neural network pruning method for pattern recognition based on compressed excitation attention as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a neural network pruning method for pattern recognition based on compressed excitation attention. When the neural network pruning method for pattern recognition based on compressed excitation attention is executed by the processor, it implements the steps of the neural network pruning method for pattern recognition based on compressed excitation attention as described in any one of claims 1-7.

Images