An image classification model generation method, device, equipment and storage medium
By acquiring and deleting the filters corresponding to the inactive mask matrix during the training process of the image classification model, the problem of cumbersome pruning methods in the existing technology is solved, and efficient pruning and training are achieved under the limited resources of terminal devices.
Patent Information
- Application Number
- CN202310825795.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-06
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-07-06
AI Technical Summary
Existing pruning methods are performed after the neural network model has been trained, which is cumbersome and time-consuming, making it difficult to efficiently prune neural network models under the limited resources of terminal devices.
During the training of an image classification model, the inactivation mask matrix of the feature extraction layer is obtained and the corresponding filter is deleted, which reduces the training complexity and improves efficiency, avoiding subsequent fine-tuning.
By directly pruning the filters in the feature extraction layer during training, the model training complexity is reduced and the model training efficiency is improved, thus enhancing the pruning efficiency.
Smart Images
Figure CN116778254B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to an image classification model generation method, apparatus, device and storage medium. Background Technology
[0002] With the rapid development of computer technology, neural network models have been widely used in image processing, speech processing, and natural language processing. Generally speaking, the better the performance of a neural network model, the larger its number of parameters and computational cost. Taking ResNet-50 in the field of image processing as an example, when a 224*224 RGB image is input into ResNet-50, it requires 25.6MB of parameters and 4.1B of floating-point computation. Therefore, terminal devices need to have substantial storage and computing resources to support a trained ResNet-50.
[0003] To enable terminal devices to support any neural network model even with limited storage and computing resources, pruning methods are often used to prune the neural network model. However, the commonly used pruning method involves pruning the trained neural network model after training is complete, followed by fine-tuning the pruned model. This pruning method is cumbersome and time-consuming. Summary of the Invention
[0004] This application provides an image classification model generation method, apparatus, device, and storage medium to reduce pruning complexity and improve pruning efficiency.
[0005] On one hand, embodiments of this application provide an image classification model generation method, the method comprising:
[0006] An image classification model is trained using images to be classified with preset labels. The image classification model includes multiple feature extraction layers, each of which includes multiple filters and corresponding multiple mask matrices. The size of each filter and the corresponding mask matrix is the same. The multiple filters are used to extract different features from the image to be classified, and each mask matrix is used to adjust the features extracted by the corresponding filter. The parameters in the multiple filters and the parameters in the multiple mask matrices are adjusted during training.
[0007] After at least one training iteration, for any feature extraction layer, the inactive mask matrix corresponding to the feature extraction layer is obtained from multiple mask matrices of the feature extraction layer. The inactive mask matrix is used to indicate the mask matrices in the feature extraction layer that do not participate in subsequent training.
[0008] After training, for any feature extraction layer, based on the inactivation mask matrix corresponding to the feature extraction layer, the filter corresponding to the inactivation mask matrix is removed from the feature extraction layer, thereby obtaining the trained image classification model.
[0009] Optionally, the image classification model further includes a classification layer and a loss function layer;
[0010] The step of training the image classification model using images to be classified with preset labels includes:
[0011] In any of the at least one training session, the following steps are performed:
[0012] Based on the filters and mask matrices of the multiple feature extraction layers, feature extraction is performed on the image to be classified to obtain the first feature;
[0013] The first feature is input into the classification layer to obtain the first predicted category;
[0014] The first predicted category and the preset label are input into the loss function layer to obtain the first loss function value; based on the first loss function value, the parameters in multiple filters of each feature extraction layer are updated;
[0015] For any feature extraction layer, a second loss function value corresponding to the feature extraction layer is determined based on multiple mask matrices of the feature extraction layer; and the parameters in the multiple mask matrices of the feature extraction layer are updated based on the first loss function value and the second loss function value.
[0016] Optionally, the step of extracting features from the image to be classified based on the filter and mask matrix of the multiple feature extraction layers to obtain the first feature includes:
[0017] For the j-th feature extraction layer, the following steps are performed: Based on the filter and mask matrix of the j-th feature extraction layer, determine the sparse filter of the j-th feature extraction layer; use the sparse filter of the j-th feature extraction layer to extract features from the feature vector corresponding to the (j-1)-th feature extraction layer to obtain the feature vector corresponding to the j-th feature extraction layer; where j>=1;
[0018] The feature vector corresponding to the last feature extraction layer is used as the first feature.
[0019] Optionally, determining the second loss function value corresponding to the feature extraction layer based on multiple mask matrices of the feature extraction layer includes:
[0020] The second norm is used to process multiple mask matrices of the feature extraction layer to obtain the first mask array of the feature extraction layer;
[0021] The first mask array of the feature extraction layer is processed using the first norm to obtain the second loss function value of the feature extraction layer.
[0022] Optionally, the feature extraction layer includes M*N mask matrices; the M*N mask matrices form a first matrix;
[0023] The step of obtaining the inactivation mask matrix corresponding to the feature extraction layer from multiple mask matrices of the feature extraction layer includes:
[0024] For the i-th training iteration in the at least one training iteration, determine the pruning amount L corresponding to the feature extraction layer in the i-th training iteration; where i>=1, 1<=L<=N;
[0025] From the first matrix, obtain the inactivation mask matrix corresponding to the feature extraction layer in the i-th training, the inactivation mask matrix including M*L mask matrices.
[0026] Optionally, it also includes:
[0027] In the case where the at least one training session includes the i-th training session and the (i+1)-th training session, the pruning amount of the feature extraction layer corresponding to the (i+1)-th training session is greater than or equal to the pruning amount corresponding to the i-th training session.
[0028] Optionally, determining the pruning amount L corresponding to the feature extraction layer in the i-th training iteration includes:
[0029] Determine the pruning rate of the feature extraction layer corresponding to the i-th training iteration;
[0030] Based on the pruning rate of the feature extraction layer in the i-th training iteration and the initial number of filters in the feature extraction layer, the pruning amount L of the feature extraction layer in the i-th training iteration is determined.
[0031] Optionally, determining the pruning rate of the feature extraction layer corresponding to the i-th training iteration includes:
[0032] Based on the preset initial pruning rate, the preset target pruning rate, the preset initial number of cycles, and the number of cycles corresponding to the i-th training, the pruning adjustment amount of the feature extraction layer corresponding to the i-th training is determined;
[0033] Based on the preset target pruning rate and the pruning adjustment amount, the pruning rate corresponding to the feature extraction layer in the i-th training is determined.
[0034] Optionally, obtaining the inactivation mask matrix corresponding to the feature extraction layer in the i-th training iteration from the first matrix includes:
[0035] Calculate the mask matrix for each column of the first matrix to obtain N mask sums;
[0036] Based on the first preset rule, L mask sums are determined from the N mask sums, and based on the first matrix, the first column index values corresponding to the L mask sums are determined respectively;
[0037] Use the M mask matrices corresponding to each first column index value in the first matrix as the deactivation mask matrix.
[0038] Optionally, the feature extraction layer further includes M*N filters, which form a second matrix;
[0039] The step of deleting the filter corresponding to the inactivation mask matrix from any feature extraction layer based on the inactivation mask matrix corresponding to the feature extraction layer includes:
[0040] For the j-th feature extraction layer, perform the following steps:
[0041] Based on the inactivation mask matrix of the j-th feature extraction layer, at least one second column index value is determined, and the M filters corresponding to each second column index value in the second matrix are deleted.
[0042] Based on the inactivation mask matrix corresponding to the (j-1)th feature extraction layer, determine at least one first row index value, and delete the N filters corresponding to each first row index value in the second matrix.
[0043] On one hand, embodiments of this application provide an image classification model generation apparatus, the apparatus comprising:
[0044] The training module is used to train an image classification model using images to be classified with preset labels. The image classification model includes multiple feature extraction layers, each of which includes multiple filters and corresponding multiple mask matrices. The size of each filter and the corresponding mask matrix is the same. The multiple filters are used to extract different features from the images to be classified, and each mask matrix is used to adjust the features extracted by the corresponding filter. The parameters in the multiple filters and the parameters in the multiple mask matrices are adjusted during the training process.
[0045] The acquisition module is used to, after at least one training session, for any feature extraction layer, acquire the inactive mask matrix corresponding to the feature extraction layer from multiple mask matrices of the feature extraction layer, wherein the inactive mask matrix is used to indicate the mask matrices in the feature extraction layer that do not participate in subsequent training.
[0046] The deletion module is used to delete the filter corresponding to the inactivation mask matrix from any feature extraction layer after training, based on the inactivation mask matrix corresponding to the feature extraction layer, thereby obtaining the trained image classification model.
[0047] Optionally, the image classification model further includes a classification layer and a loss function layer;
[0048] The training module is specifically used for:
[0049] In any of the at least one training session, the following steps are performed:
[0050] Based on the filters and mask matrices of the multiple feature extraction layers, feature extraction is performed on the image to be classified to obtain the first feature;
[0051] The first feature is input into the classification layer to obtain the first predicted category;
[0052] The first predicted category and the preset label are input into the loss function layer to obtain the first loss function value; based on the first loss function value, the parameters in multiple filters of each feature extraction layer are updated;
[0053] For any feature extraction layer, a second loss function value corresponding to the feature extraction layer is determined based on multiple mask matrices of the feature extraction layer; and the parameters in the multiple mask matrices of the feature extraction layer are updated based on the first loss function value and the second loss function value.
[0054] Optionally, the training module is specifically used for:
[0055] For the j-th feature extraction layer, the following steps are performed: Based on the filter and mask matrix of the j-th feature extraction layer, determine the sparse filter of the j-th feature extraction layer; use the sparse filter of the j-th feature extraction layer to extract features from the feature vector corresponding to the (j-1)-th feature extraction layer to obtain the feature vector corresponding to the j-th feature extraction layer; where j>=1;
[0056] The feature vector corresponding to the last feature extraction layer is used as the first feature.
[0057] Optionally, the training module is specifically used for:
[0058] The second norm is used to process multiple mask matrices of the feature extraction layer to obtain the first mask array of the feature extraction layer;
[0059] The first mask array of the feature extraction layer is processed using the first norm to obtain the second loss function value of the feature extraction layer.
[0060] Optionally, the feature extraction layer includes M*N mask matrices; the M*N mask matrices form a first matrix;
[0061] The acquisition module is specifically used for:
[0062] For the i-th training iteration in the at least one training iteration, determine the pruning amount L corresponding to the feature extraction layer in the i-th training iteration; where i>=1, 1<=L<=N;
[0063] From the first matrix, obtain the inactivation mask matrix corresponding to the feature extraction layer in the i-th training, the inactivation mask matrix including M*L mask matrices.
[0064] Optionally, the acquisition module is further configured to:
[0065] In the case where the at least one training session includes the i-th training session and the (i+1)-th training session, the pruning amount of the feature extraction layer corresponding to the (i+1)-th training session is greater than or equal to the pruning amount corresponding to the i-th training session.
[0066] Optionally, the acquisition module is specifically used for:
[0067] Determine the pruning rate of the feature extraction layer corresponding to the i-th training iteration;
[0068] Based on the pruning rate of the feature extraction layer in the i-th training iteration and the initial number of filters in the feature extraction layer, the pruning amount L of the feature extraction layer in the i-th training iteration is determined.
[0069] Optionally, the acquisition module is specifically used for:
[0070] Based on the preset initial pruning rate, the preset target pruning rate, the preset initial number of cycles, and the number of cycles corresponding to the i-th training, the pruning adjustment amount of the feature extraction layer corresponding to the i-th training is determined;
[0071] Based on the preset target pruning rate and the pruning adjustment amount, the pruning rate corresponding to the feature extraction layer in the i-th training is determined.
[0072] Optionally, the acquisition module is specifically used for:
[0073] Calculate the mask matrix for each column of the first matrix to obtain N mask sums;
[0074] Based on the first preset rule, L mask sums are determined from the N mask sums, and based on the first matrix, the first column index values corresponding to the L mask sums are determined respectively;
[0075] Use the M mask matrices corresponding to each first column index value in the first matrix as the deactivation mask matrix.
[0076] Optionally, the feature extraction layer further includes M*N filters, which form a second matrix;
[0077] The deletion module is specifically used for:
[0078] For the j-th feature extraction layer, perform the following steps:
[0079] Based on the inactivation mask matrix of the j-th feature extraction layer, at least one second column index value is determined, and the M filters corresponding to each second column index value in the second matrix are deleted.
[0080] Based on the inactivation mask matrix corresponding to the (j-1)th feature extraction layer, determine at least one first row index value, and delete the N filters corresponding to each first row index value in the second matrix.
[0081] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described image classification model generation method.
[0082] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the image classification model generation method described above.
[0083] In this embodiment, the image classification model is first trained using images to be classified with preset labels. After at least one training iteration, for any feature extraction layer, the corresponding inactive mask matrix is obtained from multiple mask matrices of the feature extraction layer. The inactive mask matrix indicates which mask matrices in the feature extraction layer will not participate in subsequent training, reducing the complexity and improving the efficiency of model training. After training, for any feature extraction layer, based on the corresponding inactive mask matrix, the filter corresponding to the inactive mask matrix is removed from the feature extraction layer, thus obtaining the trained image classification model without further fine-tuning. This further improves model training efficiency and pruning efficiency. Attached Figure Description
[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0085] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;
[0086] Figure 2 A flowchart illustrating an image classification model generation method provided in this application embodiment;
[0087] Figure 3 This is a schematic diagram of the structure of an image classification model provided in an embodiment of this application;
[0088] Figure 4 This is a schematic diagram of the structure of a filter for a feature extraction layer provided in an embodiment of this application;
[0089] Figure 5 A schematic diagram illustrating the training process of an image classification model provided in an embodiment of this application;
[0090] Figure 6 A schematic diagram illustrating a process for obtaining the inactivation mask matrix of a feature extraction layer, provided in an embodiment of this application;
[0091] Figure 7 A schematic diagram of a process for determining the pruning amount L provided in an embodiment of this application;
[0092] Figure 8 A flowchart illustrating the process of determining a deactivation mask matrix is provided in an embodiment of this application.
[0093] Figure 9 This is a schematic diagram of the structure of a first matrix provided in an embodiment of this application;
[0094] Figure 10 This is a schematic diagram of the structure of a deactivation mask matrix provided in an embodiment of this application;
[0095] Figure 11 This is a schematic diagram of the structure of a second matrix provided in an embodiment of this application;
[0096] Figure 12 This is a schematic diagram of the structure of a second matrix provided in an embodiment of this application;
[0097] Figure 13 This is a schematic diagram of the structure of a second matrix provided in an embodiment of this application;
[0098] Figure 14This is a schematic diagram of the structure of an image classification model generation device provided in an embodiment of this application;
[0099] Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0100] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0101] refer to Figure 1 This is a system architecture diagram of an image classification model generation system applicable to the embodiments of this application. The system architecture diagram of the image classification model generation system includes at least a terminal device 101 and an image classification model generation system 102.
[0102] The terminal device 101 is equipped with a target application for generating an image classification model. This application can be a pre-installed client, a web application, or a mini-program embedded in other applications. The terminal device 101 can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these.
[0103] The image classification model generation system 102 serves as the backend server for the target application, providing services to the target application. The image classification model generation system 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0104] The terminal device 101 and the image classification model generation system 102 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.
[0105] Terminal device 101 responds to the user's operation to generate an image classification model by sending an instruction to generate an image classification model to image classification model generation system 102. Image classification model generation system 102 trains the image classification model using images to be classified with preset labels. The image classification model includes multiple feature extraction layers, each including multiple filters and corresponding mask matrices. Each filter and its corresponding mask matrix are of the same size. Multiple filters are used to extract different features from the image to be classified, and each mask matrix is used to adjust the features extracted by the corresponding filter. The parameters in the multiple filters and the multiple mask matrices are adjusted during training. After at least one training iteration, for any feature extraction layer, a deactivation mask matrix corresponding to the feature extraction layer is obtained from the multiple mask matrices of the feature extraction layer. The deactivation mask matrix indicates the mask matrices in the feature extraction layer that do not participate in subsequent training. After training, for any feature extraction layer, based on the deactivation mask matrix corresponding to the feature extraction layer, the filter corresponding to the deactivation mask matrix is deleted from the feature extraction layer, thereby obtaining the trained image classification model.
[0106] based on Figure 1 The system architecture diagram described above illustrates the flow of an image classification model generation method provided in this application embodiment, as follows: Figure 2 As shown, the process of this method is as follows: Figure 1 The image classification model generation system 102 shown executes the following steps:
[0107] Step S201: Train the image classification model using images to be classified with preset labels.
[0108] In this embodiment, the image classification model includes multiple feature extraction layers, classification layers, and loss function layers. The structure of the image classification model is as follows: Figure 3 As shown.
[0109] Each feature extraction layer includes multiple filters and corresponding mask matrices. These filters can also be called convolution kernels. The number of filters in each feature extraction layer can be the same or different. The size of the filters in each feature extraction layer can also be the same or different.
[0110] For any feature extraction layer, each filter has the same size, and each mask matrix also has the same size. Each filter and its corresponding mask matrix are of the same size.
[0111] For example, for any feature extraction layer, which includes 2*3 filters, each filter being 3*3 in size, the 2*3 filters of this feature extraction layer are as follows: Figure 4As shown. This feature extraction layer also includes 2*3 mask matrices, each of which is 3*3 in size.
[0112] The filter can be initialized with preset values or randomly, such as using decimals between 0 and 1. The mask matrix can also be initialized with preset values, such as using the preset value 1 to randomly initialize the mask matrix. The parameters in the filter and the mask matrix are adjusted during training.
[0113] In the embodiments of this application, multiple filters included in any feature extraction layer are used to extract different features from the image to be classified, and each mask matrix is used to adjust the features extracted by the corresponding filter.
[0114] Optionally, the image classification model is trained using images to be classified with preset labels. In any of the at least one training iteration, the following can be performed: Figure 5 The following steps are shown:
[0115] Step S501: Based on the filters and mask matrix of multiple feature extraction layers, feature extraction is performed on the image to be classified to obtain the first feature.
[0116] In this embodiment of the application, the following steps can be performed for the j-th feature extraction layer:
[0117] First, based on the filter and mask matrix of the j-th feature extraction layer, determine the sparse filter of the j-th feature extraction layer. Then, use the sparse filter of the j-th feature extraction layer to extract features from the feature vector corresponding to the (j-1)-th feature extraction layer, obtaining the feature vector corresponding to the j-th feature extraction layer; where j>=1. Use the feature vector corresponding to the last feature extraction layer as the first feature.
[0118] Specifically, multiplying the corresponding positions of the filter and mask matrix of the j-th feature extraction layer yields the sparse filter of the j-th feature extraction layer. Then, convolving the sparse filter of the j-th feature extraction layer with the feature vector corresponding to the (j-1)-th feature extraction layer yields the feature vector corresponding to the j-th feature extraction layer.
[0119] For example, the j-th feature extraction layer includes 2*3 filters, each of size 3*3, and also includes 2*3 mask matrices, each of size 3*3. Taking one of the filters in the j-th feature extraction layer as an example, the values of this filter are shown in Table 1.
[0120] Table 1.
[0121] 0.2 0.3 0.5 0.4 0.1 0.5 0.6 0.7 0.9
[0122] The mask matrix corresponding to this filter is shown in Table 2.
[0123] Table 2.
[0124] 1 0 1 1 0 1 1 1 1
[0125] The coefficient filters obtained by multiplying the above filters and mask matrices at corresponding positions are shown in Table 3.
[0126] Table 3.
[0127]
[0128]
[0129] The other five filters in the j-th feature extraction layer and their corresponding mask matrices are obtained using a method similar to that described above, and will not be repeated here. Therefore, the j-th feature extraction layer comprises a total of 6 sparse filters.
[0130] Step S502: Input the first feature into the classification layer to obtain the first predicted category.
[0131] Step S503: Input the first predicted category and the preset label into the loss function layer to obtain the first loss function value.
[0132] In the embodiments of this application, the loss function corresponding to the loss function layer can be any of the following: Mean Squared Error Loss Function, Cross-entropy Loss Function, etc.
[0133] Step S504: Based on the first loss function value, update the parameters in multiple filters of each feature extraction layer.
[0134] When the loss function corresponding to the loss function layer is different, the first loss function value obtained is different, and the update method of the parameters in the multiple filters of each feature extraction layer is also different, which will not be elaborated here.
[0135] Step S505: For any feature extraction layer, determine the second loss function value corresponding to the feature extraction layer based on the multiple mask matrices of the feature extraction layer; and update the parameters in the multiple mask matrices of the feature extraction layer based on the first loss function value and the second loss function value.
[0136] Specifically, the value of the second loss function corresponding to this feature extraction layer can be determined in the following way:
[0137] First, the second norm is used to process multiple mask matrices of the feature extraction layer to obtain the first mask array of the feature extraction layer. Then, the first norm is used to process the first mask array of the feature extraction layer to obtain the second loss function value of the feature extraction layer.
[0138] The value of the second loss function can be determined by the following formula:
[0139]
[0140] Where A represents multiple mask matrices of the feature extraction layer, and ||||2 represents the second norm. This represents the first norm.
[0141] Specifically, a first increment is first determined based on the first loss function value, and a second increment is determined based on the second loss function value. The parameters in the multiple mask matrices of the feature extraction layer are updated using the first increment, and then the parameters in the multiple mask matrices of the feature extraction layer are updated using the second increment.
[0142] Step S202: After at least one training session, for any feature extraction layer, obtain the inactivation mask matrix corresponding to the feature extraction layer from multiple mask matrices of the feature extraction layer.
[0143] Specifically, after each training session, for any given feature extraction layer, the inactivation mask matrix corresponding to the feature extraction layer can be obtained from the multiple mask matrices of the feature extraction layer.
[0144] Alternatively, after any training iteration, for any feature extraction layer, the inactivation mask matrix corresponding to the feature extraction layer can be obtained from the multiple mask matrices of the feature extraction layer.
[0145] In this embodiment, the inactivation mask matrix is used to indicate the mask matrix in the feature extraction layer that does not participate in subsequent training.
[0146] Step S203: After training, for any feature extraction layer, based on the inactivation mask matrix corresponding to the feature extraction layer, delete the filter corresponding to the inactivation mask matrix from the feature extraction layer, thereby obtaining the trained image classification model.
[0147] In this embodiment, the image classification model is first trained using images to be classified with preset labels. After at least one training iteration, for any feature extraction layer, the corresponding inactive mask matrix is obtained from multiple mask matrices of the feature extraction layer. The inactive mask matrix indicates which mask matrices in the feature extraction layer will not participate in subsequent training, reducing the complexity and improving the efficiency of model training. After training, for any feature extraction layer, based on the corresponding inactive mask matrix, the filter corresponding to the inactive mask matrix is removed from the feature extraction layer, thus obtaining the trained image classification model without further fine-tuning. This further improves model training efficiency and pruning efficiency.
[0148] Optionally, for any feature extraction layer, the feature extraction layer includes M*N mask matrices and M*N filters, wherein the M*N mask matrices form a first matrix and the M*N filters form a second matrix. The mask matrix is a two-dimensional matrix and the filter is also a two-dimensional matrix, and M and N are both integers greater than 1.
[0149] Optionally, in step S202 above, obtaining the inactivation mask matrix corresponding to the feature extraction layer from the multiple mask matrices of the feature extraction layer may include, for example: Figure 6 The following steps are shown:
[0150] Step S601: For the i-th training in at least one training session, determine the pruning amount L corresponding to the feature extraction layer in the i-th training session; where i>=1, 1<=L<=N.
[0151] It should be understood that, in the case that at least one training session includes the i-th training session and the (i+1)-th training session, the pruning amount of the feature extraction layer in the (i+1)-th training session is greater than or equal to the pruning amount in the i-th training session.
[0152] The pruning amount L for different feature extraction layers in the i-th training iteration can be the same or different.
[0153] Specifically, determining the pruning amount L corresponding to the feature extraction layer in the i-th training iteration can include, for example: Figure 7 The following steps are shown:
[0154] Step S701: Determine the pruning rate corresponding to the feature extraction layer in the i-th training iteration.
[0155] Specifically, firstly, based on the preset initial pruning rate, preset target pruning rate, preset initial number of epochs, and the number of epochs corresponding to the i-th training iteration, the pruning adjustment amount corresponding to the feature extraction layer in the i-th training iteration is determined. Then, based on the preset target pruning rate and the pruning adjustment amount, the pruning rate corresponding to the feature extraction layer in the i-th training iteration is determined.
[0156] The pruning rate of the feature extraction layer during the i-th training iteration can be expressed by formula (2):
[0157] p i =p tar +(p0-p tar )*(1-(e t -e0)) (2)
[0158] Where, p i p represents the pruning rate of the feature extraction layer during the i-th training iteration. tar p0 represents the preset target pruning rate of the feature extraction layer, and e represents the preset initial pruning rate of the feature extraction layer. t e represents the number of epochs corresponding to the i-th training iteration of the feature extraction layer. t This indicates the preset initial number of cycles for the feature extraction layer.
[0159] To facilitate adjustment of the pruning rate for different training iterations, the pruning rate of the feature extraction layer for the i-th training iteration can be expressed by formula (3):
[0160]
[0161] Where, p i p represents the pruning rate of the feature extraction layer during the i-th training iteration. tar p0 represents the preset target pruning rate of the feature extraction layer, and e represents the preset initial pruning rate of the feature extraction layer. t e0 represents the number of epochs corresponding to the i-th training of the feature extraction layer, e0 represents the preset initial number of epochs of the feature extraction layer, Δm represents the sparse optimization frequency of the feature extraction layer, Δm is a decimal between 0 and 1, and n is the pruning decay rate of the feature extraction layer, n is any positive integer.
[0162] It should be understood that, in order to improve the robustness and accuracy of image classification models, different preset target pruning rates p can be set for different feature extraction layers. tar The preset initial pruning rate p0, preset initial number of cycles e0, sparse optimization frequency Δm, and pruning decay rate n are defined. To reduce the complexity of the image classification model generation method, the same preset target pruning rate p can be set for different feature extraction layers. tar The preset initial pruning rate is p0, the preset initial number of cycles is e0, the sparse optimization frequency is Δm, and the pruning decay rate is n.
[0163] Step S702: Based on the pruning rate of the feature extraction layer in the i-th training and the initial number of filters in the feature extraction layer, determine the pruning amount L of the feature extraction layer in the i-th training.
[0164] In this embodiment of the application, the initial number of filters in the feature extraction layer is M*N, and the M*N filters form a second matrix with N columns.
[0165] In a first possible implementation, the pruning rate p corresponding to the feature extraction layer in the i-th training iteration is calculated. i The product L with the number of columns N of the second matrix of the feature extraction layer can1 and the product L can1 L represents the pruning amount corresponding to the i-th training iteration of the feature extraction layer.
[0166] The pruning amount L corresponding to the feature extraction layer in the i-th training can be expressed by formula (4):
[0167] L = p i *N (4)
[0168] Where L represents the pruning amount of the feature extraction layer in the i-th training iteration, and p i This represents the pruning rate of the feature extraction layer during the i-th training iteration, and N represents the number of columns in the second matrix of the feature extraction layer.
[0169] A second possible implementation involves obtaining a preset pruning threshold L for the feature extraction layer. 剪枝阈值 From the preset pruning threshold L 剪枝阈值 And the pruning rate p corresponding to the feature extraction layer in the i-th training iteration. i The product L with the number of columns N of the second matrix of the feature extraction layer can1 In the process, the smaller value is selected as the pruning amount L corresponding to the i-th training iteration of the feature extraction layer.
[0170] The pruning amount L corresponding to the feature extraction layer in the i-th training iteration can be expressed by formula (5):
[0171] L = min (p i *N,L 剪枝阈值 (5)
[0172] Where L represents the pruning amount of the feature extraction layer in the i-th training iteration, and p i L represents the pruning rate of the feature extraction layer during the i-th training iteration, N represents the number of columns in the second matrix of the feature extraction layer, and L represents the pruning rate of the second matrix of the feature extraction layer. 剪枝阈值 This represents the preset pruning threshold for the feature extraction layer.
[0173] Step S602: Obtain the inactivation mask matrix corresponding to the feature extraction layer in the i-th training from the first matrix, wherein the inactivation mask matrix includes M*L mask matrices.
[0174] In the embodiments of this application, the inactivation mask matrix corresponding to each training of the feature extraction layer can be determined. The inactivation mask matrix determined by the feature extraction layer in the i-th training is not the same as or partially the same as the inactivation mask matrix determined by the feature extraction layer in the (i+1)-th training.
[0175] The inactivation mask matrix corresponding to the feature extraction layer mentioned in step S202 above includes the inactivation mask matrix determined in each training session.
[0176] Specifically, the inactivation mask matrix corresponding to the feature extraction layer in the i-th training iteration is obtained from the first matrix, which may include, for example: Figure 8 The following steps are shown:
[0177] Step S801: Calculate the mask matrix for each column of the first matrix to obtain N mask sums.
[0178] In one possible implementation, the value corresponding to each mask matrix can be determined first, and then the values corresponding to each column of the mask matrix in the first matrix can be added together to obtain the mask sum corresponding to each column of the mask matrix.
[0179] The value corresponding to any mask matrix can be determined through any of the following possible implementation methods:
[0180] In one possible implementation, a value is randomly selected from the mask matrix and used as the value corresponding to the mask matrix.
[0181] A second possible implementation involves taking the average of the values in the mask matrix and using the resulting average as the value corresponding to the mask matrix.
[0182] A third possible implementation involves summing the values in the mask matrix and using the sum as the value corresponding to the mask matrix.
[0183] Step S802: Based on the first preset rule, determine L mask sums from N mask sums, and based on the first matrix, determine the first column index value corresponding to each of the L mask sums.
[0184] Alternatively, the L mask sums with the smallest mask sums can be determined from the N mask sums.
[0185] Step S803: Use the M mask matrices corresponding to each first column index value in the first matrix as deactivation mask matrices.
[0186] After determining the inactivation mask matrix, all values in the inactivation mask matrix can be set to 0.
[0187] For example, the first matrix consists of 2*3 mask matrices, each of which is 3*3 in size. The first matrix is as follows: Figure 9 As shown.
[0188] right Figure 9 The values in each mask matrix are summed to obtain the sum of each mask matrix, which is used as the value corresponding to each mask matrix, as shown in Table 4.
[0189] Table 4.
[0190] 2.4 3.4 3.4 4.2 2.8 2.9
[0191] The values corresponding to each column of the mask matrix in the first matrix are added together to obtain the mask sum corresponding to each column of the mask matrix, as shown in Table 5.
[0192] Table 5.
[0193] 6.6 6.2 6.3
[0194] Let L be 2. From the above 3 mask sums, determine the 2 smallest mask sums, which are 6.2 and 6.3, respectively. Their corresponding first column index values are 2 and 3.
[0195] Finally, the M mask matrices corresponding to the first column index values (2 and 3) in the first matrix are used as the deactivation mask matrices, as shown in the figure. Figure 10 As shown.
[0196] In this application embodiment, a method for conveniently determining the inactivation mask matrix corresponding to the feature extraction layer is provided.
[0197] Optionally, in step S203 above, for any feature extraction layer, deleting the filter corresponding to the inactivation mask matrix from the feature extraction layer based on the inactivation mask matrix corresponding to the feature extraction layer may include the following steps:
[0198] For the j-th feature extraction layer, perform the following steps: First, based on the inactivation mask matrix of the j-th feature extraction layer, determine at least one second column index value, and delete the M filters corresponding to each second column index value in the second matrix; then, based on the inactivation mask matrix corresponding to the (j-1)-th feature extraction layer, determine at least one first row index value, and delete the N filters corresponding to each first row index value in the second matrix.
[0199] For example, the first matrix consists of 2*3 mask matrices, each of which is 3*3 in size. The first matrix is as follows: Figure 9 As shown. The second matrix includes 2*3 filters, each of size 3*3. The second matrix is as follows. Figure 11 As shown.
[0200] Based on the inactivation mask matrix of the j-th feature extraction layer, at least one second column index value of 2 is determined. The filter corresponding to the second column in the second matrix is then deleted. The second matrix is as follows: Figure 12As shown.
[0201] Based on the inactivation mask matrix of the (j-1)th feature extraction layer, at least one first row index value is determined to be 1. The filter corresponding to the first row in the second matrix is then deleted. The second matrix is as follows: Figure 13 As shown. Finally, the filters included in the j-th feature extraction layer are as follows: Figure 13 As shown.
[0202] In this application embodiment, a convenient method for determining the filter of the feature extraction layer based on the deactivation mask matrix is provided.
[0203] Based on the same technical concept, embodiments of this application provide an image classification model generation apparatus, such as... Figure 14 As shown, the image classification model generation device 1400 includes:
[0204] Training module 1401 is used to train an image classification model using images to be classified with preset labels. The image classification model includes multiple feature extraction layers, each feature extraction layer includes multiple filters and corresponding multiple mask matrices. The size of each filter and the corresponding mask matrix is the same. The multiple filters are used to extract different features from the images to be classified. Each mask matrix is used to adjust the features extracted by the corresponding filter. The parameters in the multiple filters and the parameters in the multiple mask matrices are adjusted during training.
[0205] The acquisition module 1402 is used to, after at least one training session, for any feature extraction layer, acquire the inactive mask matrix corresponding to the feature extraction layer from multiple mask matrices of the feature extraction layer, wherein the inactive mask matrix is used to indicate the mask matrices in the feature extraction layer that do not participate in subsequent training.
[0206] The deletion module 1403 is used to delete the filter corresponding to the inactivation mask matrix from any feature extraction layer after training, based on the inactivation mask matrix corresponding to the feature extraction layer, so as to obtain the trained image classification model.
[0207] Optionally, the image classification model further includes a classification layer and a loss function layer;
[0208] The training module 1401 is specifically used for:
[0209] In any of the at least one training session, the following steps are performed:
[0210] Based on the filters and mask matrices of the multiple feature extraction layers, feature extraction is performed on the image to be classified to obtain the first feature;
[0211] The first feature is input into the classification layer to obtain the first predicted category;
[0212] The first predicted category and the preset label are input into the loss function layer to obtain the first loss function value; based on the first loss function value, the parameters in multiple filters of each feature extraction layer are updated;
[0213] For any feature extraction layer, a second loss function value corresponding to the feature extraction layer is determined based on multiple mask matrices of the feature extraction layer; and the parameters in the multiple mask matrices of the feature extraction layer are updated based on the first loss function value and the second loss function value.
[0214] Optionally, the training module 1401 is specifically used for:
[0215] For the j-th feature extraction layer, the following steps are performed: Based on the filter and mask matrix of the j-th feature extraction layer, determine the sparse filter of the j-th feature extraction layer; use the sparse filter of the j-th feature extraction layer to extract features from the feature vector corresponding to the (j-1)-th feature extraction layer to obtain the feature vector corresponding to the j-th feature extraction layer; where j>=1;
[0216] The feature vector corresponding to the last feature extraction layer is used as the first feature.
[0217] Optionally, the training module 1401 is specifically used for:
[0218] The second norm is used to process multiple mask matrices of the feature extraction layer to obtain the first mask array of the feature extraction layer;
[0219] The first mask array of the feature extraction layer is processed using the first norm to obtain the second loss function value of the feature extraction layer.
[0220] Optionally, the feature extraction layer includes M*N mask matrices; the M*N mask matrices form a first matrix;
[0221] The acquisition module 1402 is specifically used for:
[0222] For the i-th training iteration in the at least one training iteration, determine the pruning amount L corresponding to the feature extraction layer in the i-th training iteration; where i>=1, 1<=L<=N;
[0223] From the first matrix, obtain the inactivation mask matrix corresponding to the feature extraction layer in the i-th training, the inactivation mask matrix including M*L mask matrices.
[0224] Optionally, the acquisition module 1402 is further configured to:
[0225] In the case where the at least one training session includes the i-th training session and the (i+1)-th training session, the pruning amount of the feature extraction layer corresponding to the (i+1)-th training session is greater than or equal to the pruning amount corresponding to the i-th training session.
[0226] Optionally, the acquisition module 1402 is specifically used for:
[0227] Determine the pruning rate of the feature extraction layer corresponding to the i-th training iteration;
[0228] Based on the pruning rate of the feature extraction layer in the i-th training iteration and the initial number of filters in the feature extraction layer, the pruning amount L of the feature extraction layer in the i-th training iteration is determined.
[0229] Optionally, the acquisition module 1402 is specifically used for:
[0230] Based on the preset initial pruning rate, the preset target pruning rate, the preset initial number of cycles, and the number of cycles corresponding to the i-th training, the pruning adjustment amount of the feature extraction layer corresponding to the i-th training is determined;
[0231] Based on the preset target pruning rate and the pruning adjustment amount, the pruning rate corresponding to the feature extraction layer in the i-th training is determined.
[0232] Optionally, the acquisition module 1402 is specifically used for:
[0233] Calculate the mask matrix for each column of the first matrix to obtain N mask sums;
[0234] Based on the first preset rule, L mask sums are determined from the N mask sums, and based on the first matrix, the first column index values corresponding to the L mask sums are determined respectively;
[0235] Use the M mask matrices corresponding to each first column index value in the first matrix as the deactivation mask matrix.
[0236] Optionally, the feature extraction layer further includes M*N filters, which form a second matrix;
[0237] The deletion module 1403 is specifically used for:
[0238] For the j-th feature extraction layer, perform the following steps:
[0239] Based on the inactivation mask matrix of the j-th feature extraction layer, at least one second column index value is determined, and the M filters corresponding to each second column index value in the second matrix are deleted.
[0240] Based on the inactivation mask matrix corresponding to the (j-1)th feature extraction layer, determine at least one first row index value, and delete the N filters corresponding to each first row index value in the second matrix.
[0241] Based on the same technical concept, embodiments of this application provide a computer device, which may be a terminal or a server, such as... Figure 15 As shown, it includes at least one processor 1501 and a memory 1502 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1501 and the memory 1502 is not limited. Figure 15 Taking the connection between processor 1501 and memory 1502 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.
[0242] In this embodiment of the application, the memory 1502 stores instructions that can be executed by at least one processor 1501. By executing the instructions stored in the memory 1502, at least one processor 1501 can perform the steps included in the above-described image classification model generation method.
[0243] The processor 1501 is the control center of the computer device. It can connect to various parts of the computer device using various interfaces and lines. It generates image classification models by running or executing instructions stored in the memory 1502 and calling data stored in the memory 1502. Optionally, the processor 1501 may include one or more processing units. The processor 1501 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1501. In some embodiments, the processor 1501 and the memory 1502 can be implemented on the same chip; in some embodiments, they can also be implemented on separate chips.
[0244] Processor 1501 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0245] Memory 1502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1502 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1502 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0246] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described image classification model generation method.
[0247] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0248] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0249] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0250] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0251] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. An image classification model generation method, characterized by, The method comprises the following steps: training an image classification model by using images to be classified with preset labels; the image classification model comprises a plurality of feature extraction layers, each feature extraction layer comprises a plurality of filters and a plurality of corresponding mask matrices, the size of each filter and the corresponding mask matrix is the same, the plurality of filters are used to extract different features from the images to be classified, each mask matrix is used to adjust the features extracted by the corresponding filter, and the parameters in the plurality of filters and the parameters in the plurality of mask matrices are adjusted in the training process; after the end of at least one training, for any feature extraction layer, an inactivation mask matrix corresponding to the feature extraction layer is obtained from the plurality of mask matrices of the feature extraction layer, and the inactivation mask matrix is used to indicate the mask matrices in the feature extraction layer that do not participate in subsequent training; after the end of training, for any feature extraction layer, the filter corresponding to the inactivation mask matrix of the feature extraction layer is deleted from the feature extraction layer based on the inactivation mask matrix corresponding to the feature extraction layer, thereby obtaining a trained image classification model; The feature extraction layer includes M N mask matrices; the M N mask matrices form a first matrix; the step of obtaining the inactivation mask matrix corresponding to the feature extraction layer from the plurality of mask matrices of the feature extraction layer comprises the following steps: calculating the mask matrix of each column in the first matrix to obtain N mask sums; determining L mask sums from the N mask sums based on a first preset rule, and determining a first column index value corresponding to each of the L mask sums based on the first matrix; and taking the M mask matrices corresponding to each first column index value in the first matrix as inactivation mask matrices; The feature extraction layer further includes M N filters, the M N filters form a second matrix; the step of deleting the filter corresponding to the inactivation mask matrix from the feature extraction layer based on the inactivation mask matrix corresponding to the feature extraction layer for any feature extraction layer comprises the following steps: for the jth feature extraction layer, the following steps are performed: determining at least one second column index value based on the inactivation mask matrix of the jth feature extraction layer, and deleting the M filters corresponding to each second column index value in the second matrix; determining at least one first row index value based on the inactivation mask matrix corresponding to the j-1th feature extraction layer, and deleting the N filters corresponding to each first row index value in the second matrix.
2. The method of claim 1, wherein, the image classification model further comprises a classification layer and a loss function layer; the step of training the image classification model by using images to be classified with preset labels comprises the following steps: in any one of the at least one training, the following steps are performed: performing feature extraction on the images to be classified based on the filters and mask matrices of the plurality of feature extraction layers to obtain first features; inputting the first features into the classification layer to obtain a first predicted category; inputting the first predicted category and the preset label into the loss function layer to obtain a first loss function value; and updating the parameters in the plurality of filters of each feature extraction layer based on the first loss function value; For any feature extraction layer, a second loss function value corresponding to the feature extraction layer is determined based on a plurality of mask matrices of the feature extraction layer; and parameters in the plurality of mask matrices of the feature extraction layer are updated based on the first loss function value and the second loss function value.
3. The method of claim 2, wherein, The feature extraction of the to-be-classified image based on the plurality of filter and mask matrices of the plurality of feature extraction layers obtains first features, comprising: For the jth feature extraction layer, the following steps are performed: determining a sparse filter of the jth feature extraction layer based on the filter and the mask matrix of the jth feature extraction layer; performing feature extraction on the feature vector corresponding to the j-1th feature extraction layer by using the sparse filter of the jth feature extraction layer to obtain a feature vector corresponding to the jth feature extraction layer; wherein j>=1; The feature vector corresponding to the last feature extraction layer is taken as the first feature.
4. The method of claim 2, wherein, The determination of the second loss function value corresponding to the feature extraction layer based on the plurality of mask matrices of the feature extraction layer comprises: The plurality of mask matrices of the feature extraction layer are processed by using the second norm to obtain a first mask array of the feature extraction layer; The first mask array of the feature extraction layer is processed by using the first norm to obtain the second loss function value of the feature extraction layer.
5. The method of claim 1, wherein, The obtaining of the inactivation mask matrix corresponding to the feature extraction layer from the plurality of mask matrices of the feature extraction layer comprises: For the ith training in the at least one training, a pruning amount L corresponding to the feature extraction layer in the ith training is determined; wherein i>=1, 1<=L<=N; From the first matrix, an inactivation mask matrix corresponding to the i-th training of the feature extraction layer is obtained, and the inactivation mask matrix includes M L mask matrices.
6. The method of claim 5, wherein, Further comprising: In the case that the at least one training includes the ith training and the ith+1 training, the pruning amount corresponding to the feature extraction layer in the ith+1 training is greater than or equal to the pruning amount corresponding to the feature extraction layer in the ith training.
7. The method of claim 5, wherein, The determination of the pruning amount L corresponding to the feature extraction layer in the ith training comprises: A pruning rate corresponding to the feature extraction layer in the ith training is determined; Based on the pruning rate corresponding to the feature extraction layer in the ith training and the initial number of filters of the feature extraction layer, the pruning amount L corresponding to the feature extraction layer in the ith training is determined.
8. The method of claim 7, wherein, The determination of the pruning rate corresponding to the feature extraction layer in the ith training comprises: Based on a preset initial pruning rate, a preset target pruning rate, a preset initial period number and a period number corresponding to the ith training, a pruning adjustment amount corresponding to the feature extraction layer in the ith training is determined; Based on the preset target pruning rate and the pruning adjustment amount, the pruning rate corresponding to the feature extraction layer in the ith training is determined.
9. An image classification model generation apparatus characterized by comprising: Comprising: The training module is configured to train the image classification model by using the to-be-classified image with a preset label. The image classification model comprises a plurality of feature extraction layers, each feature extraction layer comprising a plurality of filters and a corresponding plurality of mask matrices, each filter and the corresponding mask matrix being of the same size, the plurality of filters being used to extract different features from the image to be classified, each mask matrix being used to adjust the features extracted by the corresponding filter, parameters in the plurality of filters and parameters in the plurality of mask matrices being adjusted during a training process; The acquisition module is configured to, after at least one training is completed, acquire, for any feature extraction layer, a corresponding inactive mask matrix of the feature extraction layer from a plurality of mask matrices of the feature extraction layer, the inactive mask matrix being used to indicate a mask matrix in the feature extraction layer that does not participate in subsequent training; The deletion module is configured to, after training is completed, delete, for any feature extraction layer, a filter corresponding to an inactive mask matrix of the feature extraction layer based on the corresponding inactive mask matrix of the feature extraction layer, thereby obtaining a trained image classification model. The feature extraction layer includes M N mask matrices; the M N mask matrices form a first matrix; The acquisition module is specifically configured to: calculate the mask matrix of each column in the first matrix to obtain N mask sums; determine L mask sums from the N mask sums based on a first preset rule, and determine a first column index value corresponding to each of the L mask sums based on the first matrix; and take the M mask matrices corresponding to each first column index value in the first matrix as the inactive mask matrix. The feature extraction layer further includes M N filters, the M N filters form a second matrix; The deletion module is specifically configured to: for the jth feature extraction layer, perform the following steps: determine at least one second column index value based on the inactive mask matrix of the jth feature extraction layer, and delete the M filters corresponding to each second column index value in the second matrix; determine at least one first row index value based on the inactive mask matrix corresponding to the j-1th feature extraction layer, and delete the N filters corresponding to each first row index value in the second matrix.
10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method of any one of claims 1-8.
11. A computer readable storage medium, characterized in that, The computer device stores a computer program executable by the computer device, and when the program runs on the computer device, the computer device executes the steps of the method of any one of claims 1-8.
Citation Information
Patent Citations
Unmanned aerial vehicle image real-time target detection method based on channel pruning
CN113128355A