Tool image classification model pruning method, device, and computer equipment

Abstract

The embodiment of the present invention discloses a method, device, and computer equipment for pruning a tool image classification model. The method comprises: obtaining a trained image classification model; evaluating the importance of each channel of the trained image classification model to the image classification task; specifically, for each channel in the network layer to be deleted, calculating the corresponding importance weight according to the degree of discrimination of the channel for samples of the same type and samples of different types to obtain the importance of each channel to the image classification task; deleting the channels whose importance does not meet the requirements to obtain a pruned image classification model; and fine-tuning the classification training of the pruned image classification model to determine the final image classification model. The present invention can prune certain model channels in the image classification neural network, reduce the number of parameters of the model, improve the runtime efficiency of the model, reduce resource consumption, and thus improve the accuracy of tool recognition and classification efficiency.

Description

Technical Field

[0001] The present invention relates to a deep learning network, and more specifically to a tool image classification model pruning method, device, and computer equipment. Background Art

[0002] In the process of product production, tools are needed to cut the products. Due to process requirements, a product production process requires tools of various specifications. However, on the current assembly line, tools are often selected manually and installed on machine tools to cut the products. After cutting, they are manually returned to the designated tool position. This method of manual tool selection has a high probability of misselection and management confusion.

[0003] There are many intelligent tool management cabinets in the prior art to solve the problem of intelligent tool management. However, the existing intelligent tool management cabinets generally rely on the tool information and preset tool positions entered by the user when borrowing the tool to manage the tool. When returning the tool, the user needs to put the tool into the designated space according to the instructions of the management cabinet, and then rely on the mechanical classification inside the cabinet to complete the tool type identification and storage. This tool identification method is not intelligent and it is easy to have inaccurate tool information identification and classification errors.

[0004] The tool automatic identification system in the tool management cabinet can automatically identify the tools returned by the user, thereby reducing the user's manual input of tool categories. The tool automatic identification system uses a tool identification algorithm model. The more complex the model structure of the neural network used and the more parameters it has, the richer the knowledge the network can learn and the better the learning effect. However, the high storage space and large amount of computing resource consumption make it difficult for large network models to be widely used in various hardware platforms, including tool automatic identification platforms.

[0005] Therefore, it is necessary to design a new method to trim certain model channels in the image classification neural network, reduce the number of model parameters, improve the model's runtime efficiency, and reduce resource consumption, so that it can be applied to the tool automatic recognition platform to achieve efficient recognition results. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device and computer equipment for pruning a tool image classification model.

[0007] To achieve the above object, the present invention adopts the following technical solution: a tool image classification model pruning method, comprising:

[0008] Get the trained image classification model;

[0009] Evaluate the importance of each channel to the image classification task for the trained image classification model; specifically, for each channel in the deleted network layer, calculate the corresponding importance weight according to the channel's discrimination between samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task;

[0010] Deleting the channels whose importance does not meet the requirement to obtain a cropped image classification model;

[0011] Fine-tune the classification training on the cropped image classification model to determine the final image classification model.

[0012] A further technical solution is: the step of obtaining a trained image classification model includes:

[0013] Get images with category labels to get a training dataset;

[0014] Construct a neural network model to obtain an initial model;

[0015] Inputting the training data set into the initial model for training until the initial model converges to obtain a trained image classification model;

[0016] Set the network layers to be deleted from the trained image classification model.

[0017] A further technical solution is: the step of evaluating the importance of each channel to the image classification task for the trained image classification model includes:

[0018] Input the training data into the trained image classification model, and store the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model;

[0019] For each corresponding channel in the channel feature map of the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

[0020] A further technical solution is: for each channel corresponding to the channel feature map of the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task, including:

[0021] For each channel feature map in the network layer to be deleted, initialize the importance weight corresponding to each channel feature map;

[0022] Determine whether the current number of iterations has reached the specified number and / or whether the importance weight has stabilized;

[0023] If the current number of iterations does not reach the specified number and the importance weight does not tend to be stable, a training image is randomly selected, and several images of the same type are selected with similarity that meet the requirements to obtain the first set; specifically, the similarity is the cosine distance similarity of the final feature vector;

[0024] Among the different types of images of the training images, a number of images of each type that meet the requirements of similarity are selected to obtain a second set; specifically, the similarity is the cosine distance similarity of the final feature vector;

[0025] For each corresponding channel feature map in the channel feature map of the network layer to be deleted, updating the importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set;

[0026] Record the current number of iterations, and perform the determination as to whether the current number of iterations reaches a specified number and / or whether the importance weight tends to be stable;

[0027] If the current number of iterations reaches the specified number and\or the importance weight tends to be stable, the importance weight corresponding to each channel feature map is determined as the importance of each channel to the image classification task.

[0028] A further technical solution is that for each corresponding channel feature map in the channel feature map of the network layer to be deleted, the importance weight corresponding to each channel feature map is updated according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set, including:

[0029] For each channel feature map corresponding to the channel feature map of the network layer to be deleted, w is used i =w i -dis i (r,P)+dis i (r,Q) updates the importance weight corresponding to each channel feature map, where w i is the importance weight corresponding to the i-th channel feature map, w before the equal sign i Indicates the updated importance weight, w after the equal sign i represents the importance weight before updating, dis i (r,P) represents the average distance between the channel feature map of the training image and the channel feature map of the first set, i (r, Q) represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the feature map of the ith channel of image j in the cth class of images. Image j comes from set Q, which is composed of images with different categories and similar feature vectors as image r. Set Q is the second set. N is a natural number, and N+1 is the total number of types of all data sets. K is the number of images selected from each different category, and K is less than the number of images in the type with the smallest number of images among all categories of images. K is a natural number, and N×K is the total number of images in the first set.

[0030] A further technical solution is: deleting the channels whose importance does not meet the requirements to obtain a cropped image classification model includes:

[0031] Traversing each of the network layers to be deleted to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result;

[0032] A number of channels whose importance weights meet the requirements in the sorting results are deleted to obtain a cropped image classification model.

[0033] A further technical solution is: the fine-tuning classification training of the cropped image classification model to determine the final image classification model includes:

[0034] The cropped image classification model is fine-tuned for classification training using the cross entropy loss function and the training dataset to determine the final image classification model.

[0035] The present invention also provides a tool image classification model pruning device, comprising:

[0036] A model acquisition unit, used to acquire a trained image classification model;

[0037] An evaluation unit is used to evaluate the importance of each channel to the image classification task for the trained image classification model; specifically, for each channel in the deleted network layer, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task;

[0038] A deletion unit, used for deleting the channels whose importance does not meet the requirement, so as to obtain a cropped image classification model;

[0039] The fine-tuning unit is used to perform fine-tuning classification training on the cropped image classification model to determine the final image classification model.

[0040] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0041] Compared with the prior art, the present invention has the following beneficial effects: the present invention evaluates the importance of each channel in the network layer to be deleted of the trained image classification model to the image classification task, determines the importance weight corresponding to each channel feature map, determines the channels that do not meet the requirements according to the importance weight, deletes these channels, and then fine-tunes the pruned image classification model to form the final image classification model, thereby pruning some model channels in the image classification neural network, reducing the number of model parameters, improving the model's runtime efficiency, and reducing resource consumption. The model is used in the tool image recognition method to achieve higher recognition efficiency and recognition accuracy.

[0042] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of an application scenario of the tool image classification model pruning method provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a process flow of a tool image classification model pruning method provided by an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a sub-process of a tool image classification model pruning method provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a sub-process of a tool image classification model pruning method provided by an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of a sub-process of a tool image classification model pruning method provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of a sub-process of a tool image classification model pruning method provided by an embodiment of the present invention;

[0050] Figure 7A schematic block diagram of a tool image classification model pruning device provided in an embodiment of the present invention;

[0051] Figure 8 A schematic block diagram of a model acquisition unit of a tool image classification model pruning device provided in an embodiment of the present invention;

[0052] Fig. 9 A schematic block diagram of an evaluation unit of a tool image classification model pruning device provided in an embodiment of the present invention;

[0053] Fig.10 A schematic block diagram of a weight calculation subunit of a tool image classification model pruning device provided in an embodiment of the present invention;

[0054] Fig.11 A schematic block diagram of a deletion unit of a tool image classification model pruning device provided in an embodiment of the present invention;

[0055] Fig.12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0058] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0059] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0060] See also Figure 1 and Figure 2 , Figure 1Schematic diagram of application scenarios of the tool image classification model pruning method provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a tool image classification model pruning method provided in an embodiment of the present invention. The tool image classification model pruning method is applied in a server. The server interacts with a terminal for data, and inputs an image through the terminal to perform model training. During the model training process, channels that do not meet the importance requirements are pruned, and then the pruned model is fine-tuned for classification training to form a final image classification model. The image classification model can be used to classify images, and certain model channels in the image classification neural network can be pruned, thereby reducing the number of model parameters, improving the model's runtime efficiency, and reducing resource consumption.

[0061] Figure 2 FIG. 1 is a flow chart of a tool image classification model pruning method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S140.

[0062] S110. Obtain a trained image classification model.

[0063] In this embodiment, the trained image classification model refers to a model formed by training an uncompressed neural network using images with category labels.

[0064] In one embodiment, see Figure 3 , the above-mentioned step S110 may include steps S111 to S114.

[0065] S111. Obtain images with category labels to obtain a training data set.

[0066] In this embodiment, the training data set includes a large number of images and a category corresponding to each image, and it is assumed that the total number of categories is N+1.

[0067] Specifically, these images may be images of cutting tools, but are not limited to images of cutting tools. According to the products that need to be classified in the actual scene, corresponding product images may be photographed and the category labels may be annotated.

[0068] S112. Construct a neural network model to obtain an initial model.

[0069] In this embodiment, an unpruned neural network image classification model M is constructed, such as ResNet101.

[0070] S113, inputting the training data set into the initial model for training until the initial model converges to obtain a trained image classification model.

[0071] In this embodiment, training the initial model for the training data set until the initial model converges belongs to the prior art and will not be described in detail here.

[0072] S114. Setting a network layer to be deleted for the trained image classification model.

[0073] In this embodiment, the network layer to be deleted refers to the network layer that needs to perform channel pruning. Not every network layer needs to be deleted. For example, the input layer and the output layer, which have only a single layer and a single channel, do not need to be pruned. Network layers with multiple channels and more complex parameters do need to be pruned. Which specific network layers need to be deleted can be determined according to the actual situation. For example, the number of channels and the complexity of the parameters can be set as limiting conditions to set the network layers that need to be deleted.

[0074] S120. Evaluate the importance of each channel to the image classification task for the trained image classification model.

[0075] In this embodiment, importance refers to the contribution of each channel in the image classification model to image classification.

[0076] In one embodiment, see Figure 4 , the above-mentioned step S120 may include steps S121 to S122.

[0077] S121, inputting the training data into the trained image classification model, and storing the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model.

[0078] In this embodiment, the training data is input into the trained image classification model for processing to determine the channel feature map of the network layer to be deleted corresponding to each image and the feature vector of the final output.

[0079] S122. For each corresponding channel in the channel feature map of the network layer to be deleted, calculate the corresponding importance weight according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

[0080] In this embodiment, the importance of each channel to the image classification task refers to the importance weight corresponding to the degree of discrimination of each channel in the network layer to be deleted for samples of the same type and samples of different types.

[0081] In one embodiment, see Figure 5 , the above-mentioned step S122 may include steps S1221 to S1227.

[0082] S1221. For each channel feature map in the network layer to be deleted, initialize the importance weight corresponding to each channel feature map.

[0083] In this embodiment, for each channel feature map in the network layer to be deleted, its corresponding importance weight w is initialized to 0.

[0084] S1222. Determine whether the current number of iterations reaches the specified number and / or whether the importance weight tends to be stable.

[0085] In this embodiment, the following steps S1223 to S1226 are executed until the specified number of iterations is reached or the importance weight tends to be stable, and the importance weight w of each channel in the network layer to be deleted is obtained.

[0086] S1223. If the current number of iterations does not reach the specified number and the importance weight has not stabilized, a training image is randomly selected, and several images of the same type are selected whose similarity meets the requirements to obtain a first set.

[0087] In this embodiment, the first set refers to a set consisting of a number of images of the same type as the training images whose approximation meets the requirements. Specifically, the approximation is the cosine distance similarity of the final feature vector, and the images of the same type that meet the requirements refer to the images of the first N*K approximations after sorting the approximations of all images of the same type from high to low.

[0088] Specifically, a training image r is randomly selected in the training data set, and according to its feature vector v, a set P of N*K images with the closest feature vectors are selected from images of the same type, thereby forming a first set, where N is a natural number, N+1 is the total number of types of all data sets; K is the number of images selected from each different category, K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number.

[0089] S1224. Select a number of images of different categories of the training images in each category whose similarity meets the requirements, to obtain a second set.

[0090] In this embodiment, the second set refers to a set consisting of a number of images of each class of images in different classes of training images, each of which has a similarity that meets the requirements. Specifically, the similarity is the cosine distance similarity of the final feature vector, and the number of the second set is K;

[0091] Among different types of images, each type of image selects a set Q of images with K images whose feature vectors are most similar. The similarity of feature vectors is described by the cosine distance of feature vectors, that is: sim(v 1 ,v 2 )=cos(v1 ,v 2 ).

[0092] S1225. For each corresponding channel feature map in the channel feature map of the network layer to be deleted, update the importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set.

[0093] In this embodiment, the average distance between a channel feature map and other channel feature maps represents the degree of differentiation, and the greater the average distance, the greater the degree of differentiation. Specifically:

[0094] For each channel feature map corresponding to the channel feature map of the network layer to be deleted, w is used i =w i -dis i (r,P)+dis i (r,Q) updates the importance weight corresponding to each channel feature map, where w i is the importance weight corresponding to the i-th channel feature map, w before the equal sign i Indicates the updated importance weight, w after the equal sign i represents the importance weight before updating, dis i (r,P) represents the average distance between the channel feature map of the training image and the channel feature map of the first set, i (r, Q) represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents the one-dimensional vector of the feature map of the ith channel of image j in the cth category, image j comes from set Q, set Q consists of images of different categories from image r and with similar feature vectors, set Q is the second set; N is a natural number, N+1 is the total number of types of all data sets; K is the number of images selected from each different category, K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number, N×K is the total number of images in the first set.

[0095] S1226. Record the current number of iterations and execute step S1222.

[0096] S1227. If the current number of iterations reaches the specified number and / or the importance weight tends to be stable, the importance weight corresponding to each channel feature map is determined as the importance of each channel to the image classification task.

[0097] In this embodiment, the importance weight w of each channel in the network layer to be deleted is determined i Importance of each channel for the image classification task.

[0098] S130, deleting the channels whose importance does not meet the requirement to obtain a cropped image classification model.

[0099] In this embodiment, the pruned image classification model refers to the model remaining after the channels whose importance does not meet the requirements in the trained image classification model are pruned.

[0100] The importance not meeting the requirements here means that the importance weight meets the requirements, specifically refers to the first n% of channels after the importance weight is sorted from low to high.

[0101] In one embodiment, see Figure 6 , the above-mentioned step S130 may include steps S131 to S132.

[0102] S131 , traversing each of the network layers to be deleted to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result.

[0103] In this embodiment, the sorting result refers to the result formed by sorting the importance weights of all channels in the network layer to be deleted from low to high.

[0104] S132, deleting a number of channels whose importance weights meet the requirements in the sorting results to obtain a cropped image classification model.

[0105] In this embodiment, the plurality of channels whose importance weights meet the requirements refer to the first n% of the channels in the sorting results.

[0106] In this embodiment, each network layer to be deleted is traversed, and the importance weights w of all channels in the layer are sorted. The first n% of channels after the importance weights are sorted from low to high are deleted, and after the above processing is performed on all network layer channels and corresponding weights, a deleted model is obtained. At this time, n can be determined according to the actual situation.

[0107] S140, fine-tune the cropped image classification model to determine the final image classification model.

[0108] In this embodiment, the cropped image classification model is fine-tuned and classified using a cross entropy loss function and a training data set to determine a final image classification model.

[0109] Specifically, for the cropped image classification model, the training data set is used again for fine-tuning classification training. Specifically, the cross entropy loss function and the training data set are used to set a smaller learning rate for model update, and the model is fine-tuned for classification training to obtain the final image classification model.

[0110] In this embodiment, a large model with higher accuracy is first trained, that is, a trained image classification model. After the large model is trained, the importance of the channels in the model is evaluated, and those channels with lower importance can be deleted, thereby obtaining a small model with higher accuracy and fewer channels. Since the small model has fewer channels, fewer parameters, and faster running speed, it can be easily deployed on various hardware platforms.

[0111] In this embodiment, the average distance between a channel feature map and other channel feature maps represents the discrimination degree. The larger the average distance, the greater the discrimination degree. Specifically: the importance of each channel is described by its corresponding importance weight. For each channel, its importance weight is calculated according to its discrimination degree for samples of the same type and samples of different types. The closer the channel value is in samples of the same type, the greater the importance weight is. The closer the channel value is in samples of different types, the smaller the importance weight is. Specifically, assuming that there are N+1 categories in the image data set, a training image R is randomly selected, and a set of N*K most similar sample images P is selected from the same type of images. In images of different types, a set of K most similar sample images Q is selected for each category. Different training images are repeatedly selected until the specified number of iterations is reached or the importance weight tends to be stable.

[0112] According to the degree of discrimination of each channel feature map in the neural network for samples of the same type and samples of different types, the contribution and importance of each channel to the image classification task are calculated. Model channels that are not important to downstream tasks in the image classification neural network are pruned to reduce the number of model parameters, improve the model's runtime efficiency, and reduce resource consumption.

[0113] The above-mentioned tool image classification model pruning method evaluates the importance of each channel in the network layer to be deleted of the trained image classification model to the image classification task, determines the importance weight corresponding to each channel feature map, determines the channels that do not meet the requirements according to the importance weights, deletes these channels, and then fine-tunes the pruned image classification model to form the final image classification model, thereby pruning certain model channels in the image classification neural network, reducing the number of model parameters, improving the model's runtime efficiency, and reducing resource consumption.

[0114] Figure 7 is a schematic block diagram of a tool image classification model pruning device 300 provided in an embodiment of the present invention. Figure 7 As shown, corresponding to the above tool image classification model pruning method, the present invention also provides a tool image classification model pruning device 300. The tool image classification model pruning device 300 includes a unit for executing the above tool image classification model pruning method, and the device can be configured in a server. Figure 7 The tool image classification model pruning device 300 includes a model acquisition unit 301, an evaluation unit 302, a deletion unit 303 and a fine-tuning unit 304.

[0115] The model acquisition unit 301 is used to acquire a trained image classification model; the evaluation unit 302 is used to evaluate the importance of each channel of the trained image classification model to the image classification task; specifically, for each channel in the deleted network layer, the corresponding importance weight is calculated according to the channel's discrimination between samples of the same type and samples of different types to obtain the importance of each channel to the image classification task; the deletion unit 303 is used to delete the channels whose importance does not meet the requirements to obtain a cropped image classification model; the fine-tuning unit 304 is used to perform fine-tuning classification training on the cropped image classification model to determine the final image classification model.

[0116] In one embodiment, if Figure 8 As shown, the model acquisition unit 301 includes an image acquisition subunit 3011, an initial model construction subunit 3012, a training subunit 3013 and a setting subunit 3014.

[0117] The image acquisition subunit 3011 is used to acquire images with category labels to obtain a training data set; the initial model construction subunit 3012 is used to build a neural network model to obtain an initial model; the training subunit 3013 is used to input the training data set into the initial model for training until the initial model converges to obtain a trained image classification model; the setting subunit 3014 is used to set the network layer to be deleted for the trained image classification model.

[0118] In one embodiment, if Fig. 9 As shown, the evaluation unit 302 includes a storage subunit 3021 and a weight calculation subunit 3022 .

[0119] The storage subunit 3021 is used to input the training data into the trained image classification model, and store the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model; the weight calculation subunit 3022 is used to calculate the corresponding importance weight for each channel corresponding to the channel feature map of the network layer to be deleted according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

[0120] In one embodiment, if Fig.10 As shown, the weight calculation subunit 3022 includes an initialization module 30221, a judgment module 30222, a first set determination module 30223, a second set determination module 30224, a weight update module 30225, a recording module 30226 and a determination module 30227.

[0121] The initialization module 30221 is used to initialize the importance weight corresponding to each channel feature map in the network layer to be deleted; the judgment module 30222 is used to judge whether the current number of iterations has reached the specified number of times and / or whether the importance weight has tended to be stable; the first set determination module 30223 is used to randomly select a training image if the current number of iterations has not reached the specified number of times and the importance weight has not tended to be stable, and select several images of the same type whose approximation meets the requirements to obtain the first set; specifically, the approximation is the cosine distance similarity of the final feature vector; the second set determination module 30224 is used to select several images of different types of images of the training image, each type, whose approximation meets the requirements to obtain the second set; specifically, the approximation is the cosine distance similarity of the final feature vector; a weight updating module 30225, for updating the importance weight corresponding to each channel feature map in the channel feature map of the network layer to be deleted according to the channel feature map of the training image, the channel feature map of the first set and the channel feature map of the second set; a recording module 30226, for recording the current number of iterations, and executing the judgment whether the current number of iterations reaches the specified number and / or the importance weight tends to be stable; a determination module 30227, for determining the importance weight corresponding to each channel feature map as the importance of each channel to the image classification task if the current number of iterations reaches the specified number and\or the importance weight tends to be stable.

[0122] In one embodiment, the weight updating module 30225 is used to update each channel feature map corresponding to the channel feature map of the network layer to be deleted using w i =w i -dis i (r,P)+dis i(r,Q) updates the importance weight corresponding to each channel feature map, where w i is the importance weight corresponding to the i-th channel feature map, w before the equal sign i Indicates the updated importance weight, w after the equal sign i represents the importance weight before updating, dis i (r,P) represents the average distance between the channel feature map of the training image and the channel feature map of the first set, i (r, Q) represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents the one-dimensional vector of the feature map of the ith channel of image j in the cth category, image j comes from set Q, set Q consists of images of different categories from image r and with similar feature vectors, set Q is the second set; N is a natural number, N+1 is the total number of types of all data sets; K is the number of images selected from each different category, K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number, N×K is the total number of images in the first set.

[0123] In one embodiment, if Fig.11 As shown, the deletion unit 303 includes a sorting subunit 3031 and a clipping subunit 3032 .

[0124] The sorting subunit 3031 is used to traverse each of the network layers to be deleted to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result; the cropping subunit 3032 is used to delete several channels whose importance weights meet the requirements in the sorting result to obtain a cropped image classification model.

[0125] In one embodiment, the fine-tuning unit 304 is used to perform fine-tuning classification training on the cropped image classification model using a cross entropy loss function and a training data set to determine a final image classification model.

[0126] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned tool image classification model pruning device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0127] The tool image classification model pruning device 300 can be implemented in the form of a computer program. Fig.12 Runs on the computer device shown.

[0128] See also Fig.12 , Fig.12 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0129] See also Fig.12 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0130] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a tool image classification model pruning method.

[0131] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0132] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a tool image classification model pruning method.

[0133] The network interface 505 is used to communicate with other devices over the network. Fig.12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0135] Obtain a trained image classification model; evaluate the importance of each channel of the trained image classification model to the image classification task; specifically, for each channel in the deleted network layer, calculate the corresponding importance weight according to the degree of discrimination of the channel for samples of the same type and samples of different types to obtain the importance of each channel to the image classification task; delete the channels whose importance does not meet the requirements to obtain a cropped image classification model; perform fine-tuning classification training on the cropped image classification model to determine the final image classification model.

[0136] In one embodiment, when the processor 502 implements the step of obtaining the trained image classification model, it specifically implements the following steps:

[0137] Acquire images with category labels to obtain a training data set; construct a neural network model to obtain an initial model; input the training data set into the initial model for training until the initial model converges to obtain a trained image classification model; set a network layer to be deleted for the trained image classification model.

[0138] In one embodiment, when implementing the step of evaluating the importance of each channel to the image classification task for the trained image classification model, the processor 502 specifically implements the following steps:

[0139] The training data is input into the trained image classification model, and the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model are stored; for each channel corresponding to the channel feature map of the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

[0140] In one embodiment, the processor 502 implements the step of calculating the corresponding importance weight for each channel corresponding to the channel feature map of the network layer to be deleted according to the degree of discrimination of the channel for samples of the same type and samples of different types to obtain the importance of each channel for the image classification task, specifically implementing the following steps:

[0141] For each channel feature map in the network layer to be deleted, initialize the importance weight corresponding to each channel feature map; determine whether the current number of iterations reaches the specified number and / or whether the importance weight tends to be stable; if the current number of iterations does not reach the specified number and the importance weight does not tend to be stable, randomly select a training image, and select several images of the same type whose approximation meets the requirements to obtain a first set; specifically, the approximation is the cosine distance similarity of the final feature vector; among the different types of images in the training image, select several images of each type whose approximation meets the requirements to obtain a second set; specifically, the approximation is the cosine distance similarity of the final feature vector; for each channel feature map corresponding to the channel feature map of the network layer to be deleted, update the importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set and the channel feature map of the second set; record the current number of iterations, and perform the determination of whether the current number of iterations reaches the specified number and / or whether the importance weight tends to be stable; if the current number of iterations reaches the specified number and\or the importance weight tends to be stable, determine the importance weight corresponding to each channel feature map as the importance of each channel to the image classification task.

[0142] In one embodiment, the processor 502 implements the following steps when implementing the step of updating the importance weight corresponding to each channel feature map in the channel feature map of the network layer to be deleted according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set:

[0143] For each channel feature map corresponding to the channel feature map of the network layer to be deleted, w is used i =w i -dis i (r,P)+dis i (r,Q) updates the importance weight corresponding to each channel feature map, where w i is the importance weight corresponding to the i-th channel feature map, w before the equal sign i Indicates the updated importance weight, w after the equal sign i represents the importance weight before updating, dis i (r,P) represents the average distance between the channel feature map of the training image and the channel feature map of the first set, i (r, Q) represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents the one-dimensional vector of the feature map of the ith channel of image j in the cth category, image j comes from set Q, set Q consists of images of different categories from image r and with similar feature vectors, set Q is the second set; N is a natural number, N+1 is the total number of types of all data sets; K is the number of images selected from each different category, K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number, N×K is the total number of images in the first set.

[0144] In one embodiment, when the processor 502 implements the step of deleting the channels that do not meet the importance requirements to obtain the cropped image classification model, the processor 502 specifically implements the following steps:

[0145] Each of the network layers to be deleted is traversed to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result; a number of channels whose importance weights meet the requirements in the sorting result are deleted to obtain a cropped image classification model.

[0146] In one embodiment, when the processor 502 implements the step of fine-tuning the cropped image classification model to determine the final image classification model, the processor 502 specifically implements the following steps:

[0147] The cropped image classification model is fine-tuned for classification training using the cross entropy loss function and the training dataset to determine the final image classification model.

[0148] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0149] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0150] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0151] Obtain a trained image classification model; evaluate the importance of each channel of the trained image classification model to the image classification task; specifically, for each channel in the deleted network layer, calculate the corresponding importance weight according to the degree of discrimination of the channel for samples of the same type and samples of different types to obtain the importance of each channel to the image classification task; delete the channels whose importance does not meet the requirements to obtain a cropped image classification model; perform fine-tuning classification training on the cropped image classification model to determine the final image classification model.

[0152] In one embodiment, when the processor executes the computer program to implement the step of obtaining the trained image classification model, the processor specifically implements the following steps:

[0153] Acquire images with category labels to obtain a training data set; construct a neural network model to obtain an initial model; input the training data set into the initial model for training until the initial model converges to obtain a trained image classification model; set a network layer to be deleted for the trained image classification model.

[0154] In one embodiment, when the processor executes the computer program to implement the step of evaluating the importance of each channel to the image classification task for the trained image classification model, the processor specifically implements the following steps:

[0155] The training data is input into the trained image classification model, and the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model are stored; for each channel corresponding to the channel feature map of the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

[0156] In one embodiment, when the processor executes the computer program to implement the step of calculating the corresponding importance weight for each channel in the channel feature map of the network layer to be deleted according to the degree of discrimination of the channel for samples of the same type and samples of different types to obtain the importance of each channel for the image classification task, the following steps are specifically implemented:

[0157] For each channel feature map in the network layer to be deleted, initialize the importance weight corresponding to each channel feature map; determine whether the current number of iterations reaches the specified number and / or whether the importance weight tends to be stable; if the current number of iterations does not reach the specified number and the importance weight does not tend to be stable, randomly select a training image, and select several images of the same type whose approximation meets the requirements to obtain a first set; specifically, the approximation is the cosine distance similarity of the final feature vector. In different classes of images of the training image, select several images of each class whose approximation meets the requirements to obtain a second set; specifically, the approximation is the cosine distance similarity of the final feature vector; for each channel feature map corresponding to the channel feature map of the network layer to be deleted, update the importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set and the channel feature map of the second set; record the current number of iterations, and perform the determination of whether the current number of iterations reaches the specified number and / or whether the importance weight tends to be stable; if the current number of iterations reaches the specified number and\or the importance weight tends to be stable, determine the importance weight corresponding to each channel feature map as the importance of each channel to the image classification task.

[0158] In one embodiment, when the processor executes the computer program to implement the step of updating the importance weight corresponding to each channel feature map in the channel feature map of the network layer to be deleted according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set, the following steps are specifically implemented:

[0159] For each channel feature map corresponding to the channel feature map of the network layer to be deleted, w is used i =w i -dis i (r,P)+dis i (r,Q) updates the importance weight corresponding to each channel feature map, where w i is the importance weight corresponding to the i-th channel feature map, w before the equal sign i Indicates the updated importance weight, w after the equal sign i represents the importance weight before updating, dis i(r,P) represents the average distance between the channel feature map of the training image and the channel feature map of the first set, i (r, Q) represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents the one-dimensional vector of the feature map of the ith channel of image j in the cth category, image j comes from set Q, set Q consists of images of different categories from image r and with similar feature vectors, set Q is the second set; N is a natural number, N+1 is the total number of types of all data sets; K is the number of images selected from each different category, K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number, N×K is the total number of images in the first set.

[0160] In one embodiment, when the processor executes the computer program to implement the step of deleting the channels that do not meet the importance requirements to obtain a cropped image classification model, the processor specifically implements the following steps:

[0161] Each of the network layers to be deleted is traversed to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result; a number of channels whose importance weights meet the requirements in the sorting result are deleted to obtain a cropped image classification model.

[0162] In one embodiment, when the processor executes the computer program to implement the step of fine-tuning the cropped image classification model to determine the final image classification model, the processor specifically implements the following steps:

[0163] The cropped image classification model is fine-tuned for classification training using the cross entropy loss function and the training dataset to determine the final image classification model.

[0164] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0165] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0167] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0169] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A pruning method for a tool image classification model, characterized in that: include: Get the trained image classification model; Evaluate the importance of each channel for the image classification task for the trained image classification model; Specifically, for each channel in the deleted network layer, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel for the image classification task; Deleting the channels whose importance does not meet the requirement to obtain a cropped image classification model; Fine-tune the cropped image classification model to determine the final image classification model; For each channel in the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task, including: For each channel feature map in the network layer to be deleted, initialize the importance weight corresponding to each channel feature map; Determine whether the current number of iterations has reached the specified number and / or whether the importance weight has stabilized; If the current number of iterations does not reach the specified number and the importance weight does not tend to be stable, a training image is randomly selected, and several images of the same type are selected with similarity that meet the requirements to obtain the first set; specifically, the similarity is the cosine distance similarity of the final feature vector; Among the different types of images of the training images, a number of images of each type that meet the requirements of similarity are selected to obtain a second set; specifically, the similarity is the cosine distance similarity of the final feature vector; For each channel feature map in the network layer to be deleted, updating the importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set; Record the current number of iterations, and perform the determination as to whether the current number of iterations reaches a specified number and / or whether the importance weight tends to be stable; If the current number of iterations reaches the specified number and\or the importance weight tends to be stable, the importance weight corresponding to each channel feature map is determined as the importance of each channel to the image classification task.

2. The tool image classification model pruning method according to claim 1 is characterized in that: The step of obtaining a trained image classification model comprises: Get images with category labels to get a training dataset; Construct a neural network model to obtain an initial model; Inputting the training data set into the initial model for training until the initial model converges to obtain a trained image classification model; Set the network layers to be deleted from the trained image classification model.

3. The tool image classification model pruning method according to claim 2 is characterized in that: The importance of each channel to the image classification task is evaluated for the trained image classification model, including: Input the training data into the trained image classification model, and store the channel feature map of the network layer to be deleted corresponding to each image in the training data and the final feature vector extracted by the trained image classification model; For each channel in the network layer to be deleted, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task.

4. The tool image classification model pruning method according to claim 1, characterized in that: The updating of the importance weight corresponding to each channel feature map in the to-be-deleted network layer according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set includes: For each channel feature map in the network layer to be deleted, + (r,Q) updates the importance weight corresponding to each channel feature map, where is the importance weight corresponding to the i-th channel feature map, and the Represents the updated importance weight, and the represents the importance weight before updating, represents the average distance between the channel feature map of the training image and the channel feature map of the first set, represents the average distance between the channel feature map of the training image and the channel feature map of the second set; and ,in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j, where image j comes from set P, which consists of images with the same category as the training image and similar feature vectors. ,in, Represents the one-dimensional vector of the i-th channel feature map of the training image, Represents a one-dimensional vector of the i-th channel feature map of image j in the c-th category image. Image j comes from set Q, which is composed of images with different categories and similar feature vectors as image r. Set Q is the second set. N is a natural number, and N+1 is the total number of types of all data sets. K is the number of images selected from each different category, and K is less than the number of images in the type with the smallest number of images among all categories of images, and K is a natural number. is the total number of images in the first set.

5. The tool image classification model pruning method according to claim 4 is characterized in that: The deleting of the channels whose importance does not meet the requirement to obtain a cropped image classification model includes: Traversing each of the network layers to be deleted to sort the importance weights of all channels in the network layer to be deleted to obtain a sorting result; A number of channels whose importance weights meet the requirements in the sorting results are deleted to obtain a cropped image classification model.

6. The tool image classification model pruning method according to claim 1, characterized in that: The step of fine-tuning the cropped image classification model to determine the final image classification model includes: The cropped image classification model is fine-tuned for classification training using the cross entropy loss function and the training dataset to determine the final image classification model.

7. A tool image classification model pruning device, characterized in that: include: A model acquisition unit, used to acquire a trained image classification model; An evaluation unit, used to evaluate the importance of each channel for the image classification task for the trained image classification model; Specifically, for each channel in the deleted network layer, the corresponding importance weight is calculated according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel for the image classification task; A deletion unit, used for deleting the channels whose importance does not meet the requirement, so as to obtain a cropped image classification model; A fine-tuning unit, used to perform fine-tuning classification training on the cropped image classification model to determine a final image classification model; The evaluation unit includes a weight calculation subunit; for calculating the corresponding importance weight for each channel in the network layer to be deleted according to the degree of discrimination of the channel for samples of the same type and samples of different types, so as to obtain the importance of each channel to the image classification task; The weight calculation subunit includes an initialization module, a judgment module, a first set determination module, a second set determination module, a weight update module, a recording module and a determination module; An initialization module is used to initialize the importance weight corresponding to each channel feature map in the network layer to be deleted; A judgment module is used to judge whether the current number of iterations reaches a specified number and / or whether the importance weight tends to be stable; The first set determination module is used to randomly select a training image and select a number of images of the same type whose approximation meets the requirements to obtain a first set if the current number of iterations does not reach the specified number and the importance weight does not tend to be stable; specifically, the approximation is the cosine distance similarity of the final feature vector; the second set determination module is used to select a number of images of different types of images of the training image, each type, whose approximation meets the requirements to obtain a second set; specifically, the approximation is the cosine distance similarity of the final feature vector; A weight updating module, configured to update, for each channel feature map in the network layer to be deleted, an importance weight corresponding to each channel feature map according to the channel feature map of the training image, the channel feature map of the first set, and the channel feature map of the second set; A recording module, used to record the current number of iterations, and to perform the determination as to whether the current number of iterations reaches a specified number and / or whether the importance weight tends to be stable; The determination module is used to determine the importance weight corresponding to each channel feature map as the importance of each channel to the image classification task if the current number of iterations reaches the specified number and / or the importance weight tends to be stable.

8. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 can be implemented.

Images