Neural network optimization method and device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and storage medium for optimizing a neural network, the method comprising: obtaining a target neural network, the target neural network comprising a convolution part, the number of convolution channels of the convolution part being N; obtaining a convolution channel retention factor K / N, wherein N is greater than K; performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of channels of the convolution part is K; training the pruned neural network with pre-prepared training data to obtain a trained neural network; processing the training neural network with a genetic algorithm, and selecting a final neural network based on the processing result of the genetic algorithm. The convolution channel of the target neural network is pruned by the channel retention factor, thereby improving the pruning efficiency of the convolution channel and thus improving the optimization efficiency of the neural network.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a neural network optimization method, device, electronic device and storage medium. Background Art

[0002] When classifying images, using convolutional neural networks for training is more robust than traditional feature extraction, and the extracted features are more significant. In a convolutional neural network, the corresponding image channel features are extracted through convolution kernels (also called weight parameters) to obtain feature channels with the same number of convolution kernels. That is, before training the convolutional neural network, the initial number of channels is set by the staff, that is, it is a fixed value. Since the staff cannot know the optimal number of channels, there will be a certain margin in this fixed value, and this margin will cause redundancy in the number of channels, so that the entire convolutional network requires more computing resources, the calculation time is longer, and it is more prone to overfitting. In the face of this problem, the staff can optimize the number of channels of the neural network in one training and experiment, and cut off the redundant number of channels, but this cutting method requires the staff to try again and again, and there is a problem of low cutting efficiency, which makes the optimization efficiency of the neural network low. Summary of the invention

[0003] An embodiment of the present invention provides a neural network optimization method, which can improve the efficiency of pruning redundant channels in a convolutional neural network, thereby improving the optimization efficiency of the convolutional neural network.

[0004] In a first aspect, an embodiment of the present invention provides a method for optimizing a neural network, comprising:

[0005] Obtain a target neural network, wherein the target neural network includes a convolution part, the number of convolution channels of the convolution part is N, the target neural network is used for a classification task of image attributes, and the convolution part is used to extract classification features of image attributes;

[0006] Get the convolution channel retention factor K / N, where N is greater than K;

[0007] Performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of convolution channels of the convolution part is K;

[0008] Training the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels;

[0009] The training neural network is processed by a genetic algorithm, and a final neural network is selected based on the processing result of the genetic algorithm.

[0010] Optionally, the convolution channel retention factor K / N is a random value; or K is a random value and N is a fixed value.

[0011] Optionally, performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network includes:

[0012] Performing a linear transformation on the convolution channel retention factor K / N to obtain a convolution weight parameter of N dimensions;

[0013] The convolution weight parameters of the N dimensions are dimensionally cropped according to preset rules to obtain convolution weight parameters of K dimensions, and the convolution weight parameters of the K dimensions are passed to the convolution part as the weight parameters of the convolution part, so that the number of convolution channels of the convolution part is K.

[0014] Optionally, the convolution part includes a separate convolution part and a point-by-point convolution part, the number of convolution channels of the separate convolution part is N1, the number of convolution channels of the point-by-point convolution part is N2, and the obtaining of the convolution channel retention factor K / N includes:

[0015] Obtaining a separation convolution channel retention factor K1 / N1 and a point-by-point convolution channel retention factor K2 / N2, wherein N1 is greater than K1, and N2 is greater than K2;

[0016] The step of performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruning neural network includes:

[0017] According to the separate convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2, the separate convolution part and the point-by-point convolution part are respectively pruned to obtain a pruned neural network, in which the number of convolution channels of the separate convolution part is K1, and the number of convolution channels of the point-by-point convolution part is K2.

[0018] Optionally, during multiple training runs:

[0019] The separation convolution channel retention factor K1 / N1 is any value in an arithmetic progression with D1 as the tolerance and S1 as the initial value; or K1 is any value in an arithmetic progression with D2 as the tolerance and S2 as the initial value, and N1 is a fixed value;

[0020] The point-by-point convolution channel retention factor K2 / N2 is any value in an arithmetic progression with D3 as the tolerance and S3 as the initial value; or K2 is any value in an arithmetic progression with D4 as the tolerance and S4 as the initial value, and N2 is a constant.

[0021] Optionally, the performing channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network includes:

[0022] Performing a linear transformation on the separation convolution channel retention factor K1 / N1 through a first transformation thread to obtain a separation convolution weight parameter of N1 quantity dimension;

[0023] The separation convolution weight parameters of the N1 number of dimensions are dimensionally cropped according to a preset rule to obtain separation convolution weight parameters of the K1 number of dimensions, and the separation convolution weight parameters of the K1 number of dimensions are passed to the convolution part as the weight parameters of the separation convolution part, so that the number of convolution channels of the separation convolution part is K1;

[0024] Performing a linear transformation on the point-by-point convolution channel retention factor K2 / N2 through a second transformation thread to obtain a point-by-point convolution weight parameter of N2 number dimensions;

[0025] The point-by-point convolution weight parameters of N2 dimensions are dimensionally cropped according to preset rules to obtain point-by-point convolution weight parameters of K2 dimensions, and the point-by-point convolution weight parameters of K2 dimensions are passed to the convolution part as weight parameters of the point-by-point convolution part, so that the number of convolution channels of the point-by-point convolution part is K2.

[0026] Optionally, the processing the training neural network by a genetic algorithm and selecting a final neural network based on the processing result of the genetic algorithm includes:

[0027] The training neural networks corresponding to different retention factors are processed by genetic algorithms to obtain neural network individuals and individual evaluation values ​​corresponding to the training neural networks corresponding to different convolution channel retention factors;

[0028] The best neural network individual is determined according to the individual evaluation value, and the best neural network individual is selected as the final neural network.

[0029] In a second aspect, an embodiment of the present invention provides a neural network optimization device, characterized in that the device comprises:

[0030] A first acquisition module is used to acquire a target neural network, wherein the target neural network includes a convolution part, the number of channels of the convolution part is N, the target neural network is used for the classification task of image attributes, and the convolution part is used to extract classification features of image attributes;

[0031] The second acquisition module is used to obtain a convolution channel retention factor K / N, where N is greater than K;

[0032] A cropping module, used for performing convolution channel cropping on the convolution part according to the convolution channel retention factor K / N to obtain a cropped neural network, in which the number of convolution channels of the convolution part is K;

[0033] A training module, used to train the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels;

[0034] The selection module is used to process the training neural network through a genetic algorithm and select a final neural network based on the processing result of the genetic algorithm.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the steps in the neural network optimization method as described in any one of the embodiments of the present invention.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the neural network optimization method as described in any one of the embodiments of the present invention are implemented.

[0037] In an embodiment of the present invention, a target neural network is obtained, the target neural network includes a convolution part, the number of convolution channels of the convolution part is N, the target neural network is used for the classification task of image attributes, and the convolution part is used to extract the classification features of image attributes; a convolution channel retention factor K / N is obtained, wherein N is greater than K; the convolution channel of the convolution part is pruned according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of channels of the convolution part is K; the pruned neural network is trained by pre-prepared training data to obtain a training neural network, the training data includes sample images annotated with image attribute labels; the training neural network is processed by a genetic algorithm, and a final neural network is selected based on the processing result of the genetic algorithm. By performing convolution channel pruning on the convolution part of the target neural network by the channel retention factor, pruned neural networks corresponding to different channel retention factors can be obtained, and after the pruned neural network is trained, the final neural network is selected by a genetic algorithm, thereby improving the pruned efficiency of the convolution channel, and thus improving the optimization efficiency of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a neural network optimization method provided by an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a convolution channel clipping method provided by an embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of a neural network clipping provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of cutting the number of convolution channels provided by an embodiment of the present invention;

[0043] Figure 5 is a flow chart of another neural network optimization method provided by an embodiment of the present invention;

[0044] Figure 6 It is a cropped schematic diagram of a depth-separable convolutional network provided by an embodiment of the present invention;

[0045] Figure 7 It is a structural schematic diagram of a neural network optimization device provided by an embodiment of the present invention;

[0046] Figure 8 is a schematic structural diagram of another neural network optimization device provided by an embodiment of the present invention;

[0047] Fig. 9 is a schematic structural diagram of another neural network optimization device provided by an embodiment of the present invention;

[0048] Fig.10 It is a schematic diagram of the structure of another neural network optimization device provided in an embodiment of the present invention.

[0049] Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1 , Figure 1 is a flowchart of a neural network optimization method provided by an embodiment of the present invention, such as Figure 1 As shown, the following steps are included:

[0052] 101. Obtain a target neural network.

[0053] In this step, the target neural network includes a convolution part, the number of channels of the convolution part is N, the target neural network is used for the classification task of image attributes, and the convolution part is used to extract classification features of image attributes.

[0054] The target neural network may be any convolutional neural network for image attribute classification. The convolutional neural network may perform convolution calculations on the image to extract attribute features of the image for recognition.

[0055] The number of channels of the above-mentioned convolution part is pre-set. In the convolutional neural network, the above-mentioned number of channels can be maintained by the convolution channel (called weight parameter or convolution kernel). It can be understood that a convolution channel traverses and convolves the image once, and a feature channel is obtained accordingly. For example, assuming that the input image is a three-channel RGB, when the convolution layer performs convolution calculation, the convolution channel of the convolution layer is 1*1*3*8, where 1*1 is the size of the convolution kernel, 3 is the dimension of the convolution channel, and the dimension of the convolution channel needs to be the same as the number of channels of the input image, so it is the same as the number of RGB three channels, and 8 is the number of convolution kernels, indicating that the input image is traversed and convolved 8 times, and 8 feature channels will be formed after convolution. The number of channels of the above-mentioned convolution part can be pre-set to a fixed number by the staff, that is, it can be understood that the number of convolution channels of the above-mentioned convolution part is a pre-set fixed number. For example, it can be set to 10, 50, 100, etc. It should be noted that, in the embodiment of the present invention, the number of convolution channels pre-set for the target neural network is not limited.

[0056] The above-mentioned target neural network can be a convolutional neural network built by the staff themselves, or it can be an open source convolutional neural network obtained from an open source website. It can be a trained convolutional neural network or an untrained convolutional neural network.

[0057] Furthermore, the above-mentioned target neural network can be a convolutional neural network for image classification tasks, such as a convolutional neural network for classification tasks such as personnel attribute classification, vehicle attribute classification, and object classification.

[0058] The above-mentioned personnel attribute classification may include: classification of attributes such as clothes, hats, glasses, etc., the above-mentioned vehicle attribute classification may include: classification of attributes such as brand, color, body length, etc., and the above-mentioned object classification may include: classification of attributes such as object color, object size, etc.

[0059] Furthermore, the target neural network can be a deep separable convolutional neural network. The deep separable convolutional neural network is different from the traditional multi-channel convolutional neural network. The deep separable convolutional neural network separates the depth of the convolution calculation, so that the overall calculation amount is reduced.

[0060] The above-mentioned deep separable convolutional neural network may include at least one separation convolution part and at least one point-by-point convolution part. The above-mentioned point-by-point convolution is followed by a separation convolution part to form a complete feature extraction unit. In the deep separable convolutional neural network, there may be multiple serially connected feature extraction units to improve the feature extraction capability.

[0061] The above-mentioned separation convolution part can perform channel-by-channel convolution on the image input to the separation convolution part to obtain the same feature channel as the input image channel. The above-mentioned point-by-point convolution part can perform convolution calculation on the feature channel in the channel dimension.

[0062] For example, assuming that an image has three channels of RGB, the convolution kernel in the multi-channel convolutional neural network is 3*3*3*8, the spatial size of the convolution kernel is 3*3, the channel dimension (channel depth) is 3, and the number of convolution kernels is 8, then the total amount of calculation is 3*3*3*8=216. Similarly, in the depthwise separable convolutional neural network, three RGB channels are convolved by three 3*3 spatial convolution kernels to obtain three feature channels. These three 3*3 spatial convolution kernels can be assumed to have a channel dimension of 1 by default, that is, the tensor expression of each convolution kernel is 3*3*1. Then, eight 1*1*3 point-by-point convolution kernels are used to convolve the three feature channels to obtain eight feature channels, of which the 1*1*3 convolution can be understood as a 1*1 convolution kernel with a channel dimension of 3. The total amount of calculation is 3*3*3+1*1*3*8=51.

[0063] 102. Get the convolution channel retention factor K / N.

[0064] Wherein, N is greater than K. N is a fixed number pre-set by the staff in the above step 1.

[0065] In an embodiment of the present invention, the above-mentioned convolution channel retention factor K / N is less than 1 and greater than 0, and can be a random value less than 1 and greater than 0.

[0066] In one acquisition method, when the target neural network needs to be pruned, the convolution channel retention factor can be obtained by random number generation. This method can be understood as automatically generating a random value between 0 and 1, and the user does not need to output the convolution channel retention factor.

[0067] In another acquisition method, the user can enter the N value, and the corresponding convolution channel retention factor will be automatically generated according to the preset rules. For example, when the user enters an N value of 10, the preset rule is one decimal place, and the convolution channel retention factor can be automatically selected in the discrete interval [0.1, 0.2, 0.3, 04, 0.5, 0.6, 0.7, 0.8, 0.9]. When the user enters an N value of 100, the preset rule is two decimal places, and the convolution channel retention factor can be automatically selected in the discrete interval [0.01, 0.02, ..., 0.99]. Of course, the user can also directly enter the value of the convolution channel retention factor K / N.

[0068] In another acquisition method, the user can input the N value, or the system can automatically default or automatically detect the N value. After obtaining the N value information, it randomly generates a K value from 0 to N or the user inputs the corresponding K value, and calculates K / N to obtain the corresponding convolution channel retention factor.

[0069] It should be noted that the above-mentioned convolution channel retention factor can also be called convolution channel retention ratio.

[0070] When the target neural network includes multiple convolution parts, each convolution part obtains the corresponding convolution channel retention factor, and the convolution channel retention factors obtained by different convolution parts can be the same or different. In this way, the convolution channels of multiple convolution parts can be trimmed at the same time, further improving the optimization efficiency of the neural network. It should be noted that the above-mentioned convolution channel can also be called a convolution kernel.

[0071] Furthermore, the target neural network can be a deep separable convolutional neural network. In this case, the convolution part in the target neural network can include a separable convolution part and a point-by-point convolution part. Each separable convolution part is connected in series with a point-by-point convolution part. The separable convolution part performs spatial convolution calculations on the height and width of a single channel of the image, and the point-by-point convolution part performs depth convolution calculations on multiple channels of the image. While reducing the amount of calculation, the spatial information is combined with the depth information to avoid information loss.

[0072] The number of channels of the above separation convolution part is N1, which is a fixed number (constant value) preset in the target neural network. The number of channels of the above point-by-point convolution part is N2, which is also a fixed number (constant value) preset in the target neural network.

[0073] Get the separation convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2.

[0074] The above-mentioned separation convolution part obtains the separation convolution channel retention factor K1 / N1 to crop the convolution channel of the separation convolution part to obtain the separation convolution part of the remaining K1 convolution channels; the above-mentioned point-by-point convolution part obtains the point-by-point convolution channel retention factor K2 / N2 to crop the convolution channel of the point-by-point convolution part to obtain the separation convolution part of the remaining K2 convolution channels. The above-mentioned N1 is greater than the above-mentioned K1, and the above-mentioned N2 is greater than the above-mentioned K2

[0075] The dimension of the point-by-point convolution channels is the same as the number of the separation convolution channels, and the number of the separation convolution channels is the same as the number of channels of the input image.

[0076] For example, suppose the input image is a tensor of H*W*F0, where H and W represent the height and width of the input image respectively, and F0 represents the number of channels of the input image (also called depth). When performing feature extraction through a deep separable convolutional neural network, it will first be input into the separation convolution part. At this time, the number of separation convolution channels needs to be F0, and the dimension (depth) of each separation convolution channel is 1. In traditional multi-channel convolution, the dimension of each convolution channel needs to be F0. Assuming that the convolution channel is a tensor of 3*3*1, after the convolution calculation, F0 feature channels H*W*1 are obtained as the input of the point-by-point convolution part, and its tensor form is H*W*F0. At this time, in the point-by-point convolution part, the number of convolution channels of the point-by-point convolution is F1, and the dimension is F0. Then the tensor expression of the point-by-point convolution channel is 1*1*F0. After the point-by-point convolution part is calculated, the tensor expression of the output result is H*W*F1.

[0077] In an embodiment of the present invention, the above-mentioned separation convolution channel retention factor K1 / N1 and the above-mentioned point-by-point convolution channel retention factor K2 / N2 are both less than 1 and greater than 0. Further, they can be random values ​​less than 1 and greater than 0.

[0078] In one acquisition method, when the separation convolution part and / or the point-by-point convolution part need to be cropped, the separation convolution channel retention factor and the point-by-point convolution channel retention factor can be obtained by a random number generation method. This method can be understood as automatically generating a random value between 0 and 1, and the user does not need to output the convolution channel retention factor.

[0079] In another acquisition method, the user can enter the N1 value and / or the N2 value, and the corresponding separation convolution channel retention factor and / or point-by-point convolution channel retention factor will be automatically generated according to the preset rules. For example, when the N1 value entered by the user is 10, the preset rule is one decimal place, and the separation convolution channel retention factor can be automatically selected in the discrete interval [0.1, 0.2, 0.3, 04, 0.5, 0.6, 0.7, 0.8, 0.9]. When the N2 value entered by the user is 100, the preset rule is two decimal places, and the convolution channel retention factor can be automatically selected in the discrete interval [0.01, 0.02, ..., 0.99]. Of course, the user can also directly enter the value of the separation convolution channel retention factor K1 / N1, and / or the value of the point-by-point convolution channel retention factor K2 / N2.

[0080] In another acquisition method, the user can input the N1 value and / or N2 value, or the system automatically defaults to or automatically detects the N1 value and / or N2 value. After obtaining the N1 value and / or N2 value information, it randomly generates a K1 value from 0 to N1, and / or randomly generates a K2 value from 0 to N2, or the user inputs the corresponding K1 value and / or K2 value, and calculates K1 / N1 and / or K2 / N2 to obtain the corresponding separation convolution channel retention factor and / or point-by-point convolution channel retention factor.

[0081] In another acquisition method, since the initial input image can be preprocessed to form a fixed number of channels, for example, when the input image is an RGB three-channel image, the number of channels of the input image at this time is 3, then the number of channels of the first separation convolution part can be 3; after the first separation convolution part, a point-by-point convolution part will be connected in series to extract depth information. Assuming that the number of point-by-point convolution channels of the point-by-point convolution part is K2, the output result of the point-by-point convolution part is K2 channels; at this time, the number of separation convolution channels of the second separation convolution part also needs to be cut to K2. In this way, in the deep separable convolutional neural network, only the point-by-point convolution channel retention factor K2 / N2 can be obtained to obtain the separation convolution channel retention factor K1 / N1. In this way, the amount of parameter generation can be reduced.

[0082] 103. Perform convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network.

[0083] In the above pruned neural network, the number of convolution channels in the convolution part is pruned to K.

[0084] In step 103, in the convolution channel retention factor, N is a fixed value set in advance by the target neural network, and K is the number to be retained. Therefore, the number of pruned convolution channels is N-K.

[0085] The specific convolution channel cropping is as follows Figure 2 As shown, including:

[0086] 201. Perform a linear transformation on the convolution channel retention factor K / N to obtain a convolution weight parameter of N dimensions.

[0087] In this step, the above-mentioned linear transformation can be performed by connecting at least one fully connected layer to the convolution part, and the above-mentioned fully connected layer includes linearly changing weight parameters. The convolution channel retention factor can be dimensionally increased or reduced to obtain the corresponding convolution weight parameters.

[0088] Among them, the convolution channel retention factor K / N is a value, which can also be regarded as a one-dimensional 1*1 tensor. The weight parameters in the fully connected layer can be regarded as convolution channels. Assume that the convolution channel in the fully connected layer is 1*1*32, where 3*3 is the size information of the convolution channel and 32 is the number of convolution channels. After passing through the fully connected layer, the channel retention factor increases to 32 dimensions, and the tensor form is 1*1*32.

[0089] Since the number of convolution channels in the convolution part of the target neural network is N, the convolution channel retention factor K / N needs to be upgraded through linear transformation to obtain N dimensions. Assuming that the tensor of the convolution channel of the convolution part of the target neural network is 3*3*3*N, it means that the size of the convolution channel is 3*3, the depth is 3, and the number is N. Then the convolution channel retention factor is linearly transformed through the fully connected layer to obtain the convolution weight parameter 3*3*3*N as the output of the fully connected layer.

[0090] It should be noted that the number of fully connected layers can be set by the staff. In order to achieve better fitting effect, at least two fully connected layers can be set. Furthermore, activation layers can be set between fully connected layers to further improve the fitting effect of convolution weight parameters. The specific connection structure can be as follows Figure 3 As shown in Figure 1, the convolutional layer in the target neural network is connected to a fully connected layer.

[0091] 202. Perform dimension cropping on the convolution weight parameters of N dimensions according to a preset rule to obtain convolution weight parameters of K dimensions.

[0092] In step 201, a convolution weight parameter of N dimensions is obtained through a linear transformation of the fully connected layer. The convolution weight parameter is the same as the size of the convolution channel in the convolution part. For example, if the size of the convolution channel is 3*3*3, then the size of the convolution weight parameter is also 3*3*3.

[0093] The above preset rule can be to crop the weight parameters of N-K dimensions in N dimensions, to crop the first N-K, to crop the last N-K, or to randomly crop N-K, so as to obtain the convolution weight parameters of K dimensions. For example, if the tensor of the convolution weight parameter is 3*3*3*N, it is cropped to 3*3*3*K. Figure 4 As shown, Figure 4 In the example, 3*3*3*5 is cropped to 3*3*3*4.

[0094] 203. The convolution weight parameters of the K-number dimension are passed to the convolution part as the weight parameters of the convolution part, so that the number of convolution channels of the convolution part is K.

[0095] In this step, the convolution weight parameters of the K-dimensional dimension are the pruned convolution weight parameters, which correspond to K / N in the convolution channel retention factor.

[0096] In step 202, after the convolution channel retention factor has undergone linear transformation of the fully connected layer, the output of the fully connected layer can be used as the convolution channel of the convolution part in the target neural network. Therefore, the weight parameters output by the fully connected layer can be directly clipped, and then the clipped weight parameters are passed to the convolution part in the target neural network, and the original N convolution channels are replaced by the clipped weight parameters of K dimensions, so that the number of convolution channels of the clipped convolution part is K, thereby completing the clipping of the target neural network and obtaining a clipped neural network.

[0097] like Figure 5 As shown, the present invention also provides a convolution channel pruning of a deep separable convolutional neural network, comprising:

[0098] 301. Perform a linear transformation on the separation convolution channel retention factor K1 / N1 through the first transformation thread to obtain a separation convolution weight parameter of N1 quantity dimension.

[0099] 302. Perform dimension cropping on the separation convolution weight parameters of N1 dimensions according to a preset rule to obtain separation convolution weight parameters of K1 dimensions.

[0100] 303. The separation convolution weight parameter of the K1 quantity dimension is passed to the convolution part as the weight parameter of the separation convolution part, so that the number of convolution channels of the separation convolution part is K1.

[0101] 304. Perform a linear transformation on the point-by-point convolution channel retention factor K2 / N2 through a second transformation thread to obtain a point-by-point convolution weight parameter of N2 dimension.

[0102] 305. The point-by-point convolution weight parameters of N2 dimensions are dimensionally cropped according to a preset rule to obtain point-by-point convolution weight parameters of K2 dimensions.

[0103] 306. The point-by-point convolution weight parameter of the K2 quantity dimension is passed to the convolution part as the weight parameter of the point-by-point convolution part, so that the number of convolution channels of the point-by-point convolution part is K2.

[0104] Among them, the above steps 301-303 are for cutting the separation convolution part, and the above steps 304-306 are for cutting the point-by-point convolution part. The cutting method is the same as Figure 2 The embodiments are similar and will not be described in detail here.

[0105] It should be noted that both the first transformation thread and the second transformation thread can be linearly transformed through the fully connected layer, and the number of fully connected layers in the first transformation thread can be the same as or different from the number of fully connected layers in the second transformation thread. In addition, the first transformation thread performs dimensionality increase in terms of the number dimension and the spatial size, that is, the 1*1*1*1 separation convolution channel retention factor is linearly transformed into a tensor form of 3*3*1*N1; and the second transformation thread performs dimensionality increase in terms of the number dimension and the depth dimension, that is, the 1*1*1*1 point-by-point convolution channel retention factor is linearly transformed into a tensor form of 1*1*N1*N2.

[0106] In a deep separable convolutional neural network, the specific connection structure can be as follows Figure 6 As shown, the separation convolution layer and the point-by-point convolution layer are respectively connected to the fully connected layer. Of course, in the first transformation thread and the second transformation thread, when there are multiple fully connected layers, activation layers can also be set between the fully connected layers to improve the fitting effect of the weight parameters.

[0107] 104. The pruned neural network is trained using pre-prepared training data to obtain a trained neural network.

[0108] The above-prepared training data can be prepared according to the specific task of the target neural network. The above-mentioned training data can include sample images annotated with image attribute labels. For example, when the task of the target neural network is personnel attribute classification, the training data is personnel attribute classification images and corresponding labels. When the task of the target neural network is vehicle attribute classification, the training data is vehicle attribute classification images and corresponding labels. When the task of the target neural network is object classification, the training data is equal classification object images and corresponding labels. The above-mentioned labels can be automatically annotated by an automatic annotation system or manually annotated.

[0109] During the training process, the training data and labels are traversed, and the loss value is calculated. Back propagation is performed based on the loss value, and the convolution channel values ​​(weight values) of each convolution part are continuously updated. Since the fully connected output is directly passed to the convolution part, the entire process is differentiable, so the updated gradient error can directly act on the convolution weight parameters obtained in the linear transformation, thereby changing the convolution weight parameters obtained in the linear transformation, and then changing the convolution channel values ​​of the convolution part in the target neural network.

[0110] During multiple training processes, multiple separation convolution channel retention factors and multiple point-wise convolution channel retention factors can be obtained.

[0111] Specifically, the above-mentioned separation convolution channel retention factor K1 / N1 can be any value in an arithmetic progression with D1 as the common difference and S1 as the first term; or the above-mentioned K1 can also be any value in an arithmetic progression with D2 as the common difference and S2 as the initial value, and N1 is a constant value;

[0112] Similarly, the above-mentioned point-by-point convolution channel retention factor K2 / N2 can be any value in an arithmetic progression with D3 as the tolerance and S3 as the initial value; or the above-mentioned K2 can be any value in an arithmetic progression with D4 as the tolerance and S4 as the initial value, and N2 is a constant.

[0113] 105. The training neural network is processed by a genetic algorithm, and a final neural network is selected based on the processing result of the genetic algorithm.

[0114] After step 104, each time the training data is traversed, the convolution channel value (weight value) of the convolution part in the target neural network can be saved once.

[0115] The above-mentioned genetic algorithm is a global optimization search algorithm, which reorganizes (hybridizes) the convolution part of the trained neural network through the genetic algorithm to obtain new neural network individuals with different convolution channel values ​​(weight values), and searches for the optimal individual among all neural network individuals.

[0116] Specifically, the training neural networks corresponding to different retention factors are processed by a genetic algorithm to obtain neural network individuals and individual evaluation values ​​corresponding to the training neural networks corresponding to different convolution channel retention factors; the best neural network individual is determined according to the individual evaluation value, and the best neural network individual is selected as the final neural network.

[0117] In an embodiment of the present invention, a target neural network is obtained, the target neural network includes a convolution part, the number of convolution channels of the convolution part is N; a convolution channel retention factor K / N is obtained, wherein N is greater than K; the convolution channel of the convolution part is pruned according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of channels of the convolution part is K; the pruned neural network is trained with pre-prepared training data to obtain a trained neural network; the trained neural network is processed with a genetic algorithm, and a final neural network is selected based on the processing result of the genetic algorithm. By performing convolution channel pruning on the convolution part of the target neural network with the channel retention factor, pruned neural networks corresponding to different channel retention factors can be obtained, and after the pruned neural network is trained, the final neural network is selected with the genetic algorithm, thereby improving the pruned efficiency of the convolution channel and thus improving the optimization efficiency of the neural network.

[0118] In addition, the target neural network after pruning has significantly improved in speed and accuracy. During the experiment, taking the vehicle attribute classification task as an example, the neural network selected the deep separable convolutional neural network, and the control group was the unpruned deep separable convolutional neural network. A comparative experiment was conducted on the vehicle attribute data set. After 200 rounds of iterative training on the training set, it was tested on the test set.

[0119] This experiment creates a 1000-class vehicle attribute dataset, one of which represents a combination of vehicle brand, specific vehicle model and year, such as Toyota Camry 2012. The following results are obtained in Table 1:

[0120] speed Accuracy Uncropped 0.62ms 93.8％ After cropping 0.49ms 95.2％

[0121] Table 1

[0122] As can be seen from Table 1, after cutting out the redundant convolution channels (convolution kernels), the overfitting of the neural network can be reduced, so the accuracy on the test set will be improved. After cutting out the redundant convolution channels (convolution kernels), the redundant calculation amount is reduced, so the speed is improved.

[0123] It should be noted that the neural network optimization method provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that need to optimize the neural network.

[0124] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a neural network optimization device provided by an embodiment of the present invention, such as Figure 7 As shown, the device comprises:

[0125] A first acquisition module 701 is used to acquire a target neural network, wherein the target neural network includes a convolution part, the number of channels of the convolution part is N, the target neural network is used for the classification task of image attributes, and the convolution part is used to extract classification features of image attributes;

[0126] A second acquisition module 702 is used to acquire a convolution channel retention factor K / N, where N is greater than K;

[0127] A trimming module 703 is used to trim the convolution channel of the convolution part according to the convolution channel retention factor K / N to obtain a trimmed neural network, in which the number of convolution channels of the convolution part is K;

[0128] A training module 704 is used to train the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels;

[0129] The selection module 705 is used to process the training neural network through a genetic algorithm and select a final neural network based on the processing result of the genetic algorithm.

[0130] Optionally, the convolution channel retention factor K / N is a random value; or K is a random value and N is a fixed value.

[0131] Optional, such as Figure 8 As shown, the cutting module 703 includes:

[0132] The transformation submodule 7031 is used to perform a linear transformation on the convolution channel retention factor K / N to obtain a convolution weight parameter of N dimensions;

[0133] The cropping submodule 7032 is used to perform dimension cropping on the convolution weight parameters of the N dimensions according to preset rules to obtain convolution weight parameters of K dimensions, and pass the convolution weight parameters of the K dimensions to the convolution part as the weight parameters of the convolution part, so that the number of convolution channels of the convolution part is K.

[0134] Optionally, the convolution part includes a separate convolution part and a point-by-point convolution part, the number of convolution channels of the separate convolution part is N1, the number of convolution channels of the point-by-point convolution part is N2, and the second acquisition module 702 is further used to obtain a separate convolution channel retention factor K1 / N1 and a point-by-point convolution channel retention factor K2 / N2, wherein N1 is greater than K1, and N2 is greater than K2;

[0135] The cropping module 703 is also used to perform convolution channel cropping on the separate convolution part and the point-by-point convolution part according to the separate convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2, respectively, to obtain a cropped neural network, in which the number of convolution channels of the separate convolution part is K1, and the number of convolution channels of the point-by-point convolution part is K2.

[0136] Optionally, during multiple training runs:

[0137] The separation convolution channel retention factor K1 / N1 is any value in an arithmetic progression with D1 as the tolerance and S1 as the initial value; or K1 is any value in an arithmetic progression with D2 as the tolerance and S2 as the initial value, and N1 is a fixed value;

[0138] The point-by-point convolution channel retention factor K2 / N2 is any value in an arithmetic progression with D3 as the tolerance and S3 as the initial value; or K2 is any value in an arithmetic progression with D4 as the tolerance and S4 as the initial value, and N2 is a constant.

[0139] Optional, such as Fig. 9 As shown, the cutting module 703 includes:

[0140] A first transformation submodule 7033 is used to perform a linear transformation on the separation convolution channel retention factor K1 / N1 through a first transformation thread to obtain a separation convolution weight parameter of N1 quantity dimension;

[0141] The first clipping submodule 7034 is used to perform dimension clipping on the separation convolution weight parameters of the N1 number of dimensions according to a preset rule to obtain separation convolution weight parameters of the K1 number of dimensions, and pass the separation convolution weight parameters of the K1 number of dimensions to the convolution part as the weight parameters of the separation convolution part, so that the number of convolution channels of the separation convolution part is K1;

[0142] The second transformation submodule 7035 is used to perform a linear transformation on the point-by-point convolution channel retention factor K2 / N2 through a second transformation thread to obtain a point-by-point convolution weight parameter of N2 number dimension;

[0143] The second cropping submodule 7036 is used to perform dimension cropping on the point-by-point convolution weight parameters of N2 dimensions according to preset rules to obtain point-by-point convolution weight parameters of K2 dimensions, and pass the point-by-point convolution weight parameters of K2 dimensions to the convolution part as the weight parameters of the point-by-point convolution part, so that the number of convolution channels of the point-by-point convolution part is K2.

[0144] Optional, such as Fig.10 As shown, the selection module 705 includes:

[0145] Genetic processing submodule 7051, used to process the training neural networks corresponding to different retention factors through genetic algorithms to obtain neural network individuals and individual evaluation values ​​corresponding to the training neural networks corresponding to different convolution channel retention factors;

[0146] The individual selection submodule 7052 is used to determine the best neural network individual according to the individual evaluation value, and select the best neural network individual as the final neural network.

[0147] It should be noted that the neural network optimization device provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that need to optimize the neural network.

[0148] The neural network optimization device provided in the embodiment of the present invention can implement each process implemented by the neural network optimization method in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0149] See also Fig.11 , Fig.11 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig.11 As shown, it includes: a memory 1102, a processor 1101, and a computer program stored in the memory 1102 and executable on the processor 1101, wherein:

[0150] The processor 1101 is used to call the computer program stored in the memory 1102 and execute the following steps:

[0151] Obtain a target neural network, wherein the target neural network includes a convolution part, the number of convolution channels of the convolution part is N, the target neural network is used for a classification task of image attributes, and the convolution part is used to extract classification features of image attributes;

[0152] Get the convolution channel retention factor K / N, where N is greater than K;

[0153] Performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of convolution channels of the convolution part is K;

[0154] Training the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels;

[0155] The training neural network is processed by a genetic algorithm, and a final neural network is selected based on the processing result of the genetic algorithm.

[0156] Optionally, the convolution channel retention factor K / N is a random value; or K is a random value and N is a fixed value.

[0157] Optionally, the processor 1101 performs convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, including:

[0158] Performing a linear transformation on the convolution channel retention factor K / N to obtain a convolution weight parameter of N dimensions;

[0159] The convolution weight parameters of the N dimensions are dimensionally cropped according to preset rules to obtain convolution weight parameters of the K dimensions, and the convolution weight parameters of the K dimensions are passed to the convolution part as the weight parameters of the convolution part, so that the number of convolution channels of the convolution part is K.

[0160] Optionally, the convolution part includes a separate convolution part and a point-by-point convolution part, the number of convolution channels of the separate convolution part is N1, the number of convolution channels of the point-by-point convolution part is N2, and the obtaining of the convolution channel retention factor K / N performed by the processor 1101 includes:

[0161] Obtaining a separation convolution channel retention factor K1 / N1 and a point-by-point convolution channel retention factor K2 / N2, wherein N1 is greater than K1, and N2 is greater than K2;

[0162] The step of performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruning neural network includes:

[0163] According to the separate convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2, convolution channels are pruned on the separate convolution part and the point-by-point convolution part respectively to obtain a pruned neural network. In the pruned neural network, the number of convolution channels of the separate convolution part is K1, and the number of convolution channels of the point-by-point convolution part is K2.

[0164] Optionally, during multiple training runs:

[0165] The separation convolution channel retention factor K1 / N1 is any value in an arithmetic progression with D1 as the tolerance and S1 as the initial value; or K1 is any value in an arithmetic progression with D2 as the tolerance and S2 as the initial value, and N1 is a fixed value;

[0166] The point-by-point convolution channel retention factor K2 / N2 is any value in an arithmetic progression with D3 as the tolerance and S3 as the initial value; or K2 is any value in an arithmetic progression with D4 as the tolerance and S4 as the initial value, and N2 is a constant.

[0167] Optionally, the processor 1101 performs convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, including:

[0168] Performing a linear transformation on the separation convolution channel retention factor K1 / N1 through a first transformation thread to obtain a separation convolution weight parameter of N1 quantity dimension;

[0169] The separation convolution weight parameters of the N1 number of dimensions are dimensionally cropped according to a preset rule to obtain separation convolution weight parameters of the K1 number of dimensions, and the separation convolution weight parameters of the K1 number of dimensions are passed to the convolution part as the weight parameters of the separation convolution part, so that the number of convolution channels of the separation convolution part is K1;

[0170] Performing a linear transformation on the point-by-point convolution channel retention factor K2 / N2 through a second transformation thread to obtain a point-by-point convolution weight parameter of N2 number dimensions;

[0171] The point-by-point convolution weight parameters of N2 dimensions are dimensionally cropped according to preset rules to obtain point-by-point convolution weight parameters of K2 dimensions, and the point-by-point convolution weight parameters of K2 dimensions are passed to the convolution part as weight parameters of the point-by-point convolution part, so that the number of convolution channels of the point-by-point convolution part is K2.

[0172] Optionally, the processor 1101 processes the training neural network by using a genetic algorithm and selects a final neural network based on a processing result of the genetic algorithm, including:

[0173] The training neural networks corresponding to different retention factors are processed by genetic algorithms to obtain neural network individuals and individual evaluation values ​​corresponding to the training neural networks corresponding to different convolution channel retention factors;

[0174] The best neural network individual is determined according to the individual evaluation value, and the best neural network individual is selected as the final neural network.

[0175] It should be noted that the above-mentioned electronic devices can be applied to mobile phones, monitors, computers, servers and other devices that need to optimize neural networks.

[0176] The electronic device provided in the embodiment of the present invention can implement each process implemented by the optimization method of the neural network in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0177] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the neural network optimization method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0179] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for optimizing a neural network, characterized in that: The following steps are involved: Obtain a target neural network, wherein the target neural network includes a convolution part, the number of convolution channels of the convolution part is N, the target neural network is used for a classification task of image attributes, and the convolution part is used to extract classification features of image attributes; Get the convolution channel retention factor K / N, where N is greater than K; Performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruned neural network, in which the number of convolution channels of the convolution part is K; Training the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels; Processing the training neural network by a genetic algorithm, and selecting a final neural network based on the processing result of the genetic algorithm; The convolution part includes a separation convolution part and a point-by-point convolution part, the number of convolution channels of the separation convolution part is N1, the number of convolution channels of the point-by-point convolution part is N2, and the acquisition of the convolution channel retention factor K / N includes: Obtaining a separation convolution channel retention factor K1 / N1 and a point-by-point convolution channel retention factor K2 / N2, wherein N1 is greater than K1, and N2 is greater than K2; The step of performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruning neural network includes: According to the separate convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2, convolution channels are pruned on the separate convolution part and the point-by-point convolution part respectively to obtain a pruned neural network. In the pruned neural network, the number of convolution channels of the separate convolution part is K1, and the number of convolution channels of the point-by-point convolution part is K2.

2. The method for optimizing a neural network according to claim 1, wherein: The convolution channel retention factor K / N is a random value; or K is a random value and N is a fixed value.

3. The method for optimizing a neural network according to claim 1, wherein: The step of performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruning neural network includes: Performing a linear transformation on the convolution channel retention factor K / N to obtain a convolution weight parameter of N dimensions; The convolution weight parameters of the N dimensions are dimensionally cropped according to preset rules to obtain convolution weight parameters of the K dimensions, and the convolution weight parameters of the K dimensions are passed to the convolution part as the weight parameters of the convolution part, so that the number of convolution channels of the convolution part is K.

4. The method for optimizing a neural network according to claim 1, wherein: During multiple training sessions: The separation convolution channel retention factor K1 / N1 is any value in an arithmetic progression with D1 as the tolerance and S1 as the initial value; or K1 is any value in an arithmetic progression with D2 as the tolerance and S2 as the initial value, and N1 is a fixed value; The point-by-point convolution channel retention factor K2 / N2 is any value in an arithmetic progression with D3 as the tolerance and S3 as the initial value; or K2 is any value in an arithmetic progression with D4 as the tolerance and S4 as the initial value, and N2 is a constant.

5. The method for optimizing a neural network according to claim 1, wherein: The step of performing convolution channel pruning on the convolution part according to the convolution channel retention factor K / N to obtain a pruning neural network includes: Performing a linear transformation on the separation convolution channel retention factor K1 / N1 through a first transformation thread to obtain a separation convolution weight parameter of N1 quantity dimension; The separation convolution weight parameters of the N1 number of dimensions are dimensionally cropped according to a preset rule to obtain separation convolution weight parameters of the K1 number of dimensions, and the separation convolution weight parameters of the K1 number of dimensions are passed to the convolution part as the weight parameters of the separation convolution part, so that the number of convolution channels of the separation convolution part is K1; Performing a linear transformation on the point-by-point convolution channel retention factor K2 / N2 through a second transformation thread to obtain a point-by-point convolution weight parameter of N2 number dimensions; The point-by-point convolution weight parameters of N2 dimensions are dimensionally cropped according to preset rules to obtain point-by-point convolution weight parameters of K2 dimensions, and the point-by-point convolution weight parameters of K2 dimensions are passed to the convolution part as weight parameters of the point-by-point convolution part, so that the number of convolution channels of the point-by-point convolution part is K2.

6. The method for optimizing a neural network according to claim 1, wherein: The step of processing the training neural network by a genetic algorithm and selecting a final neural network based on the processing result of the genetic algorithm comprises: The training neural networks corresponding to different retention factors are processed by genetic algorithms to obtain neural network individuals and individual evaluation values ​​corresponding to the training neural networks corresponding to different convolution channel retention factors; The best neural network individual is determined according to the individual evaluation value, and the best neural network individual is selected as the final neural network.

7. A neural network optimization device, characterized in that: The device comprises: A first acquisition module is used to acquire a target neural network, wherein the target neural network includes a convolution part, the number of channels of the convolution part is N, the target neural network is used for the classification task of image attributes, and the convolution part is used to extract classification features of image attributes; The second acquisition module is used to obtain a convolution channel retention factor K / N, where N is greater than K; A cropping module, used for performing convolution channel cropping on the convolution part according to the convolution channel retention factor K / N to obtain a cropped neural network, in which the number of convolution channels of the convolution part is K; A training module, used to train the cropped neural network using pre-prepared training data to obtain a training neural network, wherein the training data includes sample images annotated with image attribute labels; A selection module, used for processing the training neural network through a genetic algorithm, and selecting a final neural network based on the processing result of the genetic algorithm; The convolution part includes a separation convolution part and a point-by-point convolution part, the number of convolution channels of the separation convolution part is N1, the number of convolution channels of the point-by-point convolution part is N2, and the second acquisition module is further used to acquire a separation convolution channel retention factor K1 / N1 and a point-by-point convolution channel retention factor K2 / N2, wherein N1 is greater than K1, and N2 is greater than K2; The cutting module is further used for: According to the separate convolution channel retention factor K1 / N1 and the point-by-point convolution channel retention factor K2 / N2, convolution channels are pruned on the separate convolution part and the point-by-point convolution part respectively to obtain a pruned neural network. In the pruned neural network, the number of convolution channels of the separate convolution part is K1, and the number of convolution channels of the point-by-point convolution part is K2.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the method for optimizing a neural network according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the neural network optimization method according to any one of claims 1 to 6.

Images