Network model compression method and device

By calculating the vector distance between filters in the network model, determining the center and non-center filters, and pruning and updating the parameters, the problem of network model compression in the prior art is solved, and the efficiency of the compressed network model is achieved is achieved. image recognition accuracy.

CN112580802BActive Publication Date: 2024-11-08TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011455193.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-10
Publication Date
2024-11-08
Estimated Expiration
2040-12-10

AI Technical Summary

Technical Problem

The existing network model compression method will weaken the model effect and reduce the accuracy of image recognition, making it difficult to maintain the accuracy of the model while compressing the network model.

Method used

By calculating the vector distance between any two filters in the network model, determining the central filter and the non-center filter, and pruning the non-center filter, obtaining the compressed network model composed of the central filter, and updating the filter Parameters to reduce the vector distance between the non-center filter and the central filter, and iteratively train to optimize the filter parameters.

Benefits of technology

When compressing the network model, the accuracy of the model is maintained, redundant parameters are removed, and the model size is significantly compressed without reducing the accuracy of image recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112580802B_ABST
    Figure CN112580802B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a network model compression method and device. The method includes: obtaining a network model to be compressed, wherein the network model includes at least two filters, and the filters are used to extract features from image data input to the network model; based on the filter parameters in each filter, calculating the vector distance between any two filters; determining the central filter and the non-central filter in the network model through the vector distance between any two filters, wherein the image features extracted by the central filter include part of the image features extracted by the non-central filter; pruning the non-central filter to obtain a compressed network model composed of the central filter. The technical solution of the embodiments of the present application can maintain the accuracy of the model while compressing the network model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and neural network technology, and more specifically, to a network model compression method and device. Background Art

[0002] The current network model used for image recognition consumes a lot of computing power, memory and electricity. To solve this problem, the existing technology generally uses Kmeans clustering to determine and remove redundant filters to achieve compression of the network model. However, the existing method of compressing the network model will weaken the effect of the model and reduce the accuracy of the model in recognizing images. Based on this, how to maintain the accuracy of the model while compressing the network model is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a network model compression method, device, computer program product or computer program, computer-readable medium and electronic device, which can maintain the accuracy of the model to a certain extent at least when compressing the network model.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a network model compression method is provided, the method comprising: obtaining a network model to be compressed, the network model comprising at least three filters, the filters being used to extract features from image data input into the network model; calculating a vector distance between any two filters based on filter parameters in each filter; determining a central filter and a non-central filter in the network model through the vector distance between any two filters, the image features extracted by the central filter comprising part of the image features extracted by the non-central filter; and pruning the non-central filter to obtain a compressed network model formed by the central filter.

[0006] According to one aspect of an embodiment of the present application, a network model compression device is provided, the device comprising: an acquisition unit, used to acquire a network model to be compressed, the network model comprising at least three filters, the filters being used to extract features from data input into the network model; a calculation unit, used to calculate the distance between any two filters based on filter parameters in each filter; a determination unit, used to determine a central filter and a non-central filter in the network model through the distance between any two filters, the features extracted by the non-central filter being close to the features extracted by the central filter; and a pruning unit, used to prune the non-central filter to obtain a compressed network model.

[0007] In some embodiments of the present application, based on the aforementioned scheme, the determination unit is configured as: for each target filter among the at least three filters, summing the vector distances between the target filter and filters other than the target filter to obtain the target distance sum of the target filter; based on the target distance sum, determining at least one target filter as a central filter; and determining filters other than the at least one target filter as non-central filters.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the determination unit is configured to: determine the number of the central filters to be retained according to a predetermined filter pruning ratio; sort the target filters in descending order according to the sum of the target distances; and determine the retained number of target filters with the highest sorting as the central filter.

[0009] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: an updating unit, which is used to construct a filter set for each central filter before pruning the non-central filter; for each non-central filter, determine the target central filter closest to the non-central filter, and add the non-central filter to the filter set where the target central filter is located; update the filter parameters in the network model to reduce the vector distance between the non-central filter and the central filter in the filter set.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the update unit is configured to: iteratively train the network model based on image training data; in each training process, determine the set gradient of each filter set respectively to update the filter parameters of the filters in the filter set through the set gradient.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the update unit is configured to: for each filter set in each training process, obtain the filter gradient of each filter in the filter set; based on the filter gradient of each filter, determine the filter gradient mean, and the vector distance between the filter gradient mean and each filter gradient; based on the filter gradient mean, and the vector distance between the filter gradient mean and each filter gradient, determine the set gradient of the filter set.

[0012] In some embodiments of the present application, based on the aforementioned scheme, the pruning unit includes: a cropping unit used to crop the non-central filters in the network model to obtain the network model to be optimized; an extraction unit used to perform feature extraction on the same image data through the initial network model and the network model to be optimized, respectively, to obtain a first feature and a second feature, and the initial network model is a network model that has not been compressed; a calculation unit used to calculate the difference between the first feature and the second feature; an adjustment unit used to adjust the filter parameters of the filter in the network model to be optimized based on the difference to obtain the compressed network model.

[0013] In some embodiments of the present application, based on the aforementioned scheme, the computing unit is configured to: determine a first feature vector of the image data through the first feature; determine a second feature vector of the image data based on the second feature; calculate a vector distance between the first feature vector and the second feature vector to determine the vector distance as the difference between the first feature and the second feature.

[0014] In some embodiments of the present application, based on the aforementioned scheme, the device also includes a cyclic compression starting unit, which is used to adjust the filter parameters of the filter in the network model to be optimized based on the difference, and after obtaining the compressed network model, start using the compressed network model as a new network model, and return to execute the step of obtaining the network model to be compressed to perform cyclic compression on the initial network model until a compressed network model that meets the preset compression conditions is obtained.

[0015] According to one aspect of an embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the network model compression method described in the above embodiment.

[0016] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the network model compression method described in the above embodiment is implemented.

[0017] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the network model compression method as described in the above embodiments.

[0018] In the technical solutions provided in some embodiments of the present application, the central filter and non-central filter of the model are determined by the vector distance between any two filters in the network model. Since the vector distance can reflect the similarity between filters in feature extraction to a certain extent, by removing the non-central filter determined based on the vector distance, a compressed network model composed of central filters is obtained, which can maintain the accuracy of the model when the network model is compressed. For example, for a network model used for image recognition, after compression by the network model compression method proposed in the present application, the redundant parameters in the image network model can be removed, so that the model size is compressed to a great extent, but the accuracy of the model in image recognition will not be reduced.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown;

[0022] Figure 2 A flowchart of a network model compression method according to an embodiment of the present application is shown;

[0023] Figure 3 A detailed flow chart of determining the central filter and the non-central filter in the network model according to an embodiment of the present application is shown;

[0024] Figure 4 A detailed flow chart of determining at least one target filter as a center filter based on the target distance summation according to one embodiment of the present application is shown;

[0025] Figure 5 A schematic diagram showing a method of determining a central filter based on summing target distances of filters according to an embodiment of the present application is shown;

[0026] Figure 6 A flow chart of a method before pruning the non-central filter according to an embodiment of the present application is shown;

[0027] Figure 7A detailed flow chart of updating the filter parameters in the network model according to one embodiment of the present application is shown;

[0028] Figure 8 A detailed flow chart of determining the set gradient of the filter set according to one embodiment of the present application is shown;

[0029] Fig. 9 A schematic diagram of pruning the non-central filter according to an embodiment of the present application is shown;

[0030] Fig.10 A detailed flow chart of pruning the non-central filter according to an embodiment of the present application is shown;

[0031] Fig.11 A detailed flow chart of calculating the difference between the first feature and the second feature according to one embodiment of the present application is shown;

[0032] Fig.12 A schematic diagram showing a method of adjusting filter parameters of a filter in the network model to be optimized according to an embodiment of the present application is shown;

[0033] Fig.13 A schematic diagram showing cyclic compression of an initial network model according to an embodiment of the present application is shown;

[0034] Fig.14 A block diagram of a network model compression device according to an embodiment of the present application is shown;

[0035] Fig.15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0037] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.

[0041] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0042] like Figure 1 As shown, the system architecture may include terminal devices (such as Figure 1 The embodiment of the present invention is a schematic diagram of a mobile phone 101, a tablet computer 102, a portable computer 103, a desktop computer, etc., a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as a wired communication link, a wireless communication link, etc.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0044] In one embodiment of the present application, the network model compression method proposed in the present application can be executed by the server 105. Specifically, the user can use the terminal device to upload image data for compressing the network model to the server 105. After obtaining the network model to be compressed including at least two filters, the server 105 calculates the vector distance between any two filters based on the filter parameters in each filter, and determines the central filter and non-central filter in the network model through the arbitrary vector distance. Finally, the non-central filter is pruned to obtain a compressed network model composed of the central filter. It should be noted that the above-mentioned filter is used to extract features of the image data input to the network model, wherein the image features extracted by the determined central filter include some image features extracted by the non-central filter.

[0045] It should be noted that the network model compression method provided in the embodiment of the present application is generally executed by the server 105, and accordingly, the network model compression device is generally arranged in the server 105. However, in other embodiments of the present application, the terminal device may also have similar functions as the server, so as to execute the network model compression scheme provided in the embodiment of the present application.

[0046] The model compression method provided in the embodiment of the present application relates to deep learning in the field of artificial intelligence (AI, ArtificialIntellegence), and specifically to computer vision technology (Computer Vision, CV) in deep learning technology. Computer vision is a science that studies how to make machines "see". Further, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further do graphic processing to make computer processing more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, etc., and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0047] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0048] Figure 2 A flowchart of a network model compression method according to an embodiment of the present application is shown. The network model compression method can be executed by a device having a computing processing function, such as Figure 1 Refer to the server 105 shown in Figure 2 As shown, the network model compression method includes at least steps 210 to 270, which are described in detail as follows:

[0049] In step 210, a network model to be compressed is obtained, wherein the network model includes at least two filters, and the filters are used to extract features from image data input to the network model.

[0050] Specifically, the network model (such as a neural network model) is an important aspect of realizing computer vision technology. The component unit of the network model is the network layer, and the network layer can be divided into a convolution layer, an activation layer, a pooling layer, a mapping layer, and a batch normalization layer based on different functions. In the embodiment of the present application, in most of the network layers, at least two filters (convolution kernels) are included, and the function of the filter is to extract features from the image data. It should be noted that compressing the network model can be compressing the number of filters or the size of the filters. Here, the size of the filter refers to the width and height of the filter.

[0051] Continue to refer to Figure 2 In step 230, based on the filter parameters in each filter, the vector distance between any two filters is calculated.

[0052] In the present application, the filter parameters in the filter may be one-dimensional or two-dimensional, such as 3×3 filter parameters, or three-dimensional, such as 5×5×5 filter parameters.

[0053] It should be understood that the vector distance between filters can be regarded as two filters as two points in space, where the filter parameters are regarded as the coordinates of the filters and the vector distance between the filters is regarded as the distance between the two filter points in the space.

[0054] Continue to refer to Figure 2 In step 250, the central filter and the non-central filter in the network model are determined by the vector distance between any two filters, and the image features extracted by the central filter include part of the image features extracted by the non-central filter.

[0055] In one embodiment of the present application, the central filter and the non-central filter in the network model are determined by the vector distance between any two filters, which can be as follows: Figure 3 Follow the steps shown.

[0056] join Figure 3 , shows a detailed flow chart of determining the central filter and the non-central filter in the network model according to an embodiment of the present application. Specifically comprising steps 251 to 253:

[0057] In step 251, for each target filter of the at least two filters, vector distances between the target filter and filters other than the target filter are summed to obtain a target distance sum of the target filter.

[0058] Specifically, for example, in the network layer of a network model, there are three filters, namely F1, F2, and F3. For the target filter F1, the target distance sum is L1=L(F1, F2)+L(F1, F3), for the target filter F2, the target distance sum is L2=L(F1, F2)+L(F2, F3), and for the target filter F3, the target distance sum is L3=L(F1, F3)+L(F2, F3).

[0059] Continue to refer to Figure 3 In step 252, at least one target filter is determined as a center filter based on the target distance sum.

[0060] In a specific example of this embodiment, based on the sum of the target distances, at least one target filter is determined as a center filter, which can be as follows: Figure 4 Follow the steps shown.

[0061] join Figure 4 , shows a detailed flow chart of determining at least one target filter as a center filter based on the target distance summation according to an embodiment of the present application. Specifically comprising steps 2521 to 2523:

[0062] In step 2521, the number of center filters to be retained is determined according to a predetermined filter pruning ratio.

[0063] Specifically, for example, a network layer of the network model includes 5 filters, and the predetermined filter pruning ratio is 60%, then the number of central filters that need to be retained in the network layer is 2.

[0064] In step 2522, the target filters are sorted in descending order of the sum of target distances.

[0065] In step 2523, the target filters with the highest ranking among the retained ones are determined as the central filters.

[0066] Continue to refer to Figure 3 In step 253, filters other than the at least one target filter are determined as non-central filters.

[0067] In a specific implementation of this embodiment, the central filter in a network layer of the network model may be determined according to the following formula:

[0068]

[0069] in, represents the filter space composed of different filters in a network layer of the network model; x represents a filter in the filter space; argmin represents a mathematical logic used to find the filter x in the i-th network layer with the smallest sum of distances from all filters in the network layer, which is the central filter GM of the network layer; GM represents the central filter; F i,j represents the jth filter in the i-th network layer.

[0070] In order to make those skilled in the art better understand this embodiment, Figure 5 This embodiment is described in detail:

[0071] Reference Figure 5 , shows a schematic diagram of determining a central filter based on the sum of target distances of filters according to an embodiment of the present application.

[0072] like Figure 5 As shown, in the filter space 500, there are five filters, A, B, C, D, and E. First, the distances between the filters are determined: L(A, B), L(A, C), L(A, D), L(A, E), L(B, C), L(B, D), L(B, E), L(C, D), L(C, E), and L(D, E). Among them, the target distance sum of filter A is L(A, B)+L(A, C)+L(A, D)+L(A, E); the target distance sum of filter B is L(A, B)+L(B, C)+L(B, D)+L(B, E); the target distance sum of filter C is L(A, C)+L(B, C)+L(C, D)+L(C, E); the target distance sum of filter D is L(A, D)+L(B, D)+L(C, D)+L(D, E); the target distance sum of filter E is L(A, E)+L(B, E)+L(C, E)+L(D, E).

[0073] If the predetermined filter pruning ratio is 60%, then the number of central filters that need to be retained in the filter space 500 of the network layer is 2. According to the sum of the target distances of each filter, it can be seen that the sum of the target distances of filter A and filter B in the filter space 500 is the smallest. Based on this, filter A and filter B are determined as the central filters of the network layer.

[0074] In the present application, the central filter and the non-central filter in the network model are determined by the vector distance between any two filters, which can avoid the local optimal situation of the model and overcome the problem of sensitivity to noise and isolated points in the model.

[0075] Continue to refer to Figure 2 In step 270, the non-central filters are pruned to obtain a compressed network model composed of the central filters.

[0076] In one embodiment of the present application, before pruning the non-central filter, the following steps may be performed: Figure 6 Steps shown.

[0077] Reference Figure 6 , shows a flow chart of a method before pruning the non-central filter according to an embodiment of the present application. Specifically comprising steps 261 to 263:

[0078] In step 261, for each center filter, a filter set is constructed.

[0079] In step 262, for each non-central filter, a target central filter closest to the non-central filter is determined, and the non-central filter is added to the filter set where the target central filter is located.

[0080] Specifically, for example, a network layer of the network model includes five filters, namely filters A, B, C, D, and E, among which filters A and B are central filters, filters C, D, and E are non-central filters, the central filter closest to the non-central filter C is A, the central filter closest to the non-central filter D is B, and the central filter closest to the non-central filter E is A. Therefore, filters A, C, and E constitute a filter set 1 centered on filter A, and filters B and D constitute a filter set 2 centered on filter B.

[0081] In step 263, filter parameters in the network model are updated to reduce the vector distance between the non-central filter and the central filter in the filter set.

[0082] In a specific example of this embodiment, the filter parameters in the network model can be updated as follows: Figure 7 The steps shown are performed:

[0083] Reference Figure 7 , shows a detailed flow chart of updating the filter parameters in the network model according to an embodiment of the present application. Specifically comprising steps 2631 to 2632:

[0084] In step 2631, the network model is iteratively trained based on the image training data.

[0085] In step 2632, in each training process, the set gradient of each filter set is determined respectively, so as to update the filter parameters of the filters in the filter set through the set gradient.

[0086] In a specific example of this embodiment, for each filter set in each training process, the following can be used: Figure 8 The steps shown determine the set gradient of the filter set:

[0087] Reference Figure 8 , shows a detailed flow chart of determining the set gradient of the filter set according to an embodiment of the present application. Specifically comprising steps 2633 to 2635:

[0088] In step 2633, the filter gradient of each filter in the filter set is obtained.

[0089] In step 2634, based on the filter gradients of each filter, a filter gradient mean and a vector distance between the filter gradient mean and each filter gradient are determined.

[0090] In step 2635, the set gradient of the filter set is determined based on the filter gradient mean and the vector distance between the filter gradient mean and each filter gradient.

[0091] In a specific implementation of this embodiment, the set gradient of the filter set may be determined according to the following formula:

[0092]

[0093] in, represents the mean filter gradient in a filter set, Represents the mean filter gradient in a filter set and the vector distance between the gradients of each filter.

[0094] Furthermore, the filter parameters of the filters in the filter set may be updated by the following formula:

[0095] F (j)’ +τΔF (j) →F (j)

[0096] Among them, F (j) represents the updated filter parameters of the filters in the filter set; (j)’ represents the filter parameters of the filters in the filter set before updating; ΔF(j) represents the set gradient of the filter set.

[0097] In order to make those skilled in the art better understand this embodiment, Fig. 9 This embodiment is described in detail:

[0098] Reference Fig. 9 , showing a schematic diagram of pruning the non-central filter according to an embodiment of the present application.

[0099] like Fig. 9 As shown, before the network model is iteratively trained based on the image training data 901, the network model includes two filter sets 902, each of which includes a center filter 904 and a non-center filter 903. The image training data 901 can obtain the first feature map 905 of the image training data by iteratively training each filter in the network model before.

[0100] In the process of iterative training of the network model based on the image training data 901, the filter parameters in the filter set are continuously updated through the set gradients in each filter set, so that the vector distance between the non-central filter and the central filter in the filter set is reduced, and the filters in different filter sets are not similar.

[0101] After iterative training of the network model based on the image training data 901, the non-center filters and the center filters in each filter set are further approached, so that the second feature map 906 extracted by the non-center filters and the center filters in each filter set is further approached to the image training data 901.

[0102] After pruning the non-central filters in each filter set, a compression network model is obtained, and the image training data 901 is input into the compression network model to obtain a third feature map 907.

[0103] In the present application, before pruning the non-central filters in each filter set, the network model is trained using image training data so that the non-central filters and central filters in each filter set in the network model are close to each other (i.e., the features of the image data extracted by the non-central filters and central filters in each filter set are further similar). The advantage of this is that the accuracy of the network model can be guaranteed while pruning the non-central filters in the network model.

[0104] In one embodiment of the present application, the non-central filter is pruned to obtain a compressed network model, which can be specifically performed as follows: Fig.10 Follow the steps shown.

[0105] See also Fig.10 , shows a detailed flow chart of pruning the non-central filter according to an embodiment of the present application. Specifically comprising steps 271 to 274:

[0106] In step 271, the non-central filters in the network model are trimmed to obtain the network model to be optimized.

[0107] In step 272, feature extraction is performed on the same image data using the initial network model and the network model to be optimized to obtain a first feature and a second feature, wherein the initial network model is a network model that has not been compressed.

[0108] In step 273, the difference between the first feature and the second feature is calculated.

[0109] In step 274, based on the difference, filter parameters of the filter in the network model to be optimized are adjusted to obtain the compressed network model.

[0110] In a specific implementation of this embodiment, the difference between the first feature and the second feature can be calculated as follows: Fig.11 Follow the steps shown.

[0111] See also Fig.11 , shows a detailed flow chart of calculating the difference between the first feature and the second feature according to an embodiment of the present application. Specifically comprising steps 2731 to 2733:

[0112] In step 2731, a first feature vector of the image data is determined using the first feature.

[0113] In step 2732, a second feature vector of the image data is determined based on the second feature.

[0114] In step 2733, a vector distance between the first feature vector and the second feature vector is calculated to determine the vector distance as a difference between the first feature and the second feature.

[0115] In the present application, the vector distance between the first feature vector and the second feature vector can be used to characterize the difference (similarity) between the first feature and the second feature, that is, the larger the vector distance, the greater the difference and the smaller the similarity.

[0116] Specifically, the vector distance can be measured by a loss function. The loss function can be a mean square error loss function (MSE Loss, Mean Squared Error Loss) or an L1 loss function (L1 Loss), etc. It can be understood that the loss function is not limited to the types listed above.

[0117] In order to make those skilled in the art better understand this embodiment, Fig.12 This embodiment is described in detail:

[0118] Reference Fig.12 , showing a schematic diagram of adjusting the filter parameters of the filter in the network model to be optimized according to an embodiment of the present application.

[0119] like Fig.12 As shown, by inputting the same image data 1201 into the initial network model 1202 and the network model to be optimized 1203 (i.e., the pruned network model) respectively, a first feature corresponding to the initial network model 1202 and a second feature corresponding to the network model to be optimized 1203 are obtained, and the filter parameters of the filter in the network model to be optimized 1203 are further adjusted according to the difference between the first feature and the second feature (i.e., the vector distance 1204) to obtain a compressed network model.

[0120] In the present application, by adjusting the filter parameters of the filter in the network model to be optimized, the difference between the second feature and the first feature can be reduced, further ensuring that the accuracy of the network model is maintained when the network model is compressed.

[0121] In one embodiment of the present application, after adjusting the filter parameters of the filter in the network model to be optimized based on the difference in step 274 to obtain the compressed network model, the compressed network model can also be used as a new network model and return to execute step 210 (i.e., the step of obtaining the network model to be compressed) to perform cyclic compression on the initial network model until a compressed network model that meets the preset compression conditions is obtained.

[0122] In order to make those skilled in the art better understand the present application, Fig.13 This embodiment is described in detail:

[0123] Reference Fig.13 , showing a schematic diagram of cyclic compression of an initial network model according to an embodiment of the present application.

[0124] like Fig.13As shown, the initial network model 1301 is compressed for the first time through the network model compression method proposed in the present application to obtain the network model 1302 after the first compression, and the network model 1302 after the first compression is continuously compressed in a loop according to the network model compression method proposed in the present application until the preset compression condition is met and the loop compression is stopped to obtain the final compressed network model 1303.

[0125] It should be noted that the preset compression condition can be a set reduction ratio for the number of filters in the network model, or a set number of model compression times. It can be understood that the preset compression condition is not limited to the types listed above.

[0126] In the present application, the network model is compressed multiple times, which can further ensure the accuracy and effect of the model at the same model compression ratio compared to the one-time compression of the network model.

[0127] In addition, based on the technical solution proposed in this application, the inventor of the present application compresses the face network model and the REID network model respectively.

[0128] Among them, for the face network model (820dpn model), when the compression ratio is 35%, the false pass rate of the model is one in a billion, and the accuracy rate decreases by 0.46% compared with the uncompressed model.

[0129] For the ResNet200 model of the REID network model, when the compression ratio is 40%, the model running speed is reduced by 124ms compared with the uncompressed model, and the accuracy decreases by 1.72% compared with the uncompressed model.

[0130] For the ResNet101+IBN model of the REID network model, when the compression ratio is 40%, the model running speed is reduced by 70ms compared with the uncompressed model, and the accuracy decreases by 0.8% compared with the uncompressed model.

[0131] The technical solution of the above-mentioned embodiment of the present application determines the central filter and non-central filter of the model through the vector distance between any two filters in the network model. Since the vector distance can reflect the similarity between filters in feature extraction to a certain extent, by removing the non-central filter determined based on the vector distance, a compressed network model composed of central filters is obtained, which can maintain the accuracy of the model while compressing the network model.

[0132] The following describes an apparatus embodiment of the present application, which can be used to execute the network model compression method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the network model compression method of the present application.

[0133] Fig.14 A block diagram of a network model compression device according to an embodiment of the present application is shown.

[0134] Reference Fig.14 As shown, a network model compression device 1400 according to an embodiment of the present application includes: an acquisition unit 1401, a calculation unit 1402, a determination unit 1403 and a pruning unit 1404.

[0135] Among them, the acquisition unit 1401 is used to obtain the network model to be compressed, and the network model includes at least two filters, and the filters are used to extract features from the data input into the network model; the calculation unit 1402 is used to calculate the distance between any two filters based on the filter parameters in each filter; the determination unit 1403 is used to determine the central filter and the non-central filter in the network model through the distance between any two filters, and the features extracted by the non-central filter are close to the features extracted by the central filter; the pruning unit 1404 is used to prune the non-central filter to obtain a compressed network model.

[0136] In some embodiments of the present application, based on the aforementioned scheme, the determination unit 1403 is configured as: for each target filter among the at least two filters, summing the vector distances between the target filter and filters other than the target filter to obtain the target distance sum of the target filter; based on the target distance sum, determining at least one target filter as a central filter; and determining filters other than the at least one target filter as non-central filters.

[0137] In some embodiments of the present application, based on the aforementioned scheme, the determination unit 1403 is configured to: determine the number of the central filters to be retained according to a predetermined filter pruning ratio; sort the target filters in descending order according to the sum of the target distances; and determine the retained number of target filters with the highest sorting as the central filter.

[0138] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: an updating unit, which is used to construct a filter set for each central filter before pruning the non-central filter; for each non-central filter, determine the target central filter closest to the non-central filter, and add the non-central filter to the filter set where the target central filter is located; update the filter parameters in the network model to reduce the vector distance between the non-central filter and the central filter in the filter set.

[0139] In some embodiments of the present application, based on the aforementioned scheme, the update unit is configured to: iteratively train the network model based on image training data; in each training process, determine the set gradient of each filter set respectively to update the filter parameters of the filters in the filter set through the set gradient.

[0140] In some embodiments of the present application, based on the aforementioned scheme, the update unit is configured to: for each filter set in each training process, obtain the filter gradient of each filter in the filter set; based on the filter gradient of each filter, determine the filter gradient mean, and the vector distance between the filter gradient mean and each filter gradient; based on the filter gradient mean, and the vector distance between the filter gradient mean and each filter gradient, determine the set gradient of the filter set.

[0141] In some embodiments of the present application, based on the aforementioned scheme, the pruning unit 1404 includes: a cropping unit used to crop the non-central filters in the network model to obtain the network model to be optimized; an extraction unit used to perform feature extraction on the same image data through the initial network model and the network model to be optimized, respectively, to obtain a first feature and a second feature, and the initial network model is a network model that has not been compressed; a calculation unit used to calculate the difference between the first feature and the second feature; an adjustment unit used to adjust the filter parameters of the filter in the network model to be optimized based on the difference to obtain the compressed network model.

[0142] In some embodiments of the present application, based on the aforementioned scheme, the computing unit is configured to: determine a first feature vector of the image data through the first feature; determine a second feature vector of the image data based on the second feature; calculate a vector distance between the first feature vector and the second feature vector to determine the vector distance as the difference between the first feature and the second feature.

[0143] In some embodiments of the present application, based on the aforementioned scheme, the device also includes a cyclic compression starting unit, which is used to adjust the filter parameters of the filter in the network model to be optimized based on the difference, and after obtaining the compressed network model, start using the compressed network model as a new network model, and return to execute the step of obtaining the network model to be compressed to perform cyclic compression on the initial network model until a compressed network model that meets the preset compression conditions is obtained.

[0144] Fig.15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0145] It should be noted that Fig.15 The computer system 1500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0146] like Fig.15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1502 or the program loaded from the storage part 1508 to the random access memory (RAM) 1503, such as executing the method described in the above embodiment. In the RAM 1503, various programs and data required for system operation are also stored. The CPU 1501, the ROM 1502 and the RAM 1503 are connected to each other through the bus 1504. The input / output (I / O) interface 1505 is also connected to the bus 1504.

[0147] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read therefrom is installed into the storage section 1508 as needed.

[0148] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1509, and / or installed from a removable medium 1511. When the computer program is executed by a central processing unit (CPU) 1501, various functions defined in the system of the present application are executed.

[0149] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0150] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0152] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the network model compression method described in the above embodiment.

[0153] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the network model compression method described in the above embodiment.

[0154] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0155] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0156] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0157] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A network model compression method, It is characterized in that The method comprises: Acquire a network model to be compressed, wherein the network model includes at least three filters, and the filters are used to extract features of image data input to the network model; Calculate the vector distance between any two filters based on the filter parameters in each filter; Determine the central filter and the non-central filter in the network model by the vector distance between any two filters, wherein the image features extracted by the central filter include part of the image features extracted by the non-central filter; The non-central filters are pruned to obtain a compressed network model composed of the central filters.

2. The method according to claim 1, It is characterized in that Determining the central filter and the non-central filter in the network model by the vector distance between any two filters includes: For each target filter of the at least three filters, summing the vector distances between the target filter and filters other than the target filter to obtain a target distance sum of the target filter; Based on the target distance sum, determining at least one target filter as a center filter; Filters other than the at least one target filter are determined as non-central filters.

3. The method according to claim 2, It is characterized in that The step of determining at least one target filter as a center filter based on the target distance summation comprises: Determining the number of center filters to be retained according to a predetermined filter pruning ratio; Sort each target filter in descending order of the sum of target distances; The retained number of target filters with the highest ranking are determined as central filters.

4. The method according to claim 1, It is characterized in that Before pruning the non-central filter, the method further includes: For each center filter, construct a filter set; For each non-center filter, determine the target center filter closest to the non-center filter, and add the non-center filter to the filter set where the target center filter is located; The filter parameters in the network model are updated to reduce the vector distance between the non-central filter and the central filter in the filter set.

5. The method according to claim 4, It is characterized in that The updating of the filter parameters in the network model comprises: Iteratively training the network model based on the image training data; In each training process, the set gradients of each filter set are determined respectively, so as to update the filter parameters of the filters in the filter set by means of the set gradients.

6. The method according to claim 5, It is characterized in that For each filter set in each training process, the set gradient of the filter set is determined by the following steps: Obtaining a filter gradient for each filter in the filter set; Determining a filter gradient mean value and a vector distance between the filter gradient mean value and each filter gradient based on the filter gradients of each filter; The set gradient of the filter set is determined based on a filter gradient mean and a vector distance between the filter gradient mean and each filter gradient.

7. The method according to any one of claims 1 to 6, It is characterized in that The step of pruning the non-central filter to obtain a compressed network model includes: Cutting the non-central filters in the network model to obtain the network model to be optimized; Extracting features of the same image data using an initial network model and the network model to be optimized respectively to obtain a first feature and a second feature, wherein the initial network model is a network model that has not been compressed; calculating a difference between the first feature and the second feature; Based on the difference, filter parameters of the filter in the network model to be optimized are adjusted to obtain the compressed network model.

8. The method according to claim 7, It is characterized in that The calculating the difference between the first feature and the second feature comprises: Determine a first feature vector of the image data using the first feature; determining a second feature vector of the image data based on the second feature; A vector distance between the first feature vector and the second feature vector is calculated to determine the vector distance as a difference between the first feature and the second feature.

9. The method according to claim 7, It is characterized in that After adjusting the filter parameters of the filter in the network model to be optimized based on the difference to obtain the compressed network model, the method further includes: The compressed network model is used as a new network model, and the step of obtaining the network model to be compressed is returned to execute, so as to perform cyclic compression on the initial network model until a compressed network model that meets the preset compression conditions is obtained.

10. A network model compression device, It is characterized in that The device comprises: An acquisition unit, used to acquire a network model to be compressed, wherein the network model includes at least three filters, and the filters are used to extract features from data input to the network model; A calculation unit, used to calculate the distance between any two filters based on the filter parameters in each filter; A determination unit is used to determine a central filter and a non-central filter in the network model by the distance between any two filters, wherein the feature extracted by the non-central filter is close to the feature extracted by the central filter; The pruning unit is used to prune the non-central filter to obtain a compressed network model.

11. An electronic device, It is characterized in that include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the network model compression method as described in any one of claims 1 to 9.

12. A computer readable medium, It is characterized in that It carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device implements the network model compression method as described in any one of claims 1-9.

13. A computer program product, comprising computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the network model compression method as described in any one of claims 1 to 9.