Network model compression method and device, electronic equipment and readable medium

Abstract

The present disclosure provides a network model compression method, device, electronic equipment and readable medium, wherein the network model compression method comprises: determining a corresponding low-rank decomposition matrix group according to singular values of a specified matrix of an original network model to be compressed; and compressing the low-rank decomposition matrix group based on a matrix distillation loss function corresponding to the original network model. Through the embodiment of the present disclosure, the size of the network model is reduced, and the calculation rate of the network model is improved on the premise of ensuring the calculation accuracy of the network model.

Description

Technical Field

[0001] This disclosure relates to the field of network technology, and more specifically, to a network model compression method, apparatus, electronic device, and readable medium. Background Technology

[0002] Currently, deep learning models are widely used in various online tasks, such as text classification, image detection, and speech recognition. With the rise of pre-training technology, model size is growing exponentially in fields such as NLP (Natural Language Processing), CV (Computer Vision), and ASR (Automatic Speech Recognition). While large models offer higher accuracy and can adapt to various downstream tasks, their massive parameter count and slow computation speed require significant GPU computing resources, resulting in low computational efficiency and high data maintenance costs. This poses a serious challenge to efficient model inference, hindering the online deployment and industrial application of large models.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a network model compression method, apparatus, electronic device, and readable medium to overcome, at least to some extent, the problem of large network model size caused by limitations and defects in related technologies.

[0005] According to a first aspect of the present disclosure, a network model compression method is provided, comprising: determining a corresponding low-rank decomposition matrix group based on the singular values ​​of a specified matrix of the original network model to be compressed; and compressing the low-rank decomposition matrix group based on a matrix distillation loss function corresponding to the original network model.

[0006] In one exemplary embodiment of this disclosure, determining the corresponding low-rank decomposition matrix set based on the singular values ​​of a specified matrix of the original network model to be compressed includes:

[0007] The singular values ​​of a specified matrix in the original network model are determined using the singular value decomposition algorithm.

[0008] The first dimension parameter of the low-rank decomposition matrix group is determined based on the number of singular values ​​and the preset hyperparameters;

[0009] The dimensions of the singular value matrix are determined based on the original network model.

[0010] The first low-dimensional matrix and the second low-dimensional matrix are randomly initialized using a normal distribution, and the singular value matrix is ​​constructed. The low-rank decomposition matrix group includes the first low-dimensional matrix, the second low-dimensional matrix, and the singular value matrix.

[0011] In one exemplary embodiment of this disclosure, determining the dimension of the singular value matrix based on the original network model includes:

[0012] Determine the second and third dimension parameters of the specified matrix in the original network model;

[0013] The dimension of the first low-dimensional matrix is ​​determined based on the first dimension parameter and the second dimension parameter;

[0014] The dimensions of the second low-dimensional matrix are determined based on the second dimension parameter and the third dimension parameter;

[0015] The dimension of the singular value matrix is ​​determined based on the second dimension parameter.

[0016] In one exemplary embodiment of this disclosure, determining the first dimension parameter of the low-rank decomposition matrix group based on the number of singular values ​​and preset hyperparameters includes:

[0017] The initial value of the first dimension parameter is determined based on the number of singular values ​​and the hyperparameter;

[0018] The target value of the first dimension parameter is determined based on the dimension of the specified matrix in the original network model;

[0019] Create iteration parameters for the initial values ​​of the first dimension parameters;

[0020] The initial value of the first dimension parameter is iteratively calculated based on the iteration parameters until the target value of the first dimension parameter is reached.

[0021] In one exemplary embodiment of this disclosure, determining the target value of the first dimension parameter based on the dimension of a specified matrix in the original network model includes:

[0022] Determine the minimum dimension of the specified matrix in the original network model;

[0023] The target value of the first dimension parameter is determined based on the minimum value and the preset ratio value.

[0024] In one exemplary embodiment of this disclosure, compressing the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model includes:

[0025] The sample is input into the matrix corresponding to the original network model, and the output result of the original network model is determined and denoted as the first output result;

[0026] The sample is input into the low-rank decomposition matrix group to be compressed, and the output result of the low-rank decomposition matrix group is determined and denoted as the second output result.

[0027] Determine the output loss based on the first output result and the second output result;

[0028] Determine the regularized loss corresponding to the singular value matrix;

[0029] Determine the processing loss of the original network model for the samples;

[0030] The low-rank decomposition matrix group is compressed based on the output loss, the regularization loss, and the processing loss.

[0031] In one exemplary embodiment of this disclosure, determining the regularization loss corresponding to the singular value matrix includes:

[0032] The regularized loss of the L0 norm corresponding to the singular value matrix is ​​calculated based on the L0 optimization algorithm.

[0033] In one exemplary embodiment of this disclosure, determining the processing loss of the original network model for the sample includes:

[0034] The processing task for the samples is determined to be a classification task;

[0035] Determine the classification loss of the original network model after processing the classification task, wherein the processing loss includes the classification loss.

[0036] According to a second aspect of the present disclosure, a network model compression apparatus is provided, comprising:

[0037] The determination module is set to determine the corresponding low-rank decomposition matrix set based on the singular values ​​of the specified matrix of the original network model to be compressed.

[0038] The compression module is configured to compress the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model.

[0039] According to a third aspect of this disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method as described in any of the preceding methods based on instructions stored in the memory.

[0040] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a program stored thereon that, when executed by a processor, implements the network model compression method as described in any of the preceding claims.

[0041] In this embodiment, the corresponding low-rank decomposition matrix group is determined based on the singular values ​​of the specified matrix of the original network model to be compressed, and the low-rank decomposition matrix group is compressed based on the matrix distillation loss function corresponding to the original network model. This reduces the size of the network model and improves the computational speed of the network model while ensuring the computational accuracy of the network model.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0044] Figure 1 A schematic diagram of an exemplary system architecture for which the network model compression scheme of the present invention can be applied is shown;

[0045] Figure 2 This is a flowchart of a network model compression method according to an exemplary embodiment of this disclosure;

[0046] Figure 3 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0047] Figure 4 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0048] Figure 5 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0049] Figure 6 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0050] Figure 7 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0051] Figure 8 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0052] Figure 9 This is a flowchart of another network model compression method in an exemplary embodiment of this disclosure;

[0053] Figure 10 This is a schematic diagram of the architecture of a network model compression scheme in an exemplary embodiment of this disclosure;

[0054] Figure 11 This is a block diagram of a network model compression apparatus according to an exemplary embodiment of the present disclosure;

[0055] Figure 12 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0056] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0057] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0058] Figure 1 A schematic diagram of an exemplary system architecture for which the network model compression scheme of embodiments of the present invention can be applied is shown.

[0059] like Figure 1As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0060] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0061] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers, etc.

[0062] In some embodiments, the network model compression method provided in this invention is generally executed by server 105, and correspondingly, the network model compression device is generally located in terminal device 103 (or terminal device 101 or 102). In other embodiments, some terminals may have functions similar to those of the server device to execute this disclosure.

[0063] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0064] Figure 2 This is a flowchart of a network model compression method in an exemplary embodiment of this disclosure.

[0065] refer to Figure 2 Network model compression methods may include:

[0066] Step S202: Determine the corresponding low-rank decomposition matrix group based on the singular values ​​of the specified matrix of the original network model to be compressed.

[0067] Step S204: Compress the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model.

[0068] In this embodiment, the corresponding low-rank decomposition matrix group is determined based on the singular values ​​of the specified matrix of the original network model to be compressed, and the low-rank decomposition matrix group is compressed based on the matrix distillation loss function corresponding to the original network model. This reduces the size of the network model and improves the computational speed of the network model while ensuring the computational accuracy of the network model.

[0069] In one exemplary embodiment of this disclosure, the specified matrix may be a weight matrix.

[0070] The following section provides a detailed explanation of each step in the network model compression method.

[0071] In one exemplary embodiment of this disclosure, such as Figure 3 As shown, determining the corresponding low-rank decomposition matrix set based on the singular values ​​of the specified matrix of the original network model to be compressed includes:

[0072] Step S302: Determine the singular values ​​of a specified matrix in the original network model using the singular value decomposition algorithm.

[0073] Step S304: Determine the first dimension parameter of the low-rank decomposition matrix group based on the number of singular values ​​and the preset hyperparameters.

[0074] Step S306: Determine the dimension of the singular value matrix based on the original network model.

[0075] Step S308: Initialize the first low-dimensional matrix and the second low-dimensional matrix randomly using a normal distribution, and construct the singular value matrix. The low-rank decomposition matrix group includes the first low-dimensional matrix, the second low-dimensional matrix, and the singular value matrix.

[0076] In one exemplary embodiment of this disclosure, such as Figure 4 As shown, determining the dimensions of the singular value matrix based on the original network model includes:

[0077] Step S402: Determine the second-dimensional parameters and the third-dimensional parameters of the specified matrix in the original network model.

[0078] Step S404: Determine the dimension of the first low-dimensional matrix based on the first dimension parameter and the second dimension parameter.

[0079] Step S406: Determine the dimension of the second low-dimensional matrix based on the second dimension parameter and the third dimension parameter.

[0080] Step S408: Determine the dimension of the singular value matrix based on the second dimension parameter.

[0081] In one exemplary embodiment of this disclosure, the computational cost of the neural network model is reduced by constructing a singular value matrix, and the dimensionality of the entire computational architecture is reduced by using a first low-dimensional matrix and a second low-dimensional matrix. After replacing the original network model, the model size and data volume are effectively reduced.

[0082] In one exemplary embodiment of this disclosure, both the first low-dimensional matrix and the second low-dimensional matrix are relative to the dimension of the specified matrix.

[0083] In one exemplary embodiment of this disclosure, such as Figure 5 As shown, determining the first dimension parameter of the low-rank decomposition matrix group based on the number of singular values ​​and preset hyperparameters includes:

[0084] Step S502: Determine the initial value of the first dimension parameter based on the number of singular values ​​and the hyperparameter.

[0085] Step S505: Determine the target value of the first dimension parameter based on the dimension of the specified matrix in the original network model.

[0086] Step S506: Create iteration parameters for the initial values ​​of the first dimension parameters.

[0087] Step S508: Iteratively calculate the initial value of the first dimension parameter according to the iteration parameters until the target value of the first dimension parameter is reached.

[0088] In one exemplary embodiment of this disclosure, such as Figure 6 As shown, determining the target value of the first dimension parameter based on the dimension of the specified matrix in the original network model includes:

[0089] Step S602: Determine the minimum dimension of the specified matrix in the original network model.

[0090] Step S604: Determine the target value of the first dimension parameter based on the minimum value and the preset ratio value.

[0091] In one exemplary embodiment of this disclosure, by determining the minimum value of the dimension of a specified matrix in the original network model and determining the target value of the first dimension parameter based on the minimum value and a preset ratio, the singular value matrix can be minimized, thereby reducing the dimension of the low-rank decomposition matrix group and thus reducing the dimension of the original network model.

[0092] In one exemplary embodiment of this disclosure, such as Figure 7 As shown, compressing the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model includes:

[0093] Step S702: Input the sample into the matrix corresponding to the original network model, and determine the output result of the original network model, which is denoted as the first output result.

[0094] Step S704: Input the sample into the low-rank decomposition matrix group to be compressed, and determine the output result of the low-rank decomposition matrix group, denoted as the second output result.

[0095] Step S707: Determine the output loss based on the first output result and the second output result.

[0096] Step S708: Determine the regularized loss corresponding to the singular value matrix.

[0097] Step S710: Determine the processing loss of the original network model for the sample.

[0098] Step S712: Compress the low-rank decomposition matrix group according to the output loss, the regularization loss, and the processing loss.

[0099] In one exemplary embodiment of this disclosure, a total loss function is determined by the output loss, the regularization loss, and the processing loss. The parameters of the low-rank decomposition matrix group are optimized by iterating the total loss function until convergence, thereby achieving compression of the low-rank decomposition matrix group.

[0100] In one exemplary embodiment of this disclosure, such as Figure 8 As shown, determining the regularized loss corresponding to the singular value matrix includes:

[0101] Step S802: Calculate the regularized loss of the L0 norm corresponding to the singular value matrix based on the L0 optimization algorithm.

[0102] In one exemplary embodiment of this disclosure, such as Figure 9 As shown, determining the processing loss of the original network model for the samples includes:

[0103] Step S902: Determine that the processing task of the sample is a classification task.

[0104] Step S904: Determine the classification loss of the original network model after processing the classification task, wherein the processing loss includes the classification loss.

[0105] Figure 10 This is a schematic diagram of the architecture of a network model compression scheme in an exemplary embodiment of this disclosure.

[0106] like Figure 10 As shown, the network model compression scheme of the exemplary embodiment of this disclosure mainly includes two parts: a low-rank decomposition matrix group 1006 and a matrix distillation loss 1008. For a given weight matrix W 1004 of the original model, the weight matrix W 1004 can be represented by a set of low-dimensional matrices with low parameterization.

[0107] The low-quality decomposition matrix group mainly consists of two low-dimensional matrices U (dimension d1×r) and V (dimension r×d2), and a singular value matrix Σ, whose lowest dimension is r. It is mainly used to reduce the model parameter W1004 (dimension d1×d2) and improve the computation speed.

[0108] In addition, matrix distillation loss is mainly used to transfer the deep network information represented by the original matrix to a low-rank decomposition matrix group 1006, and continuously optimize the minimum dimension r of the matrix group, so as to reduce r as much as possible while ensuring the accuracy of the model.

[0109] In the network model compression scheme of the exemplary embodiments of this disclosure, the step of determining the low-rank decomposition matrix group 1006 includes:

[0110] (1) Obtain the singular values ​​of matrix W1004 in the original model using SVD (Singular Value Decomposition algorithm), assuming they are {λ1,λ2,...,λ1}. K}, where λ k The first singular value of W1004 is represented. The matrix W1004 contains at most K singular values ​​(K≤min(d1,d2)).

[0111] (2) Set the initialization r0 = α*K, where α is a hyperparameter and its initial value can be set to 2. Use a normal distribution to randomly initialize each element of the low-quality matrices U and V, and at the same time construct a diagonal matrix Σ = diagonal({λ1,λ2,...,λ...). K}).

[0112] (3) Design a linear parameter scheduler for parameter r, with an initial value of r0 and a final target value of r. s << min(d1,d2), as the iteration progresses, r gradually decreases to the given final target value. After initialization, the model to be compressed (a low-rank decomposition matrix group with 1006 parameters) can learn and adapt well to the knowledge in the original model and data.

[0113] In one embodiment of this disclosure, the final target value is generally selected manually according to actual needs, and is usually set to about 1 / 30 to 1 / 20 of min(d1,d2).

[0114] In one embodiment of this disclosure, the final target value ranges from [8, 64].

[0115] In the network model compression scheme of the exemplary embodiments of this disclosure, for a specific downstream task (e.g., a classification task), for a given input X∈R d1 At the t-th iteration, the expected value of r is r tSpecifically, the low-rank decomposition matrix group 1006 can be compressed using the following steps:

[0116] (1) For each W 1004 in the original model, calculate the output f(W) = σ(W) of the original model. T X), where σ is the activation function corresponding to the weight connection of the neural network.

[0117] (2) For the low-rank decomposition matrix group 1006 that needs to be compressed, the low-rank decomposition matrix group 1006 can be characterized as g(U∑V), and g(U∑V)=σ( ... T X), select the largest r from Σ. t Parameters Mask it to 1, and set the mask of other elements to 0 (so that the calculated results of the rows and columns corresponding to U and V are all 0).

[0118] (3) Decompose the low-rank matrix group 1006 (a compressible matrix of rank r) in the t-th iteration. The derivation process of the second-order Taylor expansion is shown in Equations (1) and (2) below:

[0119]

[0120]

[0121] In the above formula, for some linear activation functions, the second derivative (g″) is 0, containing only the information of the first derivative (g′). For other non-linear activation functions, the second derivative provides additional fitting information.

[0122] Where U represents the left singular matrix, V represents the right singular matrix, and ∑ represents the singular value matrix (only the diagonal has a value λ). i (and all others are 0), Characterizing the compressible matrix at step t, Let W = 1004, where W represents the original weights of the matrix to be compressed, σ represents the activation function, σ′ represents the first derivative of the activation function, σ″ represents the second derivative of the activation function, and X represents the input sample matrix 1002. T The transpose matrix represents the input sample, and g represents the neural network function with parameter W = 1004.

[0123] (4) Based on the output values ​​obtained before and after this matrix decomposition, the output loss of the original model and the decomposed model is calculated as shown in the following formula (3):

[0124]

[0125] (5) Introduce the L0 optimization algorithm to calculate the regularization loss ι of the L0 norm of the diagonal matrix Σ parameter in the low-rank decomposition matrix group 1006. 0-norm In addition, it is necessary to introduce the classification distillation loss of downstream tasks. pred Thus, the expression for the overall calculated loss ι is shown in the following formula (4):

[0126] ι=ι r +ι 0-norm +ι pred , formula (4).

[0127] (6) Optimize the parameters of the low-rank matrix group U, V, and Σ using the loss function above until the model converges. This will allow you to learn the low-rank matrix parameters that retain the knowledge of the original model to the maximum extent.

[0128] After completing the neural network compression according to the above steps, the inference calculation process based on the compressed neural network model is as follows:

[0129] Select the largest r from the Σ matrix. s The decomposed matrix group is compressed by setting one parameter to 0, then setting the other parameters to 0, and simultaneously deleting the rows and columns corresponding to these 0 elements from U and V. and At this point, the computational complexity and storage size of the neural network are reduced to Because r s Since the value is << min(d1,d2), this ratio is very small, resulting in a significant speedup for the model.

[0130] Based on this, the neural network compression scheme defined in the embodiments of this disclosure can be optimized and compressed for any neural network matrix, which has high versatility and is very useful for industrial implementation and deployment.

[0131] Corresponding to the above method embodiments, this disclosure also provides a network model compression apparatus, which can be used to execute the above method embodiments.

[0132] Figure 11 This is a block diagram of a network model compression apparatus according to an exemplary embodiment of the present disclosure.

[0133] refer to Figure 11 The network model compression device 1100 may include:

[0134] The determination module 1102 is configured to determine the corresponding low-rank decomposition matrix group based on the singular values ​​of the specified matrix of the original network model to be compressed.

[0135] The compression module 1104 is configured to compress the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model.

[0136] In one exemplary embodiment of this disclosure, the determining module 1102 is further configured to:

[0137] The singular values ​​of a specified matrix in the original network model are determined using the singular value decomposition algorithm.

[0138] The first dimension parameter of the low-rank decomposition matrix group is determined based on the number of singular values ​​and the preset hyperparameters;

[0139] The dimensions of the singular value matrix are determined based on the original network model.

[0140] The first low-dimensional matrix and the second low-dimensional matrix are randomly initialized using a normal distribution, and the singular value matrix is ​​constructed. The low-rank decomposition matrix group includes the first low-dimensional matrix, the second low-dimensional matrix, and the singular value matrix.

[0141] In one exemplary embodiment of this disclosure, the determining module 1102 is further configured to:

[0142] Determine the second and third dimension parameters of the specified matrix in the original network model;

[0143] The dimension of the first low-dimensional matrix is ​​determined based on the first dimension parameter and the second dimension parameter;

[0144] The dimensions of the second low-dimensional matrix are determined based on the second dimension parameter and the third dimension parameter;

[0145] The dimension of the singular value matrix is ​​determined based on the second dimension parameter.

[0146] In one exemplary embodiment of this disclosure, the determining module 1102 is further configured to:

[0147] The initial value of the first dimension parameter is determined based on the number of singular values ​​and the hyperparameter;

[0148] The target value of the first dimension parameter is determined based on the dimension of the specified matrix in the original network model;

[0149] Create iteration parameters for the initial values ​​of the first dimension parameters;

[0150] The initial value of the first dimension parameter is iteratively calculated based on the iteration parameters until the target value of the first dimension parameter is reached.

[0151] In one exemplary embodiment of this disclosure, the determining module 1102 is further configured to:

[0152] Determine the minimum dimension of the specified matrix in the original network model;

[0153] The target value of the first dimension parameter is determined based on the minimum value and the preset ratio value.

[0154] In one exemplary embodiment of this disclosure, the compression module 1104 is further configured to:

[0155] The sample is input into the matrix corresponding to the original network model, and the output result of the original network model is determined and denoted as the first output result;

[0156] The sample is input into the low-rank decomposition matrix group to be compressed, and the output result of the low-rank decomposition matrix group is determined and denoted as the second output result.

[0157] Determine the output loss based on the first output result and the second output result;

[0158] Determine the regularized loss corresponding to the singular value matrix;

[0159] Determine the processing loss of the original network model for the samples;

[0160] The low-rank decomposition matrix group is compressed based on the output loss, the regularization loss, and the processing loss.

[0161] In one exemplary embodiment of this disclosure, the compression module 1104 is further configured to:

[0162] The regularized loss of the L0 norm corresponding to the singular value matrix is ​​calculated based on the L0 optimization algorithm.

[0163] In one exemplary embodiment of this disclosure, determining the processing loss of the original network model for the sample includes:

[0164] The processing task for the samples is determined to be a classification task;

[0165] Determine the classification loss of the original network model after processing the classification task, wherein the processing loss includes the classification loss.

[0166] Since the functions of the network model compression device 1100 have been described in detail in their respective method embodiments, they will not be repeated here.

[0167] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0168] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0169] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0170] The following reference Figure 12 To describe an electronic device 1200 according to an embodiment of the present invention. Figure 12 The electronic device 1200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0171] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, and a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210).

[0172] The storage unit stores program code that can be executed by the processing unit 1210, causing the processing unit 1210 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1210 can perform the method shown in the embodiments of this disclosure.

[0173] Storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include a read-only memory (ROM) 12203.

[0174] Storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0175] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0176] Electronic device 1200 can also communicate with one or more external devices 1240 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). Communication can be made through input / output (I / O) interface 1250. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1260. As shown, network adapter 1260 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0177] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0178] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0179] The program product for implementing the above-described method according to embodiments of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0180] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0181] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. The propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0182] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0183] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0184] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0185] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and concept of this disclosure are indicated by the claims.

Claims

1. A network model compression method, applicable to computing network models for any of the following tasks: text classification, image detection, and speech recognition, wherein the network model is deployed on an electronic device, characterized in that... The network model compression method includes: The corresponding low-rank decomposition matrix set is determined based on the singular values ​​of the specified matrix of the original network model to be compressed, including: The singular values ​​of a specified matrix in the original network model are determined using the singular value decomposition algorithm. The first dimension parameter of the low-rank decomposition matrix group is determined based on the number of singular values ​​and the preset hyperparameters; The dimensions of the singular value matrix are determined based on the original network model. The first low-dimensional matrix and the second low-dimensional matrix are randomly initialized using a normal distribution, and the singular value matrix is ​​constructed. The low-rank decomposition matrix group includes the first low-dimensional matrix, the second low-dimensional matrix, and the singular value matrix. The low-rank decomposition matrix group is compressed based on the matrix distillation loss function corresponding to the original network model, including: The sample is input into the matrix corresponding to the original network model, and the output result of the original network model is determined and denoted as the first output result; The sample is input into the low-rank decomposition matrix group to be compressed, and the output result of the low-rank decomposition matrix group is determined and denoted as the second output result. Determine the output loss based on the first output result and the second output result; Determine the regularized loss corresponding to the singular value matrix; Determine the processing loss of the original network model for the samples; The low-rank decomposition matrix group is compressed based on the output loss, the regularization loss, and the processing loss.

2. The network model compression method as described in claim 1, characterized in that, Determining the dimensions of the singular value matrix based on the original network model includes: Determine the second and third dimension parameters of the specified matrix in the original network model; The dimension of the first low-dimensional matrix is ​​determined based on the first dimension parameter and the second dimension parameter; The dimensions of the second low-dimensional matrix are determined based on the second dimension parameter and the third dimension parameter; The dimension of the singular value matrix is ​​determined based on the second dimension parameter.

3. The network model compression method as described in claim 2, characterized in that, The first dimension parameter of the low-rank decomposition matrix group is determined based on the number of singular values ​​and preset hyperparameters, including: The initial value of the first dimension parameter is determined based on the number of singular values ​​and the hyperparameter; The target value of the first dimension parameter is determined based on the dimension of the specified matrix in the original network model; Create iteration parameters for the initial values ​​of the first dimension parameters; The initial value of the first dimension parameter is iteratively calculated based on the iteration parameters until the target value of the first dimension parameter is reached.

4. The network model compression method as described in claim 3, characterized in that, Determining the target value of the first dimension parameter based on the dimension of the specified matrix in the original network model includes: Determine the minimum dimension of the specified matrix in the original network model; The target value of the first dimension parameter is determined based on the minimum value and the preset ratio value.

5. The network model compression method as described in claim 1, characterized in that, Determining the regularized loss corresponding to the singular value matrix includes: The regularized loss of the L0 norm corresponding to the singular value matrix is ​​calculated based on the L0 optimization algorithm.

6. The network model compression method as described in claim 1, characterized in that, Determining the processing loss of the original network model for the samples includes: The processing task for the samples is determined to be a classification task; Determine the classification loss of the original network model after processing the classification task, wherein the processing loss includes the classification loss.

7. A network model compression device, suitable for computing network models for any of the following tasks: text classification, image detection, and speech recognition, wherein the network model is deployed in an electronic device, characterized in that... The network model compression device includes: The determination module is configured to determine the corresponding low-rank decomposition matrix set based on the singular values ​​of a specified matrix of the original network model to be compressed, including: The singular values ​​of a specified matrix in the original network model are determined using the singular value decomposition algorithm. The first dimension parameter of the low-rank decomposition matrix group is determined based on the number of singular values ​​and the preset hyperparameters; The dimensions of the singular value matrix are determined based on the original network model. The first low-dimensional matrix and the second low-dimensional matrix are randomly initialized using a normal distribution, and the singular value matrix is ​​constructed. The low-rank decomposition matrix group includes the first low-dimensional matrix, the second low-dimensional matrix, and the singular value matrix. The compression module is configured to compress the low-rank decomposition matrix group based on the matrix distillation loss function corresponding to the original network model, including: The sample is input into the matrix corresponding to the original network model, and the output result of the original network model is determined and denoted as the first output result; The sample is input into the low-rank decomposition matrix group to be compressed, and the output result of the low-rank decomposition matrix group is determined and denoted as the second output result. Determine the output loss based on the first output result and the second output result; Determine the regularized loss corresponding to the singular value matrix; Determine the processing loss of the original network model for the samples; The low-rank decomposition matrix group is compressed based on the output loss, the regularization loss, and the processing loss.

8. An electronic device, characterized in that, include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the network model compression method as described in any one of claims 1-6 based on instructions stored in the memory.

9. A computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the network model compression method as described in any one of claims 1-6.

Images