Compression methods, compression devices, and deployment methods for neural network models

Abstract

This invention provides a method, apparatus, and deployment method for compressing neural network models. The compression method includes: acquiring a trained first neural network model; inputting training data into the first neural network model to obtain a first output result and tail features, where the tail features are the features of the last convolutional layer of the first neural network model; splitting the tail features into K groups of features, where K is an integer greater than 1; training K sub-task models based on the K groups of features, where the loss function of the output features of the K sub-task models and the K groups of features is less than a first preset threshold; replacing the network before the output layer in the first neural network model with the K sub-task models to obtain a compressed second neural network model. This invention simulates the tail features of the first neural network model using K lightweight sub-task models, thereby reducing the parameters and computational load of the neural network model and lowering the requirements for computing resources and memory.

Description

Technical Field

[0001] This invention relates primarily to the field of artificial intelligence technology, and in particular to a method, apparatus, and deployment method for compressing neural network models. Background Technology

[0002] In practical applications of artificial intelligence, there are often high requirements for model computation speed and network transmission speed. A common approach is to deploy neural network models directly on edge devices, thereby saving time for data to travel up to cloud devices and down to user devices.

[0003] However, as the demand for machine intelligence in the field of artificial intelligence increases, neural network models are becoming increasingly complex, requiring significantly more computation and storage. Edge devices have limited computing and storage resources, making it difficult to deploy neural network models efficiently on them. Therefore, neural network models need to be compressed to achieve lightweight deployment. Model compression is an effective method to address the drawbacks of large model size and slow operation. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device and method for compressing neural network models, thereby solving the problem that computationally intensive neural network models are difficult to deploy efficiently on edge devices.

[0005] To address the aforementioned technical problems, this invention provides a method for compressing a neural network model, comprising: acquiring a trained first neural network model; inputting training data into the first neural network model to obtain a first output result and tail features, wherein the tail features are features of the last convolutional layer of the first neural network model; splitting the tail features into K groups of features, where K is an integer greater than 1; training K sub-task models based on the K groups of features, wherein the loss function of the output features of the K sub-task models and the K groups of features is less than a first preset threshold; replacing the network before the output layer in the first neural network model with the K sub-task models to obtain a compressed second neural network model.

[0006] Optionally, training K sub-task models based on the K sets of features includes: constructing K untrained initial sub-task models; using each set of features in the K sets as a target output of the initial sub-task model; inputting the training data into the initial sub-task model to obtain the actual output; and iteratively training the initial sub-task model based on the sub-loss function of the actual output and the target output to obtain a trained sub-task model.

[0007] Optionally, the initial subtask model is the Mobilenet v3 model.

[0008] Optionally, the method further includes: determining whether the computational cost of each of the K sub-task models is less than a preset second preset threshold; if not, increasing the value of K and reconstructing the second neural network model.

[0009] Optionally, the method further includes: determining whether the computation time of each sub-task model in the K sub-task models is less than a preset third preset threshold; if not, increasing the value of K and reconstructing the second neural network model.

[0010] Optionally, the method further includes: inputting the training data into the second neural network model to obtain a second output result; and fine-tuning the parameters of the second neural network model based on the first output result and the second output result to obtain an optimized third neural network model.

[0011] Optionally, the loss function includes distance loss.

[0012] To address the aforementioned technical problems, the present invention provides a compression device for a neural network model, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the compression method described above.

[0013] To address the aforementioned technical problems, the present invention provides a computer-readable medium storing computer program code, wherein the computer program code is compressed as described above when executed by a processor.

[0014] To address the aforementioned technical problems, this invention provides a method for deploying a neural network model, comprising: obtaining a compressed second neural network model using the method described above; deploying each subtask model in the second neural network model on an edge device; and deploying the output layer in the second neural network model on a main control device.

[0015] To address the aforementioned technical problems, this invention provides a deployment system for a neural network model, comprising: multiple edge devices, each edge device deploying a sub-task model of a second neural network model, the number of edge devices being equal to the number of the sub-task models; and a master control device on which the output layer of the second neural network model is deployed.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] The neural network model compression method and device of the present invention reduce the parameters and computational load of the neural network model by splitting the tail features of the first neural network model into K groups of features, training K lightweight sub-task models based on the K groups of features, and replacing the original feature extraction network with the K sub-task models. This reduces the requirements for computing resources and memory, and allows it to run widely on resource-constrained edge devices. In the deployment method of the neural network model of the present invention, the amount of data interaction between edge devices is minimal, and the computing paths between devices are independent. Therefore, even if there are performance differences between these devices, a single device will not slow down due to the computing bottleneck of other devices, thereby maximizing the computational efficiency of the entire neural network model. Attached Figure Description

[0018] The accompanying drawings are included to provide a further understanding of this application; they are incorporated into and constitute a part of this application. The drawings illustrate embodiments of this application and, together with this specification, serve to explain the principles of the invention. In the drawings:

[0019] Figure 1 This is a flowchart of a method for compressing a neural network model according to an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the structure of a first neural network model according to an embodiment of the present invention;

[0021] Figure 3 yes Figure 1 A flowchart of an embodiment of step S14;

[0022] Figure 4 This is a schematic diagram of training K sub-task models based on K sets of features according to an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of the structure of a second neural network model according to an embodiment of the present invention;

[0024] Figure 6 yes Figure 1 A flowchart of the compression method for the neural network model in the optimized embodiment;

[0025] Figure 7 This is a system block diagram of a neural network model compression device according to an embodiment of this application;

[0026] Figure 8 This is a flowchart of a method for deploying a neural network model according to an embodiment of the present invention;

[0027] Figure 9 This is a block diagram of a deployment system for a neural network model according to an embodiment of the present invention. Detailed Implementation

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this application. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0029] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0030] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0031] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, these terms have no special meaning and therefore should not be construed as limiting the scope of protection of this application. In addition, although the terminology used in this application is selected from commonly known and used terms, some terms mentioned in this application's specification may have been chosen by the applicant according to his or her judgment, and their detailed meanings are explained in the relevant sections of this description. Moreover, this application should be understood not only through the actual terms used, but also through the meaning implied by each term.

[0032] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0033] Figure 1 This is a flowchart of a method for compressing a neural network model according to an embodiment of the present invention. Figure 1 As shown, the neural network model compression method 100 includes the following steps:

[0034] Step S11: Obtain the trained first neural network model.

[0035] The first neural network model can be a neural network model trained based on labeled data. The first neural network model of the present invention has the characteristics of a large number of parameters and a large amount of computation.

[0036] In some embodiments, before step S11, the method includes: obtaining a first neural network model through supervised learning using labeled data. Supervised learning is a training method in machine learning, which refers to the process of adjusting the parameters of a neural network model using a set of data with known labels to achieve the required performance.

[0037] Step S12: Input the training data into the first neural network model to obtain the first output result and tail features. The tail features are the features of the last convolutional layer of the first neural network model.

[0038] Training data refers to data excluding labels. Training data can be data generated by removing labels from labeled data, or data that has never been manually labeled.

[0039] Figure 2 This is a schematic diagram of the structure of a first neural network model according to an embodiment of the present invention. Figure 2 As shown, the first neural network model 200 includes N layers. The Nth layer is the output layer, and the (N-1)th layer is the last convolutional layer. The features output by the (N-1)th layer are defined as tail features. The output layer includes a pooling layer, a linear regression layer, a softmax classification layer, or a combination of these layers. Taking face recognition as an example, the first neural network model 200 is input with training data (e.g., a face image). Based on the training data, the first neural network model 200 outputs the tail features extracted by the (N-1)th layer (the face features corresponding to the image) and a first output result (whether the recognition passes).

[0040] Step S13: Split the tail features into K groups of features, where K is an integer greater than 1.

[0041] Continue to refer to Figure 2 ,like Figure 2 As shown, the tail features are split into group 1, group 2, ..., group K. The initial value of K can be equal to the number of current edge devices. There are two reasons for choosing to split the tail features: first, the features of this layer have the highest semantic information representation ability in the entire network; second, the output of this layer can generally be divided into several channel outputs, which has a certain degree of decoupling.

[0042] Step S14: Train K sub-task models based on K sets of features. The loss function of the output features of the K sub-task models and the K sets of features is less than a first preset threshold.

[0043] Figure 3 yes Figure 1 A flowchart of an embodiment of step S14 is shown below. Figure 3 As shown, training K sub-task models based on K sets of features includes:

[0044] Step S141: Construct K untrained initial sub-task models. Optionally, the initial sub-task models are MobileNet v3 models. The MobileNet v3 model is a classic lightweight convolutional model, characterized by few parameters, low computational cost, and short inference time.

[0045] Step S142: Take each of the K groups of features as the target output of an initial subtask model.

[0046] Step S143: Input the training data into the initial subtask model to obtain the actual output results;

[0047] Step S144: Iteratively train the initial subtask model based on the sub-loss function of the actual output result and the target output result to obtain the trained subtask model.

[0048] The following combination Figure 4 Steps S142 to S144 will be explained. Figure 4 This is a schematic diagram illustrating the training of K sub-task models based on K sets of features according to an embodiment of the present invention. Figure 4 As shown, the first sub-task model is trained based on the first set of features of the first neural network model, and so on, until the Kth sub-task model is trained based on the Kth set of features. For example, the first set of features is used as the learning object of the first sub-task model, that is, the first set of features is the target output of the first sub-task model. The second set of features is used as the target output of the second sub-task model, and so on, with the Kth set of features used as the target output of the Kth sub-task model.

[0049] The training data is input into each initial subtask model to obtain the actual output of each initial subtask model. For example, inputting the training data into the first subtask model yields the actual output of the first subtask model, and inputting the training data into the Kth subtask model yields the actual output of the Kth subtask model.

[0050] Calculate the sub-loss function based on the actual output and the target output, and then calculate the loss function between the output features of the K sub-task models and the K sets of features based on the sub-loss function:

[0051] L f =L2((f t1 ), (f s1 ))+L2((f t2 ), (f s2 ))+…+L2((f tk ), (f sk ))

[0052] Where L f Let L2((f) be the loss function. t1 ), (f s1 L2((f) is the sub-loss function of the first sub-task model. t2 ), (f s2 L2((f) is the sub-loss function of the second sub-task model. tk ), (f sk ()) is the sub-loss function of the Kth sub-task model, f t1 f is the target output of the first subtask model. s1 f represents the actual output of the first subtask model. t2 f is the target output of the second subtask model. s2 f represents the actual output of the second subtask model. tk f is the target output of the Kth subtask model. sk L2 represents the actual output of the Kth subtask model, and L2 represents the distance loss.

[0053] Judgment loss function L f If the loss function is less than the first preset threshold, then adjust the parameters of the K sub-task models and iteratively train the sub-task models until the loss function is less than the first preset threshold, thus obtaining the trained K sub-task models.

[0054] Step S15: Replace the network before the output layer in the first neural network model with K sub-task models to obtain the compressed second neural network model.

[0055] Figure 5This is a schematic diagram of the structure of a second neural network model according to an embodiment of the present invention. Figure 5 As shown, the second neural network model 500 includes an output layer and K sub-task models. The K sub-task models are designated as the first sub-task model…the Kth sub-task model. The first sub-task model simulates and outputs the first set of features of the first neural network model…the Kth sub-task model simulates and outputs the Kth set of features of the first neural network model. Thus, the first neural network model, which has a large number of parameters and high computational cost, has been compressed, resulting in a lightweight second neural network model.

[0056] Figure 6 yes Figure 1 A flowchart illustrating the compression method for the neural network model in the optimized embodiment. (See attached flowchart.) Figure 6 As shown, the neural network model compression method 600 also includes:

[0057] Step S16: Determine whether the computational cost of each of the K sub-task models is less than the second preset threshold. If not, increase the value of K and proceed to step S13 to reconstruct the second neural network model.

[0058] In practical applications, each sub-task model is deployed on an edge device. Edge devices have limited computing and storage resources, so a second preset threshold can be determined based on these resources. When the computational cost of a sub-task model is below the second preset threshold, the edge device can load the sub-task model and its framework, fully inputting data and obtaining the expected output. Conversely, when the computational cost of a sub-task model is equal to or greater than the second preset threshold, the edge device struggles to execute it successfully. Therefore, before deploying the neural network model on the edge device, it's necessary to calculate whether the computational cost of each of the K sub-task models is less than the preset second threshold. If not, the value of K needs to be increased, i.e., the number of sub-task models needs to be increased to reduce the computational cost of each model. Based on the increased value of K, the tail features are re-splitted, and the second neural network model is reconstructed.

[0059] In some embodiments, the method further includes determining whether the computation time of each of the K sub-task models is less than a preset third preset threshold. If not, the value of K is increased, and the second neural network model is reconstructed. In practical applications, users also have requirements for the computation time of the sub-task models. For example, they may want to obtain the output result within 0.1 seconds. Otherwise, if the computation time is too long and feedback cannot be obtained for a long time, it will reduce the efficiency of the second neural network model. In this example, while meeting the computational requirements, it can be determined whether the computation time of each of the K sub-task models is less than a preset third preset threshold. If not, the value of K is increased, that is, the number of sub-task models is increased, so as to shorten the computation time of each sub-task model. Based on the increased value of K, the tail features are re-splitted, and the second neural network model is reconstructed.

[0060] In some embodiments, the method further includes inputting training data into a second neural network model to obtain a second output result; and fine-tuning the parameters of the second neural network model based on the first and second output results to obtain an optimized third neural network model. For example, calculating the result loss function based on the first and second output results, and fine-tuning the parameters of the second neural network model according to the criterion of minimizing the result loss function to obtain the optimized third neural network model.

[0061] The neural network model compression method of this invention reduces the parameters and computational load of the neural network model by splitting the tail features of the first neural network model into K groups of features, training K lightweight sub-task models based on the K groups of features, and replacing the original feature extraction network with the K sub-task models. It has lower requirements for computing resources and memory, and can run widely on resource-constrained edge devices.

[0062] This application also includes a compression device for a neural network model, comprising a memory and a processor. The memory stores instructions executable by the processor; the processor executes these instructions to implement the aforementioned compression method for the neural network model.

[0063] Figure 7 This is a system block diagram of a neural network model compression device according to an embodiment of this application. (Reference) Figure 7As shown, the compression device 700 for the neural network model may include an internal communication bus 701, a processor 702, a read-only memory (ROM) 703, a random access memory (RAM) 704, and a communication port 705. When applied to a personal computer, the compression device 700 for the neural network model may also include a hard disk 706. The internal communication bus 701 enables data communication between the components of the compression device 700 for the neural network model. The processor 702 can make judgments and issue prompts. In some embodiments, the processor 702 may consist of one or more processors. The communication port 705 enables data communication between the compression device 700 for the neural network model and external devices. In some embodiments, the compression device 700 for the neural network model can send and receive information and data from a network through the communication port 705. The compression device 700 for the neural network model may also include different forms of program storage units and data storage units, such as the hard disk 706, the read-only memory (ROM) 703, and the random access memory (RAM) 704, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor 702. The processor executes these instructions to implement the main part of the method. The results of the processor's processing are transmitted to the user device via the communication port and displayed on the user interface.

[0064] The above-described operation method can be implemented as a computer program, stored in the hard disk 706, and loaded into the processor 702 for execution to implement the neural network model compression method of this application.

[0065] This application also includes a computer-readable medium storing computer program code that, when executed by a processor, implements the compression method of the aforementioned neural network model.

[0066] When a neural network model compression method is implemented as a computer program, it can also be stored as an article of manufacture in a computer-readable storage medium. For example, computer-readable storage media can include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical discs (e.g., compact discs (CDs), digital multifunction discs (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), cards, sticks, key drives). Furthermore, the various storage media described herein can represent one or more devices and / or other machine-readable media used for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media (and / or storage media) capable of storing, containing, and / or carrying code and / or instructions and / or data.

[0067] Figure 8 This is a flowchart of a method for deploying a neural network model according to an embodiment of the present invention. Figure 8As shown, the deployment method 800 for the neural network model includes the following steps:

[0068] Step S81: Obtain the compressed second neural network model using a neural network model compression method;

[0069] Step S82: Deploy each subtask model in the second neural network model on an edge device;

[0070] Step S83: Deploy the output layer of the second neural network model on the main control device.

[0071] The present invention also provides a deployment system for a neural network model, comprising: multiple edge devices, each edge device having a sub-task model of a second neural network model deployed thereon, the number of edge devices being equal to the number of the sub-task models; and a master control device having an output layer of the second neural network model deployed thereon. Figure 9 This is a block diagram of a deployment system for a neural network model according to an embodiment of the present invention. Figure 9 As shown, device 1 is the master control device, device 2, device 3, and device K are slave devices. Each slave device deploys a sub-task model, while the master control device deploys the output layer of the second neural network model. Slave devices can be edge devices. Taking face recognition as an example, the master control device receives data input (a user image). The master control device distributes the user image to each slave device via local area network routing. Each slave device returns one image feature at layer N-1. The master control device waits for and aggregates all N-1 layer image features from all slave devices. The master control device inputs all N-1 layer image features into the output layer to obtain the final output result of the model, which can indicate whether the face recognition passes or fails.

[0072] The deployment method and system of the neural network model of the present invention have very little data interaction between edge devices and the computing paths between devices are independent of each other. Therefore, even if there are performance differences between these devices, a single device will not slow down due to the computing bottleneck of other devices, thereby maximizing the computing efficiency of the entire neural network model.

[0073] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or combinations thereof.

[0074] Some aspects of this application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a "data block," "module," "engine," "unit," "component," or "system." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. Furthermore, aspects of this application may manifest as computer products residing in one or more computer-readable media, including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical discs (e.g., compressed CDs, digital multifunction DVDs, etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0075] A computer-readable medium may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and so on, or suitable combinations thereof. A computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signals, or similar media, or any combination of the above media.

[0076] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

Claims

1. A method for compressing a neural network model, characterized in that, include: Obtain the trained first neural network model; The face image is input into the first neural network model to obtain the first output result and the tail feature. The tail feature is the feature of the last convolutional layer of the first neural network model. The first output result is whether the face recognition is successful. The tail feature is the face feature corresponding to the face image. The tail feature is split into K groups of features, where K is an integer greater than 1; K sub-task models are trained based on the K sets of features. Each sub-task model is deployed on an edge device. It is determined whether the computational cost of each sub-task model is less than a second preset threshold. If so, the sub-task models are iteratively trained until the loss function of the output features of the K sub-task models and the K sets of features is less than a first preset threshold. The second preset threshold is determined based on the computing and storage resources of the edge device. The K sub-task models are used to replace the network before the output layer in the first neural network model to obtain the compressed second neural network model.

2. The method as described in claim 1, characterized in that, Training K sub-task models based on the K sets of features includes: Construct K untrained initial sub-task models; Each of the K groups of features is used as the target output of the initial subtask model; The face image is input into the initial subtask model to obtain the actual output result; The initial subtask model is iteratively trained based on the sub-loss function of the actual output result and the target output result to obtain the trained subtask model.

3. The method as described in claim 2, characterized in that, The initial subtask model is the Mobilenet v3 model.

4. The method as described in claim 1, characterized in that, Also includes: If the computational cost of each of the K sub-task models is greater than or equal to the second preset threshold, increase the value of K and reconstruct the second neural network model.

5. The method as described in claim 1, characterized in that, Also includes: Determine whether the computation time of each of the K sub-task models is less than a preset third threshold. If not, increase the value of K and reconstruct the second neural network model.

6. The method as described in claim 1, characterized in that, Also includes: The face image is input into the second neural network model to obtain the second output result; Based on the first and second output results, the parameters of the second neural network model are fine-tuned to obtain the optimized third neural network model.

7. The method according to any one of claims 1 to 6, characterized in that, The loss function includes distance loss.

8. A compression device for a neural network model, characterized in that, include: Memory is used to store instructions that can be executed by the processor; A processor for executing the instructions to implement the method as described in any one of claims 1-7.

9. A computer-readable medium storing computer program code that, when executed by a processor, implements the method as claimed in any one of claims 1-7.

10. A method for deploying a neural network model, characterized in that, include: The compressed second neural network model is obtained by using the method described in any one of claims 1 to 7; Each sub-task model in the second neural network model is deployed on an edge device; The output layer of the second neural network model is deployed on the main control device.

11. A deployment system for a neural network model, characterized in that, include: Multiple edge devices, each edge device deploying a sub-task model of the second neural network model obtained by the method as described in any one of claims 1 to 7, wherein the number of multiple edge devices is equal to the number of the sub-task models; The main control device has the output layer of the second neural network model deployed on it.

Images