Optimization processing method and device for neural network model

Abstract

The present disclosure provides an optimization processing method and device for a neural network model. The method includes determining at least one target optimization layer of the neural network model based on the number of neurons in each layer of the neural network model; for each target optimization layer in the at least one target optimization layer, adding a virtual layer between the target optimization layer and the upper layer of the target optimization layer, the neurons of the virtual layer are respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer; training the neural network model after adding the virtual layer to obtain an optimized neural network model. The device includes a target optimization layer determination unit, a virtual layer adding unit and a model training unit. Based on a lossless compression method that is mathematically proven to be equivalent, the neural network model is effectively compressed, the number of parameters of the model during training and use is reduced, and storage space is saved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computer technology, and more specifically, to an optimization processing method and apparatus for a neural network model. Background Art

[0002] Neural network models, such as Deep Neural Networks (DNN) models, are widely used in many fields due to their strong ability to fit data. Generally speaking, the fitting ability of DNN is positively correlated with the feature dimension of its input, the number of hidden layer neurons, and the number of hidden layers, which also means that storing a DNN model requires a lot of memory units.

[0003] Take a 7-layer DNN model as an example. Assuming that the number of neurons in each layer is [1000, 2000, 1000, 500, 100, 50, 1] (assuming that the first layer is the input layer and the last layer is the output layer), the required model parameters are 1000*2000+2000*1000+1000*500+500*100+100*50+50*1=4555050. The space for storing the model parameters is 4555050*4=18220200, which is about 18M of storage space. In fact, in the field of image and speech, the required neuron width and depth are usually ten or a hundred times that in the example, which also means that the online system needs more memory units to store the model. Therefore, how to effectively compress the size of the model is particularly important, especially when the memory units are generally insufficient on mobile devices.

[0004] The existing model compression method mainly replaces the numerical value of the model parameter with a smaller data unit for storage. For example, the weight of the neural network model was originally stored as a 4-byte floating point number. This compression method compresses the floating point number to 2 bytes. The problem with this compression method is that the 2-byte floating point number does not have as large a range and high precision as the 4-byte floating point number, so the performance of the model will be affected. Summary of the invention

[0005] In view of the above problems, the present disclosure provides a method and device for optimizing a neural network model. The method and device are used to compress the neural network model with little loss in model prediction performance.

[0006] According to one aspect of the present disclosure, there is provided an optimization processing method for a neural network model, comprising: determining at least one target optimization layer of the neural network model based on the number of neurons in each layer in the neural network model; for each target optimization layer in the at least one target optimization layer, adding a virtual layer between the target optimization layer and an upper layer of the target optimization layer, the neurons of the virtual layer being respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer; and training the neural network model after adding the virtual layer to obtain an optimized neural network model.

[0007] Optionally, in an example of the above aspect, based on the number of neurons in each layer in the neural network model, determining at least one target optimization layer of the neural network model includes: for each layer in the neural network model, calculating the product of the number of neurons in the layer and the number of neurons in the upper layer; and determining the layer for which the calculated product is greater than a preset threshold as the target optimization layer.

[0008] Optionally, in an example of the above aspect, the optimization processing method of the neural network model further includes: determining the number of neurons of the added virtual layer for each target optimization layer in the at least one target optimization layer.

[0009] Optionally, in an example of the above aspect, the number of neurons in the virtual layer is determined based on the number of neurons in the target optimization layer, the number of neurons in an upper layer of the target optimization layer, and the compression ratio adopted.

[0010] Optionally, in an example of the above aspect, the compression ratio adopted includes multiple predetermined compression ratios, and an optimized neural network model is obtained for each compression ratio of the multiple predetermined compression ratios, and the method also includes: selecting a neural network model with a smaller compression ratio and smaller prediction performance loss from the multiple optimized neural network models as the final optimized neural network model.

[0011] Optionally, in an example of the above aspect, in order to select a neural network model with a smaller prediction performance loss, arranging multiple optimized neural network models according to the prediction performance loss from small to large includes: inputting the same data into an unoptimized neural network model and each optimized neural network model for prediction; comparing the absolute value of the difference between each optimized prediction result and a benchmark prediction result and arranging them in order from small to large, wherein the optimized prediction result is predicted using the optimized neural network model, and the benchmark prediction result is predicted using the unoptimized neural network model; and arranging each optimized neural network model in the arrangement order of the corresponding optimized prediction results, so as to obtain multiple optimized neural network models arranged from small to large according to the prediction performance loss.

[0012] Optionally, in an example of the above aspect, before adding the virtual layer, the connection between the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer is fully connected; after adding the virtual layer, the connection between the neurons of the virtual layer and the neurons of the target optimization layer is fully connected, and the connection between the neurons of the virtual layer and the neurons of the upper layer of the target optimization layer is fully connected.

[0013] According to another aspect of the present disclosure, there is also provided an optimization processing device for a neural network model, comprising: a target optimization layer determination unit, configured to determine at least one target optimization layer of the neural network model based on the number of neurons in each layer in the neural network model; a virtual layer adding unit, configured to add a virtual layer between the target optimization layer and the upper layer of the target optimization layer for each target optimization layer in the at least one target optimization layer, the neurons of the virtual layer being respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer; and a model training unit, configured to train the neural network model after the virtual layer is added to obtain an optimized neural network model.

[0014] Optionally, in an example of the above aspect, the target optimization layer determination unit includes: a calculation module, configured to calculate, for each layer in the neural network model, the product of the number of neurons in the layer and the number of neurons in the upper layer; and a target optimization layer determination module, configured to determine the layer for which the calculated product is greater than a preset threshold as the target optimization layer.

[0015] Optionally, in an example of the above aspect, the optimization processing device of the neural network model also includes: a neuron number determination unit, configured to determine the number of neurons of the added virtual layer for each target optimization layer in the at least one target optimization layer.

[0016] Optionally, in an example of the above aspect, the number of neurons in the virtual layer is determined based on the number of neurons in the target optimization layer, the number of neurons in an upper layer of the target optimization layer, and the compression ratio adopted.

[0017] Optionally, in an example of the above aspect, the compression ratio adopted includes multiple predetermined compression ratios, and an optimized neural network model is obtained for each compression ratio of the multiple predetermined compression ratios, and the device includes: a model selection unit, configured to select a neural network model with a smaller compression ratio and smaller prediction performance loss from the multiple optimized neural network models as the final optimized neural network model.

[0018] Optionally, in an example of the above aspect, the model selection unit includes: a prediction module, configured to input the same data into an unoptimized neural network model and each optimized neural network model for prediction; a prediction result comparison module, configured to compare the absolute value of the difference between each optimized prediction result and a benchmark prediction result and arrange them in order from small to large, wherein the optimized prediction result is predicted using the optimized neural network model, and the benchmark prediction result is predicted using the unoptimized neural network model; and a model arrangement module, configured to arrange each optimized neural network model in the arrangement order of the corresponding optimized prediction results, so as to obtain a plurality of optimized neural network models arranged in order from small to large according to the prediction performance loss.

[0019] According to another aspect of the present disclosure, there is also provided a computing device, comprising: at least one processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the optimization processing method of the neural network model as described above.

[0020] According to another aspect of the present disclosure, a non-transitory machine-readable storage medium is also provided, which stores executable instructions, and when the instructions are executed, the machine performs the optimization processing method of the neural network model as described above.

[0021] Based on the mathematically proven equivalent lossless compression method, the optimization processing method and device of the neural network model disclosed in the present invention can effectively compress the size of the neural network model, reduce the number of parameters of the neural network model during training and use, save storage space, reduce the amount of calculation, and at the same time, the model compression causes little loss in model prediction performance. Due to the reduction in the number of parameters, the neural network model improves its robustness when subjected to noise interference while achieving the same prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A further understanding of the nature and advantages of the present disclosure may be achieved by referring to the following drawings.In the accompanying drawings, similar components or features may have the same reference numeral.

[0023] Figure 1 A schematic diagram of the structure of an example of a neural network model;

[0024] Figure 2 is a flowchart of an optimization processing method of a neural network model according to an embodiment of the present disclosure;

[0025] Figure 3 A flowchart of a method for arranging multiple optimized neural network models according to an example in ascending order of prediction performance loss;

[0026] Figure 4A A schematic diagram of the structure of an example of an unoptimized neural network model;

[0027] Figure 4B for Figure 4A The schematic diagram of the optimized structure of the neural network model shown in ;

[0028] Figure 5 Schematic diagram of the structure of a neural network model optimization processing device 500 according to an embodiment of the present disclosure;

[0029] Figure 6 is a structural schematic diagram of an optimization processing device 600 for a neural network model according to another embodiment of the present disclosure;

[0030] Figure 7 The present invention is a block diagram of a computing device for optimizing processing of a neural network model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and is not a limitation of the scope of protection, applicability or examples set forth in the claims. The function and arrangement of the elements discussed can be changed without departing from the scope of protection of the present disclosure. Various examples can omit, replace or add various processes or components as needed. For example, the described method can be performed in an order different from the described order, and various steps can be added, omitted or combined. In addition, the features described relative to some examples can also be combined in other examples.

[0032] As used herein, the term "including" and its variations represent open terms, meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0033] The neural network model is a highly nonlinear dynamic system. A large number of neurons of the same form are connected together to form a neural network model. These neurons are associated in layers. Although the structure and function of each neuron are not complicated, the dynamic behavior of the neural network model is very complex. Therefore, the neural network model can be used to express various phenomena in the actual physical world.

[0034] Figure 1 A structural diagram of an example of a neural network model. Figure 1 Each node in is a neuron. The neural network model includes an input layer on the input side, an output layer on the output side, and an intermediate layer between the input layer and the output layer. Each layer takes the previous layer as input, and the output result is passed to the next layer. x1, x2, and x3 are input values. For the output of the second layer and have:

[0035]

[0036]

[0037]

[0038] For the output of the third layer have:

[0039]

[0040] The above neural network model starts from the output h of the i-th layer i Output h to layer i+1 i+1 The propagation process can be expressed in matrix form as follows:

[0041] f(h i+1 )=w i+1 f(h i )+b (1)

[0042] Among them, w i+1 is the weight matrix of the i+1th layer, whose dimension is Di ×D i+1 , D i is the number of neurons in the i-th layer, D i+1 is the number of neurons in the i+1th layer. It can be seen that the weights of each neuron in the i+1th layer correspond to the connections between the neuron in the i+1th layer and at least one neuron in the ith layer.

[0043] Figure 2 It is a flowchart of a method for optimizing a neural network model according to an embodiment of the present disclosure.

[0044] like Figure 2 As shown, in block 210, at least one target optimization layer of the neural network model is determined based on the number of neurons in each layer in the neural network model.

[0045] For each layer in the neural network model, the product of the number of neurons N in the layer and the number of neurons M in the upper layer is calculated, and the layer whose calculated product M*N is greater than a preset threshold, such as 100,000, is determined as the target optimization layer. One or more layers in a neural network model can be determined as the target optimization layer.

[0046] In block 220, for each target optimization layer in at least one target optimization layer, a virtual layer is added between the target optimization layer and the upper layer of the target optimization layer, and the neurons of the virtual layer are respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer. As one aspect of this embodiment, a layer in the neural network model, such as the third layer, can be determined as the target optimization layer, and then a virtual layer is added between the third layer and the second layer of the neural network model. As another aspect of this embodiment, multiple layers in the neural network model, such as the fourth layer and the eighth layer, can be determined as the target optimization layers, and then a virtual layer is added between the fourth layer and the third layer of the neural network model, and a virtual layer is added between the eighth layer and the seventh layer.

[0047] Fully connected means that every neuron in one layer is connected to all neurons in the other layer. Every neuron in the fully connected layer is connected to all neurons in the previous layer, for example Figure 1 As shown in the second layer, the fully connected layer has the most weights compared to the layers with other connection forms. Before adding the virtual layer, the connection between the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer can be fully connected. After adding the virtual layer, the connection between the neurons of the virtual layer and the neurons of the target optimization layer can be fully connected, and the connection between the neurons of the virtual layer and the neurons of the upper layer of the target optimization layer can be fully connected.

[0048] As an optional implementation, the number of neurons in the added virtual layer is directly set to a predetermined number.

[0049] As another optional implementation, for each target optimization layer in at least one target optimization layer, the number of neurons in the virtual layer added is determined so that the number of neurons in the virtual layer added is

[0050] Here, N is the number of neurons in the target optimization layer, and M is the number of neurons in the layer above the target optimization layer. Further, the number of neurons in the virtual layer is determined based on the number of neurons in the target optimization layer, the number of neurons in the layer above the target optimization layer, and the compression ratio used. For example, the number of neurons Z in the virtual layer can be determined by the following formula:

[0051]

[0052] Here, x is the compression ratio adopted, N is the number of neurons in the target optimization layer, and M is the number of neurons in the layer above the target optimization layer.

[0053] In block 230, the neural network model with the virtual layer added is trained to obtain an optimized neural network model. After training the neural network model with the virtual layer added, the weights corresponding to the connections between each neuron in each layer of the neural network model and at least one neuron in the previous layer are obtained.

[0054] The compression ratio x used may include multiple predetermined compression ratios such as 10%, 20%, and 30%, etc., and an optimized neural network model is obtained for each of the multiple predetermined compression ratios. Based on the criterion of smaller prediction performance loss and the criterion of smaller compression ratio, the final optimized neural network model is selected from the multiple optimized neural network models. Therefore, the multiple optimized neural network models are arranged from small to large according to the prediction performance loss, so as to select a neural network model with smaller prediction performance loss from the multiple optimized neural network models. The following will illustrate the method of arranging multiple optimized neural network models from small to large according to the prediction performance loss by example.

[0055] Figure 3 A flowchart of a method for arranging multiple optimized neural network models from small to large according to prediction performance loss based on an example.

[0056] like Figure 3 As shown, in block 310, the same data is input into the unoptimized neural network model and each optimized neural network model for prediction.

[0057] In block 320, the absolute values ​​of the differences between the optimized prediction results and the benchmark prediction results are compared and arranged in ascending order, wherein the optimized prediction results are predicted using the optimized neural network model and the benchmark prediction results are predicted using the unoptimized neural network model.

[0058] In block 330, the optimized neural network models are arranged in the order of the corresponding optimization prediction results, so as to obtain a plurality of optimized neural network models arranged from small to large in terms of prediction performance loss.

[0059] Figure 4A A structural diagram of an example of an unoptimized neural network model. Figure 4B for Figure 4A The schematic diagram of the optimized structure of the neural network model is shown in the figure, where the dotted nodes represent the added Figure 4A Neurons in the virtual layer between layer 3 and layer 2 of the neural network model shown.

[0060] Figure 4A The output matrix f(h3) of the third layer in the unoptimized neural network model shown is as follows:

[0061] f(h3)=w3f(h2)+b (2)

[0062] Among them, w3 is the weight matrix of the third layer, and its dimension is D2×D3, D2 is the number of neurons in the second layer, and D3 is the number of neurons in the third layer.

[0063] Assuming that the dimension of the weight matrix w3 is M×N, the matrix w3 can be expressed as follows:

[0064] w M×N =a M×Z b Z×N (3)

[0065] The matrix w3 contains M×N values. After being decomposed into two matrices a3 and b3, it contains a total of M×Z+Z×N values. <M×N。

[0066] therefore, Figure 4B The output matrix f(h3) of the third layer in the optimized neural network model shown can be expressed as follows:

[0067] f(h3)=a3b3f(h2)+b (4)

[0068] The above mathematically proves that the weight matrices before and after compression are equivalent, so in theory matrix decomposition will not affect the prediction performance of the model.

[0069] Based on the mathematically proven equivalent lossless compression method, the optimization processing method of the neural network model in the above embodiment can effectively compress the size of the neural network model, reduce the number of parameters of the neural network model during training and use, save storage space, reduce the amount of calculation, and at the same time, the model compression causes little loss in model prediction performance. Due to the reduction in the number of parameters, the neural network model improves its robustness when subjected to noise interference while achieving the same prediction accuracy.

[0070] Figure 5 FIG. 5 is a schematic diagram of a structure of an optimization processing device 500 for a neural network model according to an embodiment of the present disclosure. Figure 5 As shown, the optimization processing device 500 of the neural network model of this embodiment includes a target optimization layer determination unit 510, a virtual layer adding unit 520 and a model training unit 530.

[0071] The target optimization layer determination unit 510 is configured to determine at least one target optimization layer of the neural network model based on the number of neurons in each layer of the neural network model. The operation of the target optimization layer determination unit 510 can refer to the above reference Figure 2 The operation of block 210 described above. The target optimization layer determination unit 510 may include a calculation module and a target optimization layer determination module. The calculation module is configured to calculate the product of the number of neurons in the layer and the number of neurons in the upper layer for each layer in the neural network model. The target optimization layer determination module is configured to determine the layer whose product of the number of neurons in the layer calculated by the calculation module and the number of neurons in the upper layer is greater than a preset threshold value, such as 1000,000, as the target optimization layer.

[0072] The virtual layer adding unit 520 is configured to add a virtual layer between each target optimization layer in at least one target optimization layer and an upper layer of the target optimization layer, wherein the neurons of the virtual layer are respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer. The operation of the virtual layer adding unit 520 can refer to the above reference Figure 2 The operation of block 220 is described.

[0073] The model training unit 530 is configured to train the neural network model after adding the virtual layer to obtain an optimized neural network model. The operation of the model training unit 530 can refer to the above reference Figure 2 The operation of block 230 is described.

[0074] Figure 6 FIG. 6 is a schematic diagram of a structure of an optimization processing device 600 for a neural network model according to another embodiment of the present disclosure. Figure 6As shown, the optimization processing device 600 of the neural network model of this embodiment includes: a target optimization layer determination unit 610, a virtual layer adding unit 620, a model training unit 630 and a neuron number determination unit 640.

[0075] The target optimization layer determination unit 610 is configured to determine at least one target optimization layer of the neural network model based on the number of neurons in each layer of the neural network model. The operation of the target optimization layer determination unit 610 can refer to the above reference Figure 2 The operation of block 210 is described.

[0076] The virtual layer adding unit 620 is configured to add a virtual layer between each target optimization layer in at least one target optimization layer and an upper layer of the target optimization layer, wherein the neurons of the virtual layer are respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer. The operation of the virtual layer adding unit 620 can refer to the above reference Figure 2 The operation of block 220 is described.

[0077] The model training unit 630 is configured to train the neural network model after adding the virtual layer to obtain an optimized neural network model. The operation of the model training unit 630 can refer to the above reference Figure 2 The operation of block 230 is described.

[0078] The neuron number determination unit 640 is configured to determine the number of neurons of the added virtual layer for each target optimization layer in at least one target optimization layer. Further, the number of neurons of the virtual layer can be determined based on the number of neurons of the target optimization layer, the number of neurons of the upper layer of the target optimization layer, and the compression ratio adopted. The operation of the neuron number determination unit 640 can refer to the above reference Figure 2 The operation of block 220 is described.

[0079] The compression ratio used may include multiple predetermined compression ratios such as 10%, 20%, 30%, and 40%, etc., and an optimized neural network model is obtained for each of the multiple predetermined compression ratios. The neural network model optimization processing device 600 of this embodiment may also include a model selection unit, which is configured to select a neural network model with a smaller compression ratio and a smaller prediction performance loss from the multiple optimized neural network models as the final optimized neural network model.

[0080] Furthermore, the model selection unit may include a prediction module, a prediction result comparison module, and a model arrangement module. The prediction module is configured to input the same data into the unoptimized neural network model and each optimized neural network model for prediction. The operation of the prediction module can refer to the above reference Figure 3 The prediction result comparison module is configured to compare the absolute values ​​of the differences between each optimized prediction result and the benchmark prediction result and arrange them in ascending order, wherein the optimized prediction result is predicted by the optimized neural network model and the benchmark prediction result is predicted by the unoptimized neural network model. The operation of the prediction result comparison module can refer to the above reference Figure 3 The operation of block 320 described above. The model arrangement module is configured to arrange each optimized neural network model according to the arrangement order of the corresponding optimization prediction results, and obtain multiple optimized neural network models arranged in order of prediction performance loss from small to large. The operation of the model arrangement module can refer to the above reference Figure 3 The operation of block 330 is described.

[0081] Figure 7 The present invention is a block diagram of a computing device for optimizing processing of a neural network model according to an embodiment of the present invention.

[0082] like Figure 7 As shown, the computing device 700 may include at least one processor 710, a storage 720, a memory 730, a communication interface 740, and an internal bus 750, wherein the at least one processor 710 executes at least one computer-readable instruction (i.e., the above-mentioned elements implemented in the form of software) stored or encoded in a computer-readable storage medium (i.e., the memory 720).

[0083] In one embodiment, computer executable instructions are stored in the memory 720, which when executed enable at least one processor 710 to: determine at least one target optimization layer of the neural network model based on the number of neurons in each layer in the neural network model; for each target optimization layer in the at least one target optimization layer, add a virtual layer between the target optimization layer and the upper layer of the target optimization layer, the neurons of the virtual layer are respectively connected to the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, wherein the addition of the virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer; and train the neural network model after the virtual layer is added to obtain an optimized neural network model.

[0084] It should be understood that the computer executable instructions stored in the memory 720, when executed, cause the at least one processor 710 to perform the above combined operations in various embodiments of the present disclosure. Figure 1-6 Describes the various operations and functions.

[0085] In the present disclosure, the computing device 700 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, a consumer electronic device, and the like.

[0086] According to one embodiment, a program product such as a non-transitory machine-readable medium is provided. The non-transitory machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of the present disclosure. Figure 1-6 Describes the various operations and functions.

[0087] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0088] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above-mentioned embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute a part of the present invention.

[0089] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0090] Those skilled in the art should understand that the various embodiments disclosed above can be modified and altered in various ways without departing from the essence of the invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

[0091] It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0092] In the above embodiments, the hardware unit or module can be realized by mechanical or electrical means. For example, a hardware unit, module or processor can include permanent dedicated circuit or logic (such as special processor, FPGA or ASIC) to complete the corresponding operation. The hardware unit or processor can also include programmable logic or circuit (such as general-purpose processor or other programmable processor), which can be temporarily set by software to complete the corresponding operation. Specific implementation (mechanical method or dedicated permanent circuit or temporary circuit) can be determined based on cost and time consideration.

[0093] The specific embodiments described above in conjunction with the accompanying drawings describe exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of protection of the claims. The term "exemplary" used throughout this specification means "used as an example, instance or illustration" and does not mean "preferred" or "having advantages" over other embodiments. For the purpose of providing an understanding of the described technology, the specific embodiments include specific details. However, these technologies can be implemented without these specific details. In some instances, in order to avoid making the concepts of the described embodiments difficult to understand, well-known structures and devices are shown in block diagram form.

[0094] The above description of the present disclosure is provided to enable any person of ordinary skill in the art to implement or use the present disclosure. Various modifications to the present disclosure will be apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but is consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A model compression method for a neural network model implemented by a computing device, wherein the neural network model includes a neural network model applied to image or speech processing, and the model compression method includes: Determining at least one target optimization layer of the neural network model based on the number of neurons in each layer of the neural network model; For each target optimization layer in the at least one target optimization layer, a virtual layer is added between the target optimization layer and an upper layer of the target optimization layer, the neurons of the virtual layer are fully connected with the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer, respectively, wherein the number of neurons of the added virtual layer reduces the number of connections between the target optimization layer and the upper layer of the target optimization layer; Training the neural network model after adding the virtual layer to obtain an optimized neural network model; and The optimized neural network model is stored in the mobile device.

2. The model compression method according to claim 1, wherein: Determining at least one target optimization layer of the neural network model based on the number of neurons in each layer of the neural network model includes: For each layer in the neural network model, calculate the product of the number of neurons in the layer and the number of neurons in the upper layer; and The layer whose calculated product is greater than a preset threshold is determined as the target optimization layer.

3. The model compression method according to claim 1, further comprising: For each target optimization layer in the at least one target optimization layer, the number of neurons in the added virtual layer is determined.

4. The model compression method according to claim 3, wherein: The number of neurons in the virtual layer is determined based on the number of neurons in the target optimization layer, the number of neurons in the upper layer of the target optimization layer, and the adopted compression ratio.

5. The model compression method according to claim 4, wherein: The compression ratio used includes a plurality of predetermined compression ratios, and an optimized neural network model is obtained for each compression ratio of the plurality of predetermined compression ratios. The method further includes: A neural network model with a smaller compression ratio and smaller prediction performance loss is selected from multiple optimized neural network models as the final optimized neural network model.

6. The model compression method according to claim 5, wherein: Selecting a neural network model with smaller prediction performance loss from multiple optimized neural network models includes: The same data is input into the unoptimized neural network model and each optimized neural network model for prediction; Comparing the absolute values ​​of the differences between each optimized prediction result and a benchmark prediction result and arranging them in ascending order, wherein the optimized prediction result is predicted using the optimized neural network model, and the benchmark prediction result is predicted using the unoptimized neural network model; and Arrange the optimized neural network models according to the arrangement order of the corresponding optimization prediction results, and obtain multiple optimized neural network models arranged from small to large in terms of prediction performance loss, so as to select a neural network model with smaller prediction performance loss from the multiple optimized neural network models.

7. The model compression method according to claim 1, wherein: Before adding the virtual layer, the connection between the neurons of the target optimization layer and the neurons of the upper layer of the target optimization layer is fully connected; After the virtual layer is added, the connection between the neurons of the virtual layer and the neurons of the target optimization layer is fully connected, and the connection between the neurons of the virtual layer and the neurons of the upper layer of the target optimization layer is fully connected.

8. A computing device for implementing model compression of a neural network model, comprising: at least one processor; as well as A memory storing instructions, which, when executed by the at least one processor, enable the at least one processor to perform the model compression method according to any one of claims 1 to 7.

9. A non-transitory machine-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the model compression method according to any one of claims 1 to 7.

Images