Model optimization method and device, electronic equipment and storage medium

By cutting and sharing the middle layer of large neural network models and fusing with low-rank decomposition layers, the problem of slow inference speed caused by the huge number of model parameters is solved, and the reduction of model size and complexity and accuracy are achieved.

CN119066600APending Publication Date: 2024-12-03BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310626358.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The huge number of parameters in large neural network models leads to slow inference speed, and existing tailoring and compression methods will lead to a decrease in model accuracy.

Method used

By cropping the intermediate layer in the model based on preset optimization rules, sharing parameters of the cropped model, and then fusing the shared intermediate layer with the low-rank decomposition layer to obtain the optimized model.

Benefits of technology

The model size is reduced, the model's expression ability and performance is improved, the model's size and complexity are reduced, while maintaining high accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066600A_ABST
    Figure CN119066600A_ABST
Patent Text Reader

Abstract

The invention discloses a model optimization method and device, electronic equipment and a storage medium, and the main technical scheme comprises the steps: firstly, cutting a middle layer in a model based on a preset optimization rule, and obtaining a cut model; secondly, carrying out parameter sharing on each middle layer in the cut model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is model parameter sharing among the intermediate layers; and finally, fusing the intermediate layer and the low-rank decomposition layer after sharing the parameters to obtain the optimized model. Wherein the low-rank hierarchical layer is a low-rank decomposition matrix. The model is cut through the preset optimization rule, the parameters of the middle layer in the cut model are further shared, the size of the model is reduced, and in order to enhance the expression ability of the model parameters, a low-rank decomposition layer is fused in the model, the model performance is improved, and the size and complexity of the model are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a model optimization method and device, an electronic device, and a storage medium. Background Art

[0002] With the development of deep learning technology, large models are increasingly used in fields such as natural language processing. However, these models have a large number of parameters and their size increases accordingly, which leads to higher and higher requirements on the hardware used during model training or use.

[0003] At present, in order to reduce the hardware requirements of the model, the neural network model is often pruned and compressed. For example, the neural network model is pruned and compressed by sparse training, that is, the network weights and scaling factors are jointly trained, and the neural network channels with small scaling factors are directly removed. However, this method will cause the model accuracy to drop seriously and cannot maintain the accuracy of the original model. Summary of the invention

[0004] The present disclosure provides a model optimization method, device, electronic device and storage medium, the main purpose of which is to solve the problem of slow reasoning speed caused by a large number of model parameters.

[0005] According to a first aspect of the present disclosure, a method for optimizing a model is provided, comprising:

[0006] Crop the middle layer in the model based on the preset optimization rules to obtain the cropped model;

[0007] Sharing parameters of the intermediate layers in the pruned model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing the model parameters between the intermediate layers;

[0008] The intermediate layer after sharing the parameters is fused with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

[0009] Optionally, the trimming of the intermediate layers in the model based on a preset optimization rule to obtain the trimmed model includes:

[0010] Based on the sample data, determine the model accuracy of the model before trimming;

[0011] Clipping the first intermediate layer of the model, and determining the model accuracy of the clipped model based on the sample data;

[0012] If the difference between the model accuracy of the cropped model and the model accuracy of the model before cropping is greater than or equal to a preset accuracy threshold, the cropping is canceled and the cropping of the second intermediate layer continues.

[0013] If the difference between the model accuracy of the pruned model and the model accuracy of the model before pruned is less than the pre-accuracy threshold, the second intermediate layer continues to be pruned.

[0014] Optionally, the trimming of the intermediate layers in the model based on a preset optimization rule to obtain the trimmed model further includes:

[0015] After the accuracy of the cropped model reaches the target accuracy, the cropping is stopped to obtain the cropped model;

[0016] Alternatively, after the dimension of the cropped model reaches a preset dimension threshold, cropping is stopped to obtain the cropped model.

[0017] Optionally, the sharing of parameters of the intermediate layers in the pruned model includes:

[0018] Grouping the intermediate layers based on the first preset number and the connection order of the intermediate layers, wherein there are at least two intermediate layers in one group;

[0019] Model parameters are shared for intermediate layers in the same group.

[0020] Optionally, the step of sharing parameters of the intermediate layers in the pruned model further includes:

[0021] The intermediate layers are randomly grouped, and model parameters are shared among the intermediate layers in the same group.

[0022] Optionally, the step of fusing the intermediate layer after sharing each parameter with the low-rank decomposition layer to obtain the optimized model further includes:

[0023] Decomposing the low-rank decomposition layer to obtain a first matrix and a second matrix;

[0024] The first matrix, the second matrix, and the linear layer in the middle layer are fused.

[0025] Optionally, after fusing the intermediate layer after each parameter is shared with the low-rank decomposition layer to obtain the optimized model, the method further includes:

[0026] The optimized model is trained according to a second preset amount of training data.

[0027] According to a second aspect of the present disclosure, there is provided a model optimization device, comprising:

[0028] A trimming unit, used for trimming the intermediate layers in the model based on a preset optimization rule to obtain the trimmed model;

[0029] A parameter sharing unit, used to share parameters of each intermediate layer in the pruned model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing model parameters between the intermediate layers;

[0030] A fusion unit is used to fuse the intermediate layer and the low-rank decomposition layer after the parameters are shared, respectively, to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

[0031] Optionally, the cutting unit includes:

[0032] A determination module, used for determining the model accuracy of the model before clipping based on sample data;

[0033] A clipping module, used for clipping the first intermediate layer of the model and determining the model accuracy of the clipped model based on the sample data;

[0034] The cropping module is also used to cancel the cropping when the model accuracy of the cropped model is less than or equal to a preset accuracy threshold, and continue to crop the second intermediate layer.

[0035] The cropping module is further configured to continue cropping the second intermediate layer when the model accuracy of the cropped model is greater than or equal to the pre-accuracy threshold.

[0036] Optionally, the clipping-based unit is further used for:

[0037] After the accuracy of the cropped model reaches the target accuracy, the cropping is stopped to obtain the cropped model;

[0038] Alternatively, after the dimension of the cropped model reaches a preset dimension threshold, cropping is stopped to obtain the cropped model.

[0039] Optionally, the parameter sharing unit includes:

[0040] A grouping module, used for grouping the intermediate layers based on a first preset number and a connection sequence relationship of the intermediate layers, wherein there are at least two intermediate layers in a group;

[0041] The parameter sharing module is used to share model parameters for the intermediate layers in the same group.

[0042] Optionally, the parameter sharing unit is further used for:

[0043] The intermediate layers are randomly grouped, and model parameters are shared among the intermediate layers in the same group.

[0044] Optionally, the fusion unit further includes:

[0045] A decomposition module, used for decomposing the low-rank decomposition layer to obtain a first matrix and a second matrix;

[0046] A fusion module is used to fuse the first matrix, the second matrix and the linear layer in the middle layer.

[0047] Optionally, the method further includes:

[0048] The training unit is used to fuse the intermediate layer and the low-rank decomposition layer after the parameters are shared in the fusion unit to obtain the optimized model, and then train the optimized model according to a second preset amount of training data.

[0049] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0050] at least one processor; and

[0051] a memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0053] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0054] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0055] The optimization method, device, electronic device and storage medium of the model provided by the present disclosure, the main technical scheme includes: first, the intermediate layers in the model are pruned based on preset optimization rules to obtain the pruned model; second, the parameters of each intermediate layer in the pruned model are shared; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is the sharing of model parameters between the intermediate layers; finally, the intermediate layers after the parameter sharing are respectively fused with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix. Compared with the related art, the embodiment of the present application prunes the model through preset optimization rules and further shares the parameters of the intermediate layers in the pruned model, thereby reducing the size of the model, and in order to enhance the expressiveness of the model parameters, the low-rank decomposition layer is fused in the model to improve the model performance and reduce the size and complexity of the model.

[0056] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0058] Figure 1 A schematic diagram of a flow chart of a model optimization method provided by an embodiment of the present disclosure;

[0059] Figure 2 A schematic diagram of a flow chart of a model optimization method provided by an embodiment of the present disclosure;

[0060] Figure 3 A schematic diagram of the structure of a model optimization device provided in an embodiment of the present disclosure;

[0061] Figure 4 A schematic diagram of the structure of a model optimization device provided in an embodiment of the present disclosure;

[0062] Figure 5 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0063] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0064] The following describes the model optimization method, device, electronic device and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0065] Figure 1 A flowchart of a model optimization method provided in an embodiment of the present disclosure.

[0066] like Figure 1 As shown, the method comprises the following steps:

[0067] Step 101, trimming the middle layer in the model based on preset optimization rules to obtain the trimmed model.

[0068] In one implementable method of an embodiment of the present application, the preset optimization rules are pre-configured, adaptive depth and dimensional cropping, wherein the dimension represents the amount of calculation, and the depth represents the number of intermediate layers; the model can generally be divided into three layers: input layer, intermediate layer, and output layer; the input layer and the output layer are used for data input and output and result output respectively, and the intermediate layer is used to extract features from the data.

[0069] In one implementable method of an embodiment of the present application, the user can set parameters according to the accuracy of the model or the size of the model quality, so that the model cropping is stopped after the accuracy of the model or the quality of the model reaches a threshold; specifically, the embodiment of the present application does not limit the setting of the preset value.

[0070] Step 102, sharing parameters of the intermediate layers in the pruned model; different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing the model parameters between the intermediate layers.

[0071] In one implementable method of the embodiment of the present application, parameter sharing is parameter sharing between each layer. For example, the model parameter corresponding to the middle layer 1 is a, the model parameter corresponding to the middle layer 2 is b, and the model parameter corresponding to the middle layer 3 is c. In the prior art, each middle layer can only use its own model parameters for calculation when performing feature extraction. After parameter sharing in the embodiment of the present application, the parameter model corresponding to the middle layer 1 is a, b, c, the parameter model corresponding to the middle layer 3 is a, b, c, and the parameter model corresponding to the middle layer 3 is a, b, c; that is, the sharing of parameters at each layer can reduce the number of model parameters and further improve the storage efficiency and the running speed during model training and actual use.

[0072] Step 103, respectively fuse the intermediate layer after sharing the parameters with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

[0073] In one implementable method of an embodiment of the present application, when extracting features of data in the intermediate layer after parameter sharing, the extracted features will be relatively similar. Therefore, the low-rank decomposition layer fusion technology is adopted to fuse the low-rank decomposition layer with the intermediate layer to improve the expressive power of the shared model parameters. For the specific low-rank decomposition layer fusion steps, please refer to the detailed implementation process of methods such as Lora in the prior art, and the embodiments of the present application will not be described one by one here.

[0074] The optimization method of the model provided by the present disclosure includes the following main technical solutions: first, the intermediate layers in the model are pruned based on preset optimization rules to obtain the pruned model; second, the parameters of the intermediate layers in the pruned model are shared; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is the sharing of model parameters between the intermediate layers; finally, the intermediate layers after the parameter sharing are respectively fused with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix. Compared with the related art, the embodiment of the present application prunes the model by preset optimization rules and further shares the parameters of the intermediate layers in the pruned model, thereby reducing the size of the model, and in order to enhance the expressive power of the model parameters, the low-rank decomposition layer is fused in the model to improve the model performance and reduce the size and complexity of the model.

[0075] In one possible implementation of the present application, when executing step 101, the balance between the structure and performance of the model needs to be considered; see Figure 2 , Figure 2 A flow chart of a model optimization method provided in an embodiment of the present disclosure; comprising:

[0076] Step 201, based on sample data, determine the model accuracy of the model before clipping.

[0077] In one possible implementation of the embodiment of the present application, when trimming the model, a back propagation method is used to calculate the importance of each layer, and dynamic trimming is performed based on the importance of the layer; the importance of the model is determined by a preset accuracy threshold.

[0078] In one implementable method of an embodiment of the present application, the model accuracy of the model before cropping is determined through sample data, and the model accuracy is determined to be 100%. A preset accuracy threshold is set according to the accuracy, for example, the preset accuracy threshold is set to 2% or 1%, etc. Specifically, the embodiment of the present application does not limit the preset accuracy threshold.

[0079] Step 202: cropping the first intermediate layer of the model, and determining the model accuracy of the cropped model based on the sample data.

[0080] The middle layers of the model are cut. When cutting, the cutting can be performed according to the preset number of middle layers and the arrangement order of the middle layers, or the preset number is not set and the middle layers are cut randomly. Specifically, the embodiment of the present application does not limit the number of middle layers cut at one time and the order of cutting the middle layers.

[0081] In one implementable method of an embodiment of the present application, after completing one cropping of the intermediate layers of the model (i.e., not caring about the number of intermediate layers cropped at one time and the order of cropping), the model accuracy of the cropped model is calculated based on the sample data; it should be noted that, in order to maintain the uniqueness of the variables, the sample data used to determine the model accuracy of the model before cropping is the same as the sample data used to determine the model accuracy of the model after cropping; the embodiment of the present application does not limit the data content of the sample data.

[0082] Step 203: If the difference between the model accuracy of the cropped model and the model accuracy of the model before cropping is greater than or equal to a preset accuracy threshold, the cropping is canceled and the cropping of the second intermediate layer continues.

[0083] In one implementable method of the embodiment of the present application, after the middle layer of the model is cropped, the accuracy of the model will change differently due to the different importance of the cropped middle layer; following the description of the above-mentioned embodiment of the application, the model accuracy of the cropped model is compared with the model progress before cropping to determine the difference between the two, and whether to perform this cropping can be determined by comparing the difference with a preset accuracy threshold; for example, the preset accuracy threshold is set to 1%, the model accuracy of the model after cropping is 98.2%, and the model accuracy of the model before cropping is 100%, then the difference is 1.8%, which is greater than the preset accuracy threshold, then it is determined that the middle layer cropped in this cropping step is more important to the model, then the cropping is canceled, and the second middle layer continues to be cropped; the second middle layer is the other middle layer except the first middle layer.

[0084] Step 204: If the difference between the model accuracy of the pruned model and the model accuracy of the model before pruned is less than the pre-accuracy threshold, continue to prune the second intermediate layer.

[0085] Following the description of step 203 above, the distance description is continued. The preset accuracy threshold is set to 1%, the model accuracy after cropping is 99.2%, and the model accuracy of the model before cropping is 100%. The difference is 0.8%, which is smaller than the preset accuracy threshold. It is determined that the first intermediate layer of this cropping is less important to the model, so the cropping of the next intermediate layer is continued. It should be noted that the description of step 203 and step 204 is only an exemplary description, and is not a specific limitation on the specific preset accuracy threshold, the model accuracy after cropping and the model accuracy before cropping, and the application embodiment does not limit this.

[0086] In one implementable method of the embodiment of the present application, following the description of the embodiment of the above application, after determining that the accuracy of the model is 99.2% after cropping the first intermediate layer, the model accuracy threshold of the model after cropping the second intermediate layer is compared with the model parameters of the model after cropping the first intermediate layer (that is, compared with 99.2% to determine the difference); that is, the model accuracy after this cropping is compared with the model accuracy obtained by the last cropping.

[0087] In an implementable manner of the embodiment of the present application, after the accuracy of the cropped model reaches the target accuracy, the cropping is stopped to obtain the cropped model; or, after the dimension of the cropped model reaches a preset dimension threshold, the cropping is stopped to obtain the cropped model; when setting the stop condition of the cropped model, there are two methods, the first is to set the target accuracy of the cropped model, for example, if the model accuracy before cropping is 100%, the target accuracy can be set to 80% or 85%, etc. Specifically, the embodiment of the present application does not limit this; in an implementable manner of the embodiment of the present application, the cropping process of the model can be repeated multiple times in actual applications. Determine the model with the smallest model dimension under the target accuracy as the cropped model.

[0088] The second method is to set a preset dimension threshold for the dimension of the cropped model. In practical applications, the cropping process of multiple pairs of models can be repeated to determine the model with the highest model accuracy under the same dimension as the cropped model. The embodiment of the present application does not limit the setting of the conditions for the end of cropping. In practical applications, it can be set according to the target requirements.

[0089] In one implementable method of an embodiment of the present application, when executing step 102 to share parameters of the intermediate layers in the cropped model, the intermediate layers are grouped and the parameters of the intermediate layers in each group are shared; when grouping, there are two methods, the first one: grouping the intermediate layers based on a first preset number and a connection order relationship of the intermediate layers, wherein there are at least two intermediate layers in a group; sharing model parameters for the intermediate layers in the same group; the preset number is an empirical value, and in actual applications, it can be set according to experience, such as setting it to 3 or 4, etc. Specifically, the embodiment of the present application does not limit this.

[0090] The second method is to randomly group the intermediate layers and share the model parameters of the intermediate layers in the same group; to enable a cross-layer sharing mechanism, that is, when grouping the models, they are not grouped according to the adjacent relationship of the models, but randomly grouped; specifically, the embodiment of the present application does not limit the grouping method.

[0091] In one implementation of the embodiment of the present application, when executing step 103, the following steps may be performed: decomposing the low-rank decomposition layer to obtain a first matrix and a second matrix; fusing the first matrix, the second matrix, and the linear layer in the middle layer. The model layer may be decomposed into two smaller matrices that may share parameters to reduce the size and complexity of the model; specifically, the model layer may be decomposed into a left matrix and a right matrix. For example, we may use a method similar to Lora to combine the matrix decomposition layer and the linear layer to form a new linear layer.

[0092] The method of integrating low-rank matrix decomposition with parameter sharing model layers can greatly improve the efficiency and performance of the model. This method can reduce the size and complexity of the model while maintaining the accuracy of the model, thereby improving storage efficiency and training speed. At the same time, this method can also improve the expressiveness of the model, making the model more efficient and accurate when processing information at different levels.

[0093] In one implementable method of an embodiment of the present application, after the model is cropped, it is necessary to train the optimized model according to a second preset amount of training data. In order to improve the performance of the cropped small model, it is necessary to pre-train it in a small amount using limited data. In this process, it is necessary to consider how to use a limited data set for training and how to design a training strategy to achieve the best performance improvement effect. At the same time, it is also necessary to consider how to apply the small model to a specific task after pre-training to achieve better performance. Specifically, the training strategy for the cropped model can be set according to actual needs, which varies according to actual needs and the specific tasks of the small model, and the embodiment of the present application does not limit this.

[0094] Adaptive depth and dimension cropping is a method of compressing models based on a balance between model structure and performance. When cropping a model, it is necessary to consider the depth and dimension of the model, as well as the balance between the model's tasks and performance. For example, for the GPT3 model, it is a large language model based on the Transformer decoder structure. Its depth and width are very large, so it consumes a lot of computing resources when deployed and used. To solve this problem, depth and dimension cropping can be used to crop it to a smaller size, thereby reducing computing costs and storage space.

[0095] In order to strike a balance between model structure pruning and performance, when pruning the depth and dimension, it is necessary to consider the degree of pruning to maintain the accuracy of the model. If the pruning is too much, the model accuracy may decrease, which will affect the application effect of the model. Therefore, it is necessary to balance the model structure and performance to achieve the best model compression effect. To this end, we propose a pruning method that maintains the depth and width of the effect, that is, an innovative method based on dynamic pruning. Dynamic pruning is an adaptive pruning method that can dynamically adjust the depth and width of the model during training to maintain the accuracy and effect of the model. Specifically, we can calculate the importance of each layer through backpropagation during training, and then dynamically prune the number of layers according to the importance to obtain a smaller model.

[0096] Point 2: Parameter sharing based on the pruned model.

[0097] This technical solution can further reduce the model size and improve storage efficiency by sharing parameters of the pruned model. This parameter sharing technology allows parameters between different layers to be shared, thereby reducing the space required for model storage. At the same time, it can also reduce the model training time and improve training efficiency.

[0098] One possible approach is to use layer-wise sharing. This technique divides the model into multiple layers, each of which contains a set of shared parameters. During training, the parameters of each layer can be shared, further reducing the size and complexity of the model. Specifically, we can divide the model into multiple layers, each of which contains multiple layers. For each layer, we can use the same parameters to share the weights in these layers. The benefit of doing this is that the number of model parameters can be reduced, thereby improving storage efficiency and training speed.

[0099] Another approach is to introduce a cross-layer sharing mechanism. This approach can share parameters between different layers and introduce some additional cross-layer connections in the model. These cross-layer connections can make the model more efficient and accurate in processing information at different levels. Specifically, we can introduce some additional parameters between different layers, which can span different layers, so that the model can better handle the flow of information. During training, we can use some gradient-based methods to update the parameters of these cross-layer connections to keep the consistency and accuracy of the model.

[0100] We can use layered sharing technology or cross-layer sharing mechanism to further reduce the model size and complexity, thereby improving storage efficiency and training speed. These methods can make the model more efficient and accurate when processing information at different levels, thereby improving the efficiency and performance of the model. The embodiment mainly uses the sharing technology.

[0101] Key point 3: Parameter sharing model layer fusion low-rank matrix decomposition.

[0102] This technical solution further enhances the expressiveness of shared parameters based on parameter sharing, and adopts the technology of low-rank matrix decomposition layer. By using low-rank matrix decomposition layer, the expressiveness of shared parameters can be improved to a higher level, thereby improving the expressiveness of the model. This technology can not only enhance the performance of the model, but also further reduce the size of the model.

[0103] In this scheme, we use the technology of low-rank matrix decomposition layer to further enhance the expressive power of parameter sharing. This method can decompose the model layer into two smaller matrices that can share parameters to reduce the size and complexity of the model. Specifically, we can decompose the model layer into a left matrix and a right matrix. For example, we can use a method similar to Lora to combine the matrix decomposition layer and the linear layer together to form a new layer. This layer can improve the network expressiveness by supplementing the low-rank matrix decomposition on the fully connected layer that shares parameters. In this layer, we can use two small matrices to splice into the original parameter-sharing matrix.

[0104] In general, this method of parameter sharing model layer fusion low-rank matrix decomposition can greatly improve the efficiency and performance of the model. This method can reduce the size and complexity of the model while maintaining the accuracy of the model, thereby improving storage efficiency and training speed. At the same time, this method can also improve the expressiveness of the model, making the model more efficient and accurate when processing information at different levels.

[0105] Point 4: Data pre-training of compressed models.

[0106] In order to improve the performance of the compressed small model, it is necessary to pre-train it with limited data. In this process, it is necessary to consider how to use limited data sets for training and how to design training strategies to achieve the best performance improvement. At the same time, it is also necessary to consider how to apply the small model to specific tasks after pre-training to achieve better performance.

[0107] Corresponding to the above-mentioned model optimization method, the present invention also proposes a model optimization device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0108] Figure 3 A schematic diagram of a structure of a model optimization device provided in an embodiment of the present disclosure, such as Figure 3 As shown, including:

[0109] A clipping unit 31, used for clipping the intermediate layer in the model based on a preset optimization rule to obtain the clipped model;

[0110] A parameter sharing unit 32, used for sharing parameters of each intermediate layer in the pruned model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing model parameters between the intermediate layers;

[0111] The fusion unit 33 is used to fuse the intermediate layer and the low-rank decomposition layer after the parameters are shared, respectively, to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

[0112] The optimization device of the model provided by the present disclosure has the following main technical solutions: first, the intermediate layers in the model are pruned based on preset optimization rules to obtain the pruned model; second, the parameters of the intermediate layers in the pruned model are shared; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is the sharing of model parameters between the intermediate layers; finally, the intermediate layers after the parameter sharing are respectively fused with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix. Compared with the related art, the embodiment of the present application prunes the model through preset optimization rules and further shares the parameters of the intermediate layers in the pruned model, thereby reducing the size of the model, and in order to enhance the expressive power of the model parameters, the low-rank decomposition layer is fused in the model to improve the model performance and reduce the size and complexity of the model.

[0113] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the cutting unit 31 includes:

[0114] A determination module 311 is used to determine the model accuracy of the model before clipping based on the sample data;

[0115] A clipping module 312, configured to clip the first intermediate layer of the model and determine the model accuracy of the clipped model based on the sample data;

[0116] The cropping module 312 is further configured to cancel cropping and continue cropping the second intermediate layer when the difference between the model accuracy of the cropped model and the model accuracy of the model before cropping is greater than or equal to a preset accuracy threshold.

[0117] The cropping module 312 is further configured to continue cropping the second intermediate layer when the difference between the model accuracy of the cropped model and the model accuracy of the model before cropping is less than the pre-accuracy threshold.

[0118] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the clipping-based unit 31 is also used for:

[0119] After the accuracy of the cropped model reaches the target accuracy, the cropping is stopped to obtain the cropped model;

[0120] Alternatively, after the dimension of the cropped model reaches a preset dimension threshold, cropping is stopped to obtain the cropped model.

[0121] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the parameter sharing unit 32 includes:

[0122] A grouping module 321, configured to group the intermediate layers based on a first preset number and a connection sequence of the intermediate layers, wherein a group contains at least two intermediate layers;

[0123] The parameter sharing module 322 is used to share model parameters of the intermediate layers in the same group.

[0124] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the parameter sharing unit 32 is also used for:

[0125] The intermediate layers are randomly grouped, and model parameters are shared among the intermediate layers in the same group.

[0126] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the fusion unit 33 also includes:

[0127] A decomposition module 331, used for decomposing the low-rank decomposition layer to obtain a first matrix and a second matrix;

[0128] The fusion module 332 is used to fuse the first matrix, the second matrix and the linear layer in the middle layer.

[0129] Furthermore, in a possible implementation of this embodiment, Figure 4 As shown, the method also includes:

[0130] The training unit 34 is used to fuse the intermediate layer and the low-rank decomposition layer after the parameters are shared in the fusion unit 33, respectively, to obtain the optimized model, and then train the optimized model according to a second preset amount of training data.

[0131] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0132] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0133] Figure 5 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0134] like Figure 5 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0135] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the optimization method of the model. For example, in some embodiments, the optimization method of the model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned model optimization method in any other appropriate manner (eg, by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0142] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0143] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete the trimming steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A model optimization method, characterized in that: include: Crop the middle layer in the model based on the preset optimization rules to obtain the cropped model; Sharing parameters of the intermediate layers in the pruned model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing the model parameters between the intermediate layers; The intermediate layer after sharing the parameters is fused with the low-rank decomposition layer to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

2. The method according to claim 1, characterized in that The method of trimming the intermediate layers in the model based on the preset optimization rules to obtain the trimmed model includes: Based on the sample data, determine the model accuracy of the model before trimming; Clipping the first intermediate layer of the model, and determining the model accuracy of the clipped model based on the sample data; If the difference between the model accuracy of the cropped model and the model accuracy of the model before cropping is greater than or equal to a preset accuracy threshold, the cropping is canceled and the cropping of the second intermediate layer continues. If the difference between the model accuracy of the pruned model and the model accuracy of the model before pruned is less than the pre-accuracy threshold, the second intermediate layer continues to be pruned.

3. The method according to claim 1, characterized in that The step of trimming the intermediate layers in the model based on the preset optimization rules to obtain the trimmed model includes: After the accuracy of the cropped model reaches the target accuracy, the cropping is stopped to obtain the cropped model; Alternatively, after the dimension of the cropped model reaches a preset dimension threshold, cropping is stopped to obtain the cropped model.

4. The method according to claim 1, characterized in that The parameter sharing of each intermediate layer in the pruned model includes: Grouping the intermediate layers based on the first preset number and the connection order of the intermediate layers, wherein there are at least two intermediate layers in one group; Model parameters are shared for intermediate layers in the same group.

5. The method according to claim 1, characterized in that The parameter sharing of each intermediate layer in the pruned model further includes: The intermediate layers are randomly grouped, and model parameters are shared among the intermediate layers in the same group.

6. The method according to claim 1, characterized in that The intermediate layer after each parameter is shared is fused with the low-rank decomposition layer to obtain the optimized model, which further includes: Decomposing the low-rank decomposition layer to obtain a first matrix and a second matrix; The first matrix, the second matrix, and the linear layer in the middle layer are fused.

7. The method according to any one of claims 1 to 6, characterized in that After fusing the intermediate layer after each parameter is shared with the low-rank decomposition layer to obtain the optimized model, the method further includes: The optimized model is trained according to a second preset amount of training data.

8. A model optimization device, characterized in that: include: A trimming unit, used for trimming the intermediate layers in the model based on a preset optimization rule to obtain the trimmed model; A parameter sharing unit, used to share parameters of each intermediate layer in the pruned model; wherein different intermediate layers correspond to different model parameters, and the parameter sharing is used for sharing model parameters between the intermediate layers; A fusion unit is used to fuse the intermediate layer and the low-rank decomposition layer after the parameters are shared, respectively, to obtain the optimized model; wherein the low-rank decomposition layer is a low-rank decomposition matrix.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.