Model parameter processing method and device, program product and electronic equipment
By quantifying parameters of large machine learning models, filtering and quantifying non-critical parameters using activation values, the problem of excessive amount of model parameters is solved, and deployment on edge devices is achieved and inference speed is improved.
Patent Information
- Application Number
- CN202411081751.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-08
AI Technical Summary
Large machine learning models such as large language models have high demands for storage and computing resources, making it difficult to deploy in lightweight scenarios such as edge devices.
By obtaining the activation value of the model, the target parameters are divided into a set of non-quantized parameters and a set of parameters to be quantized, and the parameters to be quantized are quantized to generate a simplified model to reduce the amount of parameters and resource requirements.
On the premise of ensuring model accuracy and performance, it significantly reduces the amount of model parameters, reduces the storage and computing resources requirements, and improves the model inference speed. It is suitable for edge devices and high real-time scenarios.
Smart Images

Figure CN118917374A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence and data processing technology, and in particular to a model parameter processing method, a model parameter processing device, a computer program product and an electronic device. Background Art
[0002] With the development of artificial intelligence technology, the number of parameters in machine learning models has increased rapidly. For example, large language models (LLMs) usually have tens of billions or even hundreds of billions of parameters. As a result, the amount of data in the model is very large, which may reach hundreds of GB. It requires a lot of storage and computing resources and is difficult to deploy in scenarios such as edge devices.
[0003] Therefore, how to effectively reduce the amount of data in the model is an urgent problem to be solved in the industry. Summary of the invention
[0004] The present disclosure provides a model parameter processing method, a model parameter processing device, a computer program product and an electronic device, so as to solve the problem of excessive model data volume at least to a certain extent.
[0005] According to a first aspect of the present disclosure, a model parameter processing method is provided, the method comprising: obtaining a first model; the first model comprising a plurality of target parameters; processing input data through the first model, outputting an activation value by an intermediate layer of the first model, and determining the activation value corresponding to the target parameter; dividing the target parameter into at least two sets according to the activation value; the at least two sets comprising a non-quantized parameter set and at least one parameter set to be quantized; quantizing the target parameters in the parameter set to be quantized to obtain a second model corresponding to the first model.
[0006] According to a second aspect of the present disclosure, a model parameter processing device is provided, the device comprising: a data acquisition module, configured to acquire a first model; the first model comprises a plurality of target parameters; a forward processing module, configured to process input data through the first model, output activation values from an intermediate layer of the first model, and determine activation values corresponding to the target parameters; a parameter partitioning module, configured to partition the target parameters into at least two sets according to the activation values; the at least two sets comprise a non-quantized parameter set and at least one set of parameters to be quantized; a parameter quantization module, configured to quantize the target parameters in the set of parameters to be quantized, so as to obtain a second model corresponding to the first model.
[0007] According to a third aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method of the first aspect and possible implementations thereof are implemented.
[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the model parameter processing method of the above-mentioned first aspect and its possible implementation method by executing the executable instructions.
[0009] The technical solution disclosed in this disclosure has the following beneficial effects:
[0010] On the one hand, it provides a solution for quantizing the parameters of the machine learning model, which uses activation values to filter out the parameters that need to be quantized, and retains some important parameters without quantization. This simplifies the model while ensuring the accuracy and performance of the model as much as possible, which can significantly reduce the number of model parameters and reduce storage and computing resource requirements. The model can be deployed in lightweight scenarios such as edge devices, improving the popularity and applicability of the model. On the other hand, it can improve the model reasoning speed and shorten the response time, which is conducive to its application in high real-time and interactive scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 A flow chart showing a model parameter processing method in this exemplary embodiment.
[0012] Figure 2 A flowchart for dividing target parameters into sets in this exemplary embodiment is shown.
[0013] Figure 3 A flowchart for determining an importance evaluation value in this exemplary embodiment is shown.
[0014] Figure 4 A flowchart for obtaining a fourth model in this exemplary embodiment is shown.
[0015] Figure 5 A schematic diagram showing a quantization and precision consistency process in this exemplary embodiment.
[0016] Figure 6 A schematic flow chart of a model parameter processing method in this exemplary embodiment is shown.
[0017] Figure 7 A schematic structural diagram of a model parameter processing device in this exemplary embodiment is shown.
[0018] Figure 8 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION
[0019] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings.
[0020] The accompanying drawings are schematic diagrams of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in hardware modules or integrated circuits, or in networks, processors or microcontrollers. The embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the description below, many specific details are provided to provide a full description of the embodiments of the present disclosure. However, those skilled in the art should appreciate that one or more specific details may be omitted when implementing the technical solution of the present disclosure, or one or more specific details may be replaced by other methods, components, devices, steps, etc.
[0021] Large machine learning models, represented by large language models, have excellent performance, but they also have the problem of high operation and maintenance costs. For example, large language models usually have tens of billions or even hundreds of billions of parameters and hundreds of GB of data in FP16 (half-precision floating point) format. Such a large model limits its application on general hardware devices. For example, the hardware capabilities of edge devices cannot support its storage or operation of large language models.
[0022] In view of the above problems, an exemplary embodiment of the present disclosure provides a model parameter processing method.
[0023] Figure 1 The schematic flow of the model parameter processing method is shown, which may include the following steps S110 to S140:
[0024] Step S110, obtaining a first model; the first model includes a plurality of target parameters;
[0025] Step S120, processing the input data through the first model, outputting the activation value by the middle layer of the first model, and determining the activation value corresponding to the target parameter;
[0026] Step S130, dividing the target parameters into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one to-be-quantized parameter set;
[0027] Step S140 , quantizing the target parameter in the set of parameters to be quantized to obtain a second model corresponding to the first model.
[0028] based on Figure 1 The method, on the one hand, provides a solution for quantizing the parameters of the machine learning model, which uses the activation value to filter out the parameters that need to be quantized, and retains some important parameters without quantization. In this way, the model is simplified while ensuring the accuracy and performance of the model as much as possible, which can significantly reduce the number of model parameters and reduce storage and computing resource requirements. The model can be deployed in lightweight scenarios such as edge devices, improving the popularity and applicability of the model. On the other hand, it can improve the model reasoning speed and shorten the response time, which is conducive to its application in high real-time and interactive scenarios.
[0029] Below Figure 1 Provide detailed instructions for each step.
[0030] refer to Figure 1 In step S110, a first model is obtained; the first model includes multiple target parameters.
[0031] The first model may be any machine learning model, such as a large language model. The target parameter is a simplifiable parameter in the first model, which may be all or part of the parameters in the first model. The target parameter may include weights, biases, or other parameters. In one embodiment, due to the strong correlation between weights and activation values, the target parameter may be set to include only weights.
[0032] Exemplarily, all weights in the first model may be used as target parameters. Alternatively, some weights may be screened out from all weights in the first model, such as the weights of a specific intermediate layer (such as an intermediate layer other than the first intermediate layer or the last intermediate layer) as the target parameter.
[0033] Continue to refer Figure 1 In step S120, the input data is processed by the first model, the activation value is output by the middle layer of the first model, and the activation value corresponding to the target parameter is determined.
[0034] Among them, the input data can be the data that actually needs to be processed, or it can be sample data. The present disclosure does not limit the source and specific values of the input data. After the input data is input into the first model, the first model performs forward propagation, and each intermediate layer performs layer-by-layer processing, and each intermediate layer outputs an intermediate processing result, which is called an activation value. It should be understood that the activation value in this exemplary embodiment can be data processed by the activation function or data not processed by the activation function. For example, if some intermediate layers are not set with an activation function, the intermediate processing results output by these intermediate layers also belong to activation values.
[0035] The target parameter can be matched with the activation value. The activation value is the result obtained by the intermediate layer after calculating the input data or the data output by other intermediate layers. The calculation process of the intermediate layer may use the target parameter to match the target parameter with the activation value generated by the calculation process in which it participates. The activation value corresponding to the target parameter is thus determined. For example, the calculation process of each intermediate layer usually uses the parameters of the intermediate layer, so the target parameter in each intermediate layer can be matched with the activation value output by the intermediate layer in units of the intermediate layer. Of course, the present disclosure is not limited to such a corresponding relationship.
[0036] Continue to refer Figure 1 In step S130, the target parameters are divided into at least two sets according to the activation values; the at least two sets include a non-quantized parameter set and at least one to-be-quantized parameter set.
[0037] The target parameters in the non-quantized parameter set do not need to be quantized. The target parameters in the parameter set to be quantized need to be quantized. Furthermore, if there are multiple parameter sets to be quantized, different parameter sets to be quantized may correspond to different quantization degrees.
[0038] The importance of the target parameters in the first model can be evaluated based on the activation value. Important target parameters do not need to be quantified, while unimportant target parameters can be quantified. This allows the target parameters to be screened and the accuracy of important parameters to be retained.
[0039] In one embodiment, reference Figure 2 As shown, the above-mentioned dividing the target parameters into at least two sets according to the activation values may include the following steps S210 and S220:
[0040] Step S210, calculating the importance evaluation value of the target parameter according to the activation value.
[0041] Exemplarily, a mapping relationship between the activation value and the importance evaluation value can be preset, such as a linear or nonlinear functional relationship between the two. In this way, the corresponding importance evaluation value can be calculated through the activation value. For example, the target parameter a corresponds to the activation value b, and the corresponding importance evaluation value c is calculated according to the activation value b, thereby determining that the importance evaluation value of the target parameter is c. In general, the activation value and the importance evaluation value can be positively correlated, that is, the larger the activation value, the larger the importance evaluation value.
[0042] In one embodiment, considering that the activation values of different intermediate layers have inconsistent scales, the activation values may be normalized so that the activation values of different layers are at the same scale. The activation values of each intermediate layer may be normalized separately. For example, for a certain intermediate layer, the mean and standard deviation of multiple activation values of the intermediate layer may be calculated, and the multiple activation values of the intermediate layer may be normalized according to the mean and standard deviation.
[0043] In one embodiment, reference Figure 3 As shown, the above calculation of the importance evaluation value of the target parameter according to the activation value may include the following steps S310 and S320:
[0044] Step S310, determining the contribution value of the target parameter in the corresponding activation value;
[0045] Step S320 , obtaining an importance evaluation value of the target parameter by accumulating the contribution values of the target parameter in each corresponding activation value.
[0046] Among them, it is considered that an activation value may be calculated by multiple target parameters, or a target parameter may participate in the calculation process of multiple activation values. Therefore, the contribution value of the target parameter in the corresponding activation value can be determined. For example, in a certain intermediate layer, the target parameters a1 and a2 participate in the calculation of the activation value b1, the target parameters a1, a2, and a3 participate in the calculation of the activation value b2, and the target parameters a2 and a3 participate in the calculation of the activation value b3. Since the activation value b1 is calculated by two target parameters, it can be determined that the contribution values of the two target parameters are both b1 / 2, or the weights are assigned according to the size of the target parameters and the contribution values are calculated, such as the contribution values are b1·a1 / (a1+a2) and b1·a2 / (a1+a2). The target parameter a1 participates in the calculation of two activation values b1 and b2, that is, a1 corresponds to b1 and b2, and the contribution values of a1 in b1 and b2 are calculated respectively. The contribution values are accumulated to obtain the importance evaluation value of a1. Based on the same method, the importance evaluation values of the target parameters a2 and a3 can be calculated.
[0047] The importance evaluation value calculated in the above manner can more accurately evaluate the importance of the target parameter in the first model.
[0048] Step S220, dividing the target parameters into at least two sets based on the importance evaluation values; wherein the importance evaluation values of the target parameters in the non-quantized parameter set are greater than the importance evaluation values of the target parameters in the to-be-quantized parameter set.
[0049] Among them, the target parameters with higher importance evaluation values can be divided into the non-quantized parameter set, and the target parameters with lower importance evaluation values can be divided into the parameter set to be quantized. Alternatively, the target parameters with lower importance evaluation values can be further graded to be divided into parameter sets to be quantized at different levels.
[0050] In one implementation, the above-mentioned dividing the target parameters into at least two sets based on the importance evaluation values may include the following steps:
[0051] Sort the target parameters in descending order of importance assessment value;
[0052] The sorted target parameters are divided according to a preset ratio to form at least two sets.
[0053] Among them, the preset proportion represents the proportion of each set, which can be determined according to experience or specific needs. In one embodiment, it can be determined according to the total amount of parameters of the first model or the total amount of target parameters. For example, if the total amount of parameters of the first model is less than 7 billion, the preset proportion of the non-quantized parameter set is 1%, that is, 1% of the parameters are considered to be important parameters; if the total amount of parameters of the first model is 7 billion to 13 billion, the preset proportion of the non-quantized parameter set is 0.5%; if the total amount of parameters of the first model is 13 billion to 33 billion, the preset proportion of the non-quantized parameter set is 0.1%. The present disclosure does not make specific limitations on this.
[0054] After the target parameters are sorted in order from high to low according to the importance evaluation value, the corresponding target parameters are screened out according to the preset ratio of the non-quantized parameter set and divided into the non-quantized parameter set. The remaining target parameters can all be divided into the parameter set to be quantized. Alternatively, if there is more than one parameter set to be quantized, the corresponding target parameters can be screened out in sequence according to the preset ratio of each parameter set to be quantized and divided into the corresponding parameter set to be quantized. For example, the preset ratio can be set to include: the preset ratio of the non-quantized parameter set is 1%, the preset ratio of the first-level parameter set to be quantized is 9%, and the remaining 90% belong to the second-level parameter set to be quantized, and the quantization degree corresponding to the second-level parameter set to be quantized is higher than that of the first-level parameter set to be quantized. After the target parameters are sorted in order from high to low according to the importance evaluation value, the first 1% of the target parameters are selected to form the non-quantized parameter set, 1% to 10% of the target parameters are selected to form the first-level parameter set to be quantized, and the remaining target parameters form the second-level parameter set to be quantized.
[0055] Continue to refer Figure 1 In step S140, the target parameter in the set of parameters to be quantized is quantized to obtain a second model corresponding to the first model.
[0056] The quantization process in this exemplary embodiment refers to simplifying the target parameter, which may be converting the target parameter from a relatively complex type to a relatively simple type. For example, converting the target parameter of FP16 type to IN8 (8-bit integer), INT4 (4-bit integer) or INT3 (3-bit integer) type, converting the target parameter of INT8 type to INT4 or INT3 type, and so on.
[0057] No quantization process is performed on the target parameters in the non-quantized parameter set, thereby retaining the most critical information in the first model, thereby reducing the impact of the parameter quantization process on the model performance.
[0058] After quantizing the target parameters in the set of parameters to be quantized, a new model is obtained, which is called the second model. The second model is a model after parameter quantization based on the first model. The data volume and complexity of the second model are significantly lower than those of the first model, and it is more suitable for deployment in lightweight scenarios. For example, on edge devices, the second model can be used to replace the first model.
[0059] In one implementation, the first model and the second model may be saved, and the first model or the second model may be used to process the business according to the importance of the specific business. For example, for important business, the first model may be used to process it in order to ensure its accuracy. For non-important business, the second model may be used to process it in order to improve processing efficiency.
[0060] In one embodiment, the at least two sets include at least two sets of parameters to be quantized, and different sets of parameters to be quantized correspond to different degrees of quantization. For example, a first-level set of parameters to be quantized and a second-level set of parameters to be quantized may be set, the first-level set of parameters to be quantized corresponds to a first-level degree of quantization, and the second-level set of parameters to be quantized corresponds to a second-level degree of quantization, which is higher than the first-level degree of quantization. Accordingly, the quantization processing of the target parameter in the set of parameters to be quantized may include the following steps:
[0061] According to the quantization degree corresponding to each set of parameters to be quantized, a corresponding quantization process is performed on the target parameters in each set of parameters to be quantized.
[0062] For example, for the first-level parameter set to be quantized, the target parameter therein can be converted from FP16 type to INT4 type. For the second-level parameter set to be quantized, the target parameter therein can be converted from FP16 type to INT3 type. The target parameter with a high degree of quantization has a smaller amount of data after quantization processing. In this way, hierarchical quantization processing can be achieved, which can improve the flexibility and pertinence of quantization processing.
[0063] In one embodiment, the acquisition of the first model may include: loading the first model into a volatile memory (such as RAM). The processing of the input data by the first model and the output of the activation value by the intermediate layer of the first model may include: loading the input data into a volatile memory; executing the code of the first model by the processor to process the input data, and the activation value is output by the intermediate layer of the first model. Correspondingly, after obtaining the second model corresponding to the first model, the second model may also be stored in a non-volatile memory (such as ROM) and overwrite the first model. This reduces the amount of model data stored in the non-volatile memory.
[0064] After quantizing some target parameters and obtaining the second model. There may be parameters of different types in the second model, such as the coexistence of FP16 and IN3. It may cause a certain degree of difficulty and pressure in hardware design, operation, etc. Taking this problem into consideration, in one embodiment, the second model can be processed for accuracy consistency so that the parameter types in the second model are unified, while minimizing the model performance loss caused by the accuracy consistency processing. Exemplarily, a quantization-aware training method can be used to simulate the quantization process during training so that the model can adapt to low-precision representation, thereby reducing the loss of accuracy during actual quantization. Alternatively, a method of refining quantization parameters can be used to calculate and apply different quantization parameters (such as scaling factors and zero points) to different channels respectively, so as to more finely adapt to the characteristics of each channel, thereby optimizing the quantization effect. Make the quantization of each channel more accurate and reduce the error caused by quantization.
[0065] In one embodiment, reference Figure 4 As shown, after obtaining the second model corresponding to the first model, the model parameter processing method may further include the following steps S410 to S430:
[0066] Step S410, setting a scaling factor in one or more intermediate layers of the second model to form a third model;
[0067] Step S420, training the third model to update the scaling factor;
[0068] Step S430: scaling the parameters of the third model using the updated scaling factor to obtain a fourth model corresponding to the third model.
[0069] Among them, the scaling factor is a learnable parameter. It is equivalent to setting a certain scaling factor for the parameters (such as weights) of one or more intermediate layers of the second model. The model after the scaling factor is increased is called the third model to distinguish it from the second model. The third model is trained, such as using a supervised data set for training, and the scaling factor is updated according to the gradient information by back propagation to minimize the loss function. During the training process, other parameters of the third model can also be updated. It should be noted that different scaling factors can be set for different intermediate layers, and the same or different scaling factors can be set for different parameters of the same intermediate layer. After the training is completed, the updated scaling factor is multiplied by the corresponding parameter, thereby completing the final scaling process of the parameter and achieving accuracy consistency. The model after scaling is called the fourth model. The fourth model can be saved or output as the final model.
[0070] The above precision consistency processing method can ensure that the model does not have mixed precision problems and ensure the stability of the model running on different hardware.
[0071] Figure 5 A schematic diagram of quantizing the target parameters and processing the precision consistency is shown. The model can be forward propagated first to determine the activation value distribution, and the important weights can be determined according to the activation value distribution. The important weights are non-quantized parameters, and the remaining weights are quantized parameters. It can be seen that the weights W that have not been quantized are all in FP16 format, and the weights in the third row are non-quantized parameters. The quantized parameters are quantized and converted into INT3 format, and their values are simplified. At this time, the weights Q(W) in the model exist in two formats, FP16 and INT3. Then all weights are processed for precision consistency so that the weights Q(W) are all in INT3 format. The final fourth model is thus obtained.
[0072] Figure 6 A schematic flowchart of the model parameter processing method in this exemplary embodiment is shown.
[0073] include:
[0074] Step S601, selecting the LLMs to be included in the LLM warehouse;
[0075] Step S602, forward propagation is performed on the LLM on the training set, the activation value distribution is counted, and the activation values corresponding to the weights in the LLM are determined, and the activation values can be sorted from large to small according to the activation values;
[0076] Step S603, determining the first 0.1-1% of the weights as non-quantized parameters, and performing INT3 quantization on the remaining weights;
[0077] Step S604, performing precision consistency processing on the LLM so that the weights in the LLM have the same format;
[0078] Step S605, determining whether the LLM is the last LLM, if not, returning to step S601, selecting the next LLM to be processed; if yes, continuing to step S606;
[0079] Step S606, constructing an LLM warehouse based on the processed LLM, which may include several commonly used LLMs, such as Llama, Llama-2 (Llama is the LLM model released by Meta, and Llama-2 is the second generation version), etc.; the LLM warehouse may be deployed on a terminal device or server to provide corresponding question-and-answer services;
[0080] Step S607, receiving a query statement input by a user;
[0081] Step S608: Output the LLM answer to the query statement through LLM reasoning.
[0082] The exemplary embodiment of the present disclosure also provides a model parameter processing device. Figure 7 As shown, the model parameter processing device 700 may include the following program modules:
[0083] The data acquisition module 710 is configured to acquire a first model; the first model includes a plurality of target parameters;
[0084] A forward processing module 720 is configured to process the input data through the first model, output activation values from the intermediate layer of the first model, and determine the activation values corresponding to the target parameters;
[0085] A parameter division module 730 is configured to divide the target parameter into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one to-be-quantized parameter set;
[0086] The parameter quantization module 740 is configured to perform quantization processing on the target parameter in the set of parameters to be quantized to obtain a second model corresponding to the first model.
[0087] In one embodiment, dividing the target parameters into at least two sets according to the activation value includes: calculating an importance evaluation value of the target parameter according to the activation value; dividing the target parameters into at least two sets based on the importance evaluation value; wherein the importance evaluation value of the target parameters in the non-quantized parameter set is greater than the importance evaluation value of the target parameters in the to-be-quantized parameter set.
[0088] In one embodiment, calculating the importance evaluation value of the target parameter based on the activation value includes: determining the contribution value of the target parameter in the corresponding activation value; and obtaining the importance evaluation value of the target parameter by accumulating the contribution values of the target parameter in each corresponding activation value.
[0089] In one embodiment, dividing the target parameters into at least two sets based on the importance evaluation value includes: sorting the target parameters in order from high to low according to the importance evaluation value; and dividing the sorted target parameters according to a preset ratio to form the at least two sets.
[0090] In one embodiment, the device is also used to: after obtaining the second model corresponding to the first model, set a scaling factor in one or more intermediate layers of the second model to form a third model; train the third model to update the scaling factor; and scale the parameters of the third model using the updated scaling factor to obtain a fourth model corresponding to the third model.
[0091] In one embodiment, the at least two sets include at least two sets of parameters to be quantized, and different sets of parameters to be quantized correspond to different degrees of quantization; the quantization processing of the target parameters in the set of parameters to be quantized includes: performing corresponding quantization processing on the target parameters in each set of parameters to be quantized according to the degree of quantization corresponding to each set of parameters to be quantized.
[0092] In one embodiment, the obtaining of the first model includes: loading the first model into a volatile memory; the processing of the input data by the first model, and outputting an activation value by the intermediate layer of the first model, includes: loading the input data into the volatile memory; executing the code of the first model by a processor to process the input data, and outputting an activation value by the intermediate layer of the first model; the device is also used for: after obtaining the second model corresponding to the first model, storing the second model in a non-volatile memory and overwriting the first model.
[0093] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.
[0094] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0095] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the game editing method is implemented.
[0096] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.
[0097] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.
[0098] The code of the computer program can be written in one or more programming languages. Programming languages such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).
[0099] The computer program can be carried or transmitted through electrical, magnetic, optical, electromagnetic, infrared and other signals. The electronic device can convert the signal carrying the computer program into a digital signal, and then run the computer program. When the computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, it can enable the processor of the electronic device to execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned game editing method, which includes the following steps: Step S110, obtain the first model; the first model includes a plurality of target parameters; Step S120, process the input data through the first model, output the activation value by the middle layer of the first model, and determine the activation value corresponding to the target parameter; Step S130, divide the target parameter into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one parameter set to be quantized; Step S140, quantize the target parameter in the parameter set to be quantized to obtain the second model corresponding to the first model.
[0100] The above method steps are implemented through a computer program. On the one hand, a solution is provided for quantizing the parameters of the machine learning model. The parameters that need to be quantified are screened out through activation values, and some important parameters are retained without quantification. In this way, the model is simplified while ensuring the accuracy and performance of the model as much as possible. The number of model parameters can be significantly reduced, and the storage and computing resource requirements can be reduced. The model can be deployed in lightweight scenarios such as edge devices, improving the popularity and applicability of the model. On the other hand, it can improve the model reasoning speed and shorten the response time, which is conducive to its application in high real-time and interactive scenarios.
[0101] The exemplary embodiments of the present disclosure also provide an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as a computer program. The processor executes the method steps of various exemplary embodiments of the present disclosure by executing the executable instructions.
[0102] Reference below Figure 8 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 8 The electronic device 800 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.
[0103] like Figure 8 As shown, the electronic device 800 may include: a processor 810 , a memory 820 , a bus 830 , an I / O (input / output) interface 840 , and a network adapter 850 .
[0104] The memory 820 may include a volatile memory, such as a RAM 821, a cache unit 822, and may also include a non-volatile memory, such as a ROM 823. The memory 820 may also include one or more program modules 824, such program modules 824 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 824 may include each module in the above-mentioned device.
[0105] The processor 810 may include one or more processing units. For example, the processor 810 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.
[0106] The processor 810 can be used to execute executable instructions stored in the memory 820, such as the above-mentioned game editing method, which includes the following steps: step S110, obtaining a first model; the first model includes multiple target parameters; step S120, processing the input data through the first model, outputting an activation value by the middle layer of the first model, and determining the activation value corresponding to the target parameter; step S130, dividing the target parameter into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one parameter set to be quantized; step S140, quantizing the target parameter in the parameter set to be quantized to obtain a second model corresponding to the first model.
[0107] By executing the above method steps through the processor 810, on the one hand, a solution for quantizing the parameters of the machine learning model is provided, the parameters that need to be quantized are screened out through the activation value, and some important parameters are retained without quantization, thereby simplifying the model while ensuring the accuracy and performance of the model as much as possible, which can significantly reduce the number of model parameters and reduce storage and computing resource requirements. The model can be deployed in lightweight scenarios such as edge devices, improving the popularity and applicability of the model. On the other hand, it can improve the model reasoning speed and shorten the response time, which is conducive to its application in high real-time and interactive scenarios.
[0108] The bus 830 is used to realize the connection between different components of the electronic device 800, and may include a data bus, an address bus, and a control bus.
[0109] The electronic device 800 can communicate with one or more external devices 900 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 840 .
[0110] The electronic device 800 can communicate with one or more networks through the network adapter 850. For example, the network adapter 850 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 850 can communicate with other modules of the electronic device 800 through the bus 830.
[0111] although Figure 8 Not shown, other hardware and / or software modules may also be provided in the electronic device 800, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.
[0112] As can be seen from the above, the technical solution of the present disclosure can be implemented as a method, an apparatus, a system, a computer program product, a storage medium, an electronic device, etc. Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, such as being respectively referred to as a "circuit", "module" or "system".
[0113] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are only exemplary, and the scope and spirit of the present disclosure are indicated by the claims, and any variations, uses or adaptive changes of the present disclosure should be covered, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure.
Claims
1. A model parameter processing method, characterized in that: The method comprises: Acquire a first model; the first model includes a plurality of target parameters; Processing the input data through the first model, outputting an activation value by an intermediate layer of the first model, and determining the activation value corresponding to the target parameter; Dividing the target parameters into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one parameter set to be quantized; Quantizing the target parameters in the set of parameters to be quantized to obtain a second model corresponding to the first model.
2. The method according to claim 1, characterized in that The dividing the target parameter into at least two sets according to the activation value comprises: Calculating an importance evaluation value of the target parameter according to the activation value; The target parameters are divided into at least two sets based on the importance evaluation values; wherein the importance evaluation values of the target parameters in the non-quantized parameter set are greater than the importance evaluation values of the target parameters in the to-be-quantized parameter set.
3. The method according to claim 2, characterized in that Calculating the importance evaluation value of the target parameter according to the activation value includes: Determining a contribution value of the target parameter in the corresponding activation value; The importance evaluation value of the target parameter is obtained by accumulating the contribution values of the target parameter in each corresponding activation value.
4. The method according to claim 2, characterized in that The dividing the target parameters into at least two sets based on the importance evaluation value comprises: Sorting the target parameters in descending order of importance evaluation values; The sorted target parameters are divided according to a preset ratio to form the at least two sets.
5. The method according to claim 1, wherein After obtaining the second model corresponding to the first model, the method further includes: setting scaling factors in one or more intermediate layers of the second model to form a third model; training the third model to update the scaling factor; The parameters of the third model are scaled using the updated scaling factor to obtain a fourth model corresponding to the third model.
6. The method according to claim 1, characterized in that The at least two sets include at least two sets of parameters to be quantized, and different sets of parameters to be quantized correspond to different degrees of quantization; The quantizing of the target parameter in the set of parameters to be quantized includes: According to the quantization degree corresponding to each set of parameters to be quantized, corresponding quantization processing is performed on the target parameters in each set of parameters to be quantized.
7. The method according to claim 1, characterized in that The obtaining of the first model includes: loading the first model into volatile memory; The processing of the input data by the first model, and outputting an activation value by an intermediate layer of the first model, includes: loading the input data into the volatile memory; Executing the code of the first model through a processor to process the input data, and outputting an activation value by an intermediate layer of the first model; After obtaining the second model corresponding to the first model, the method further includes: The second model is stored in a non-volatile memory and overwrites the first model.
8. A model parameter processing device, characterized in that: The device comprises: A data acquisition module is configured to acquire a first model; the first model includes a plurality of target parameters; a forward processing module configured to process input data through the first model, output activation values from the intermediate layers of the first model, and determine the activation values corresponding to the target parameters; A parameter division module is configured to divide the target parameter into at least two sets according to the activation value; the at least two sets include a non-quantized parameter set and at least one to-be-quantized parameter set; The parameter quantization module is configured to perform quantization processing on the target parameter in the set of parameters to be quantized to obtain a second model corresponding to the first model.
9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 7 by executing the executable instructions.
