Model quantification method and device, equipment and storage medium
By constructing a reinforcement learning environment and training agents, the problem of neglecting inter-layer mixture accuracy in intra-layer mixture accuracy in existing technologies is solved, and the performance of large models and training acceleration are achieved under low accuracy conditions.
Patent Information
- Application Number
- CN202510838668.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-20
- Publication Date
- 2025-10-28
AI Technical Summary
Existing model quantization methods mainly emphasize intra-layer mixing accuracy, ignoring the potential value of inter-layer mixing accuracy. This leads to problems such as high computational complexity, long solution time, or inability to solve the problem, especially in large-scale problems where combinatorial explosion is likely to occur.
A reinforcement learning-based approach is used to construct an agent environment. Hierarchical mixed precision allocation data is obtained by training agents. The reinforcement learning environment is constructed by combining KL divergence and quantization loss. Agent behavior and reward mechanisms are set, and hierarchical mixed precision quantization is performed.
It achieves performance improvement for large models under low-precision conditions, reduces environment interaction time, accelerates reinforcement learning training speed, balances local and global performance allocation, and improves the effectiveness of allocation strategies.
Smart Images
Figure CN120851084A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of model compression technology, and more specifically, to a model quantization method, apparatus, device, and storage medium. Background Art
[0002] Large Language Models (LLMs) have demonstrated outstanding performance in various natural language benchmarks, and their powerful language understanding capabilities have laid the foundation for general artificial intelligence. However, the enormous computing power and memory requirements of LLMs also pose significant challenges to practical applications and deployments, especially on edge devices.
[0003] To reduce computational burden, current model quantization methods mainly include Post-Training Quantization (PTQ) and Mixed Precision Quantization (MPQ). PTQ, as an efficient compression method, has been successfully applied to weight quantization in LLMs models. MPQ differentiates the quantization difficulty of weights based on their importance, quantizing difficult-to-quantize weights with higher precision and the remaining weights with lower precision, effectively mitigating the quantization difficulty caused by outliers.
[0004] However, current methods primarily emphasize mixed precision within layers, neglecting the potential value of mixed precision between layers. Furthermore, existing precision allocation strategies based on integer programming lack the ability to explore new solutions and are prone to getting trapped in local optima. As the problem size increases, the computational complexity grows exponentially, and solving large-scale problems may encounter the "combinatorial explosion" problem, leading to excessively long solution times or even failure to solve the problem. Summary of the Invention
[0005] According to one aspect of this application, a method for quantizing a model is provided, comprising: acquiring a preset target model and calibration data; generating a reinforcement learning environment for a preset agent through the layer to be quantized of the target model based on the calibration data; training the preset agent based on the reinforcement learning environment to obtain first quantization precision allocation data of the target model; and quantizing the target model according to the first quantization precision allocation data.
[0006] According to some embodiments, obtaining a preset target model and calibration data includes: setting quantization parameters of the target model, wherein the quantization parameters include the layer to be quantized and the expected quantization accuracy.
[0007] According to some embodiments, a reinforcement learning environment for a preset agent is generated based on calibration data through the quantized layer of a target model, including: obtaining the first input features and first output features of the quantized layer before quantization based on the calibration data; quantizing the quantized layer according to a preset first quantization rule and the expected quantization precision to obtain the quantized linear layer and quantization loss corresponding to the quantized layer; obtaining the quantized second output features through the quantized linear layer and the first input features; calculating the KL divergence of the output feature distribution of the quantized layer based on the first output features and the second output features; and constructing a reinforcement learning environment based on the KL divergence and the quantization loss.
[0008] According to some embodiments, before training a preset agent based on a reinforcement learning environment to obtain the first quantization accuracy allocation data of the target model, the behavior of the preset agent is set, including predicting the quantization accuracy of the layer to be quantized; setting the state space of the layer to be quantized corresponding to the preset agent; setting the reward mechanism corresponding to the preset agent; and setting the training rounds of the preset agent.
[0009] According to some embodiments, a reward mechanism corresponding to a preset agent is set, including: obtaining the second quantization precision allocation data of the target model output by the preset agent after a training round; and setting the global reward and local reward corresponding to the preset agent in the reinforcement learning environment based on the second quantization precision allocation data.
[0010] According to some embodiments, training a preset agent in a reinforcement learning environment to obtain first quantization accuracy allocation data for a target model includes: determining the training phase to which the current training round belongs, the training phase including a warm-up phase and a formal training phase; in the warm-up phase, training the preset agent on random behavior in a reinforcement learning environment, wherein the random behavior training of the preset agent does not include quantization accuracy prediction of the layer to be quantized; in the formal training phase, training the preset agent on quantization accuracy prediction in a reinforcement learning environment to obtain first quantization accuracy allocation data and a global reward.
[0011] According to some embodiments, quantizing a target model based on a first quantization precision allocation data includes: quantizing any layer to be quantized in the target model using the first quantization precision allocation data according to a preset second quantization rule, so as to obtain a quantized target model.
[0012] According to one aspect of this application, a model quantization apparatus is provided, comprising: a data acquisition module for acquiring a preset target model and calibration data; an environment configuration module for generating a reinforcement learning environment for a preset agent based on the calibration data and through the layer to be quantized of the target model; an agent training module for training the preset agent based on the reinforcement learning environment to obtain first quantization precision allocation data for the target model; and a model quantization module for quantizing the target model according to the first quantization precision allocation data.
[0013] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the method as described above.
[0014] According to one aspect of this application, a computer-readable storage medium is provided that stores a computer program or instructions thereon, which, when executed by a processor, implement the method as described above.
[0015] According to the embodiments of this application, hierarchical mixed precision quantization for large models is realized, which improves the performance of large models under low precision quantization conditions, significantly reduces the interaction time with the environment, accelerates the training speed of reinforcement learning, balances the performance and precision allocation between local and global aspects, and improves the effectiveness of the allocation strategy.
[0016] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application.
[0018] Figure 1 A flowchart illustrating a model quantization method according to an example embodiment of this application is shown.
[0019] Figure 2 A schematic diagram of a quantization apparatus for a model according to an example embodiment of this application is shown.
[0020] Figure 3 A block diagram of an electronic device according to an example embodiment of this application is shown. Detailed Implementation
[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.
[0022] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of these specific details, or other methods, components, materials, apparatus, or operations may be employed. In these cases, well-known structures, methods, apparatuses, implementations, materials, or operations will not be shown or described in detail.
[0023] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0024] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.
[0025] This application provides a method, apparatus, device, and storage medium for quantizing a model, which improves the performance of the model.
[0026] The following detailed description, with reference to the accompanying drawings, describes a model quantization method, apparatus, device, and storage medium according to an embodiment of this application.
[0027] Figure 1 A flowchart illustrating a model quantization method according to an example embodiment of this application is shown.
[0028] like Figure 1 As shown, in step S100, a preset target model and calibration data are obtained.
[0029] For example, in step S100, the quantization device acquires a preset target model and calibration data, and sets the quantization parameters of the target model.
[0030] The quantization device acquires the preset target model and calibration data, and sets the quantization parameters of the target model.
[0031] In some implementations, the target model can be the large Llama2_7B model. Calibration data can be obtained from the Wikitext2 dataset. Quantization parameters include the layer to be quantized and the expected quantization precision.
[0032] For example, the target model may include 32 decoder modules (LlamaDecoderLayer), using B = {B0, B1, ..., B...} 31} represents. In any module B i The text contains [mlp.up_proj,mlp.gate_proj,mlp.down_proj,self_attn.k_proj,
[0033] There are seven linear layers in total, consisting of `self_attn.v_proj`, `self_attn.q_proj`, and `self_attn.o_proj`. `mlp.*` and `self_attn.*` represent linear layers belonging to the `mlp` and `self_attn` modules, respectively. Any module B... i The linear layers in the above order are represented as B i ={L i*7+0 ,L i*7+i ,…,L i*7+6 The set of layers to be quantized in the entire target model is represented as L = {L0, L1, ..., L}. i*7+j ,…,L 223}, where 0≤i≤31, 0≤j≤6, and there are a total of 224 linear layers.
[0034] For example, the quantization device sets the expected average quantization precision of the layers to be quantized in the target model to P. e =3, the precision allocation range for each layer is P r ={p|P min ≤p≤P max ,p∈Z +}, where P min =2 and P max =4 represent the minimum and maximum precision for each layer, respectively, Z + This indicates that p are all positive integers.
[0035] In step S200, a reinforcement learning environment for a preset agent is generated based on the calibration data through the quantized layer of the target model.
[0036] For example, in step S200, based on calibration data, the quantization device calculates the KL divergence and quantization loss of the output feature distribution of the layer to be quantized through the target model, so as to construct a reinforcement learning environment for the preset agent.
[0037] Based on the calibration data, the quantization device acquires the first input features and the first output features of the layer to be quantized before quantization.
[0038] According to some embodiments, the quantization device randomly selects a preset number N (e.g., 128) of text data from the calibration data and inputs them into the target model. For the nth text data, the quantization device obtains the layer L to be quantized in the target model. l The first input feature before quantization (precision FP16, i.e., Float16, half-precision floating-point representation). and the first output feature It can be expressed by the following formula.
[0039]
[0040] Among them, H I,l / W I,l H O,l / W O,l These are the height / width of the first input feature and the height / width of the first output feature, respectively.
[0041] According to the preset first quantization rule, the quantization device quantizes the layer to be quantized with the expected quantization precision to obtain the quantized linear layer and quantization loss corresponding to the layer to be quantized.
[0042] According to some embodiments, the first quantization rule may employ the GPTQ (Gross-Platform Quantization) method. The quantization device reduces the p-value of the layer L to be quantized according to the first quantization rule as expected. l Quantization is performed to obtain the layer L to be quantized. l The corresponding quantized linear layer and quantification loss It can be expressed by the following formula.
[0043]
[0044] The quantization device obtains the quantized second output feature through the quantized linear layer and the first input feature.
[0045] According to some embodiments, the quantization device passes through the quantized linear layer and the first input features Perform calculations and obtain the second output feature after quantization of the layer to be quantized. It can be expressed by the following formula.
[0046]
[0047] Based on the first and second output features, the quantization device calculates the KL divergence of the output feature distribution of the layer to be quantized.
[0048] According to some embodiments, the quantization device can calculate the layer L to be quantized using the following formula. l KL divergence of the output feature distribution before and after quantization
[0049]
[0050] For a preset quantity of text data of number N, the quantization layer L l KL divergence value It can be expressed by the following formula.
[0051]
[0052] The quantization device constructs a reinforcement learning environment for a pre-defined agent based on the KL divergence and quantization loss of the layer to be quantized.
[0053] According to some embodiments, the quantization device quantizes the layer L to be quantized. l KL divergence and quantification loss Obtain the KL divergence lookup table (LUT) corresponding to the set of layers L to be quantized in the target model. kl And Quantization Loss Lookup Table (LUT) gl It can be expressed by the following formula.
[0054]
[0055] According to some embodiments, the quantization device is based on a KL divergence lookup table (LUT). kl And Quantization Loss Lookup Table (LUT) gl Construct a reinforcement learning environment for a pre-defined intelligent agent.
[0056] In step S300, a preset agent is trained based on a reinforcement learning environment to obtain the first quantization accuracy allocation data of the target model.
[0057] For example, in step S300, the quantization device sets the relevant parameters of the preset agent and trains the preset agent based on the reinforcement learning environment to obtain the first quantization accuracy allocation data of the target model.
[0058] Before training the preset agent, the quantization device sets the relevant parameters of the preset agent in the reinforcement learning environment.
[0059] According to some embodiments, DDPG (Deep Deterministic Policy Gradient) can be used as a reinforcement learning framework for a pre-defined agent.
[0060] According to some embodiments, the relevant parameters of the preset agent include the behavior of the preset agent, the state space of the corresponding layer to be quantized, the corresponding reward mechanism, and the number of training rounds. In the embodiments of this application, the behavior of the preset agent includes predicting the quantization accuracy of the layer to be quantized in the target model.
[0061] For example, for the target model's unquantized layer L l l = i * 7 + j, where the quantization device sets the behavior a of the preset intelligent agent. l ={a|a∈P r}, and its corresponding unquantized layer L l The state space is initialized to s l ={i,j,H I,l H O,l ,is_attn,b l}, where i represents module B to which the current layer belongs. i j indicates that the current layer is module B. i The j-th linear layer, H I,l H represents the number of input channels in the current layer. O,l This indicates the number of output channels for the current layer, and `is_attn` indicates whether the current layer belongs to the `self_attn` module. l This represents the quantization precision of the current layer, initialized to the expected average quantization precision P. e =3. The value of is_attn can be expressed by the following formula.
[0062]
[0063] For example, the quantization device sets the set of state spaces corresponding to the set of layers L to be quantized in the target model as S = {s}. l |l=7*i+j,0≤i≤31,0≤j≤6}∈R 224×6 The maximum and minimum values are normalized column-wise. The state space set S can be expressed by the following formula.
[0064]
[0065] According to some embodiments, the reward mechanism corresponding to the preset agent set by the quantization device includes global rewards and local rewards, so as to balance the relationship between the global and local aspects of the target model and improve the performance of the quantized model by combining global and local rewards during the training process of the preset agent.
[0066] The quantization device first acquires the second quantization precision allocation data of the target model output by the preset agent after a training round, and then sets the corresponding reward mechanism based on the second quantization precision allocation data.
[0067] For example, for the target model's unquantized layer L l l = i * 7 + j, with data b allocated at the second quantization precision. l In this case, the preset reward R for the agent is... l The settings can be expressed by the following formula.
[0068] R l =R global +R local,l (9)
[0069] In formula (9), R global In a reinforcement learning environment, the global reward for a pre-defined agent corresponding to a set of layers L to be quantized can be expressed by the following formula.
[0070]
[0071] Wherein, NORM(·) is the maximum-minimum normalization, and its normalization interval is [n]. min ,n max ], where n min =-5,n max =1. AVG(B i ) is the calculation module B i The average quantization precision of the layers to be quantized is used to account for the influence of the number of parameters between different layers on the average precision, and it can be expressed by the following formula.
[0072]
[0073] In formula (9), R local,l In a reinforcement learning environment, the agent is pre-defined to correspond to the current layer L to be quantized. l Module B i Local rewards, within the same module B i The local rewards are the same, which can be expressed by the following formula.
[0074]
[0075] According to some embodiments, the training phase of reinforcement learning for the preset agent includes a warm-up phase and a formal training phase. The quantization device sets the number of training rounds for the preset agent to E. t =1000, the number of rounds in the preheating stage is E w =20, current training round is E cThe maximum reward for the preset intelligent agent is R. best The optimal quantization accuracy allocation strategy obtained based on the prediction results of the preset intelligent agent is A. best .
[0076] The quantization device determines the training phase to which the current training round of the preset agent belongs.
[0077] During the warm-up phase, the quantization device trains the preset intelligent agent to perform random behavior training.
[0078] According to some embodiments, when E c ≤E w At this time, the preset agent is in the warm-up phase. In the reinforcement learning environment, the quantization device controls the preset agent to randomly select behavior in the current training round to obtain batches of experience data. Furthermore, during the warm-up phase, the preset agent does not predict the quantization accuracy of the target model's layers to be quantized.
[0079] During the formal training phase, the quantization device performs quantization accuracy prediction training on the preset intelligent agent.
[0080] According to some embodiments, when E c >E w At this time, the preset agent is in the formal training phase. In the reinforcement learning environment, the quantization device controls the preset agent in the current training round to quantize each layer L in the set L of layers to be quantized in the target model. l status Predict the next action That is, predict each layer L in the set of layers L to be quantized in the current round. l Quantization accuracy And update status The training process for predicting the quantization accuracy of a pre-defined agent can be expressed by the following formula.
[0081]
[0082] Furthermore, after the current round ends, the quantization device obtains the quantization precision set of the set of layers L to be quantized in the target model. Global reward corresponding to the preset agent exist In this case, the quantization device determines the optimal quantization accuracy allocation strategy. This is used as the first quantization precision allocation data.
[0083] According to some embodiments, when E c >E t At this point, the training process of the preset agent ends. Furthermore, the preset agent updates its parameters throughout the training process through methods such as experience replay and the target network.
[0084] In step S400, the target model is quantized according to the data allocated based on the first quantization precision.
[0085] For example, in step S400, after the training of the preset agent is completed, the quantization device quantizes the target model according to the data allocated according to the first quantization precision, and obtains the quantized target model.
[0086] According to the preset second quantization rule, the quantization device quantizes any layer to be quantized in the target model by allocating data with the first quantization precision, so as to obtain the quantized target model.
[0087] According to some embodiments, the second quantization rule can adopt the SLIM (a large model mixed precision quantization method, an improved version of GPTQ) method to avoid the problem of numerical overflow that is prone to occur at lower mixed precision.
[0088] According to some embodiments, the quantization device quantizes each layer in the set L of layers to be quantized in the target model according to a second quantization rule and a first quantization precision, and obtains the quantized target model.
[0089] According to the embodiments of this application, hierarchical mixed precision quantization for large models is realized, which improves the performance of large models under low precision quantization conditions, significantly reduces the interaction time with the environment, accelerates the training speed of reinforcement learning, balances the performance and precision allocation between local and global aspects, and improves the effectiveness of the allocation strategy.
[0090] Figure 2 A schematic diagram of a quantization apparatus for a model according to an example embodiment of this application is shown.
[0091] like Figure 2 As shown, the quantization device 100 includes a data acquisition module 110, an environment configuration module 120, an agent training module 130, and a model quantization module 140.
[0092] The data acquisition module 110 acquires the preset target model and calibration data, and sets the quantization parameters of the target model. The quantization parameters include the layer to be quantized and the expected quantization accuracy.
[0093] Based on the calibration data, the environment configuration module 120 obtains the first input features and the first output features of the layer to be quantized before quantization.
[0094] According to the preset first quantization rule, the environment configuration module 120 quantizes the layer to be quantized with the expected quantization accuracy to obtain the quantized linear layer and quantization loss corresponding to the layer to be quantized.
[0095] The environment configuration module 120 obtains the quantized second output feature using the quantized linear layer and the first input feature.
[0096] Based on the first output feature and the second output feature, the environment configuration module 120 calculates the KL divergence of the output feature distribution of the layer to be quantized.
[0097] The environment configuration module 120 constructs a reinforcement learning environment for the pre-defined agent based on the KL divergence and quantization loss of the layer to be quantized.
[0098] Before training the preset agent, the agent training module 130 sets the relevant parameters of the preset agent in the reinforcement learning environment. These parameters include the behavior of the preset agent, the state space of the corresponding layer to be quantized, the corresponding reward mechanism, and the number of training rounds.
[0099] According to some embodiments, the reward mechanism corresponding to the preset intelligent agent set by the intelligent agent training module 130 includes global rewards and local rewards.
[0100] The agent training module 130 obtains the second quantization precision allocation data of the target model output by the preset agent after a training round, and then sets the corresponding reward mechanism based on the second quantization precision allocation data.
[0101] The agent training module 130 determines the training phase to which the current training round of the preset agent belongs. The reinforcement learning training phases of the preset agent include a warm-up phase and a formal training phase.
[0102] During the warm-up phase, the agent training module 130 trains the preset agent using random behavior. During this phase, the preset agent does not perform quantization accuracy prediction for the target model's quantized layers.
[0103] During the formal training phase, the agent training module 130 performs quantization precision prediction training on the preset agent to obtain the first quantization precision allocation data and global reward.
[0104] According to the preset second quantization rule, the model quantization module 140 quantizes any layer to be quantized in the target model by allocating data with the first quantization precision, so as to obtain the quantized target model.
[0105] Figure 3 A block diagram of an electronic device according to an example embodiment of this application is shown.
[0106] like Figure 3 As shown, the electronic device 600 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0107] like Figure 3As shown, the electronic device 600 is manifested in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc. The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the methods described in this specification according to the various exemplary embodiments of this application. For example, the processing unit 610 can perform, for example... Figure 1 The method shown.
[0108] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.
[0109] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0110] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0111] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0112] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. The technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.
[0113] Software products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0114] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0115] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0116] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the aforementioned functions.
[0117] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0118] The embodiments of this application have been described in detail above. These descriptions are solely for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, its specific implementation methods, and its application scope, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for quantizing a model, characterized in that, include: Acquire the preset target model and calibration data; Based on the calibration data, a reinforcement learning environment for a preset agent is generated through the quantized layer of the target model; The preset agent is trained based on the reinforcement learning environment to obtain the first quantization precision allocation data of the target model; The target model is quantized based on the data allocated according to the first quantization precision.
2. The method according to claim 1, characterized in that, Obtain the preset target model and calibration data, including: The quantization parameters of the target model are set, including the layer to be quantized and the expected quantization precision.
3. The method according to claim 2, characterized in that, Based on the calibration data, a reinforcement learning environment for a preset agent is generated through the quantized layer of the target model, including: Based on the calibration data, the first input features and first output features of the layer to be quantized before quantization are obtained; According to the preset first quantization rule, the layer to be quantized is quantized according to the expected quantization precision to obtain the quantized linear layer and quantization loss corresponding to the layer to be quantized. The quantized second output feature is obtained by using the quantized linear layer and the first input feature; Based on the first output feature and the second output feature, calculate the KL divergence of the output feature distribution of the layer to be quantized; The reinforcement learning environment is constructed based on the KL divergence and the quantization loss.
4. The method according to claim 1, characterized in that, Before training the preset agent based on the reinforcement learning environment to obtain the first quantization precision allocation data of the target model, the method further includes: The behavior of the preset intelligent agent is set, and the behavior of the preset intelligent agent includes predicting the quantization accuracy of the layer to be quantized. Set the state space of the layer to be quantized corresponding to the preset intelligent agent; Set the reward mechanism corresponding to the preset intelligent agent; Set the training rounds for the preset intelligent agent.
5. The method according to claim 4, characterized in that, Setting the reward mechanism corresponding to the preset intelligent agent includes: Obtain the second quantization precision allocation data of the target model output by the preset agent after one training round; Based on the second quantization precision allocation data, the global reward and local reward corresponding to the preset agent in the reinforcement learning environment are set.
6. The method according to claim 5, characterized in that, The preset agent is trained based on the reinforcement learning environment to obtain the first quantization precision allocation data of the target model, including: Determine the training phase to which the current training round belongs, the training phase including the warm-up phase and the formal training phase; During the warm-up phase, the preset agent is trained to perform random behavior training in the reinforcement learning environment, wherein the random behavior training of the preset agent does not include the quantization accuracy prediction of the layer to be quantized. During the formal training phase, in the reinforcement learning environment, the preset agent is trained to predict quantization accuracy in order to obtain the first quantization accuracy allocation data and the global reward.
7. The method according to claim 1, characterized in that, The target model is quantized according to the data allocated based on the first quantization precision, including: According to the preset second quantization rule, any layer to be quantized in the target model is quantized using the first quantization precision allocation data to obtain the quantized target model.
8. A quantization device for a model, characterized in that, include: The data acquisition module is used to acquire the preset target model and calibration data; An environment configuration module is used to generate a reinforcement learning environment for a preset agent based on the calibration data and the quantization layer of the target model. The agent training module is used to train the preset agent based on the reinforcement learning environment to obtain the first quantization accuracy allocation data of the target model; The model quantization module is used to quantize the target model based on the data allocated according to the first quantization precision.
9. An electronic device, characterized in that, include: one or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-7.