Post-training quantization method based on symbology measurement
Through the post-training quantization method based on symbol metrics, the quantitative index is optimized using symbol expression tree population and evolutionary algorithms, the problem of high computational complexity of Heisen matrix in traditional technology is solved, and the efficient operation of large language models on resource-constrained devices is achieved.
Patent Information
- Application Number
- CN202411628537.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-02-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In traditional post-training quantization technology, the calculation complexity of the Heisen matrix is relatively high, especially in large language models, resulting in extremely high demand for computing resources and memory.
A post-training quantization method based on symbolometrics is adopted to obtain mathematical operators and model statistical information, symbol expression tree populations are randomly generated, and quantitative indicators are optimized through evolutionary algorithms to reduce the cost of manual intervention and trial and error.
It significantly reduces the computing and storage requirements of large language models, allowing them to operate efficiently on resource-constrained devices, and alleviates the burden of quantitative indicator calculations.
Smart Images
Figure CN119514630A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of machine learning, and in particular relates to a post-training quantization method based on symbolic metrics. Background Art
[0002] With the widespread application of deep learning models, the scale and complexity of deep learning models continue to increase. Large deep learning models require a lot of computing resources and storage space during training and reasoning, which poses a huge challenge to resource-constrained devices (such as mobile devices and embedded systems). Large language models are artificial intelligence models built using deep learning technology, especially the Transformer (a deep learning model based on the self-attention mechanism) architecture. They are obtained through large-scale data training and are designed to understand and generate human language. The emergence of large language models has further exacerbated the phenomenon of increased storage and computing costs, making it more important to reduce the cost of model training.
[0003] In order to reduce the storage and computing costs of the model, post-training quantization (PTQ) technology has been proposed and widely studied. Post-training quantization technology is a technology applied after model training is completed. It is used to quantize the parameters such as the weights and activation values of the model after the model training is completed. It aims to reduce the size of the model and improve the inference speed while maintaining the performance of the model as much as possible, so that the model is more suitable for deployment on resource-constrained devices.
[0004] In traditional post-training quantization technology, in order to help determine which parameters in the model can be compressed more significantly and have less impact on model performance, a sensitivity index is usually used as a quantitative index to measure the impact of parameter changes on model performance, so as to complete the quantization of model parameters. Existing sensitivity indicators usually use the Hessian matrix, which is a matrix composed of second-order partial derivatives that can reflect the curvature information of the model's performance under weight changes, that is, the sensitivity of the model performance to changes in weight parameters.
[0005] However, in the process of using the prior art, the inventors found that the prior art has at least the following problems: The computational complexity of the Hessian matrix in traditional post-training quantization technology is high, especially in large language models when the number of model parameters is large. n When is large, its computational complexity is , which places extremely high demands on computing resources and memory. Existing methods such as GPTQ (Gradient-based Post-Training Quantization) use an approximate Hessian matrix to reduce the computational burden, but the computational cost is still high.
[0006] To this end, it is necessary to study a quantization method that can alleviate the computational burden introduced in the quantization process. Summary of the invention
[0007] The present invention aims to solve the above technical problems at least to a certain extent, and provides a post-training quantization method based on symbolic metrics.
[0008] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a post-training quantization method based on symbolic metrics, comprising: Get mathematical operators and model statistics for a specified model; According to a preset initial tree depth and an initial population size, a symbolic expression tree population is randomly generated; wherein the symbolic expression tree population includes a plurality of individuals, and any individual is a symbolic expression tree constructed according to the model statistical information and the mathematical operator; Evaluate each individual in the symbolic expression tree population and calculate the performance index of each individual; Based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals; Generate a new individual according to the parent individual, and add the new individual to the symbolic expression tree population; Optimizing the individuals in the symbol expression tree population to obtain an optimized symbol expression tree population; Evaluate each individual in the optimized symbol expression tree population, calculate the performance index of each individual, and judge whether the optimized symbol expression tree population reaches the preset target according to the performance index of each individual. If yes, use the optimized symbol expression tree population as the final symbol expression tree population. If not, update the symbol expression tree population to the optimized symbol expression tree population, and then select two individuals with the best performance index from the symbol expression tree population as parent individuals based on the performance index of each individual, until the optimized symbol expression tree population reaches the preset target. The specified model is quantized based on the final symbolic expression tree population.
[0009] In one possible design, the model statistics include activation values, gradients, and weights; and the mathematical operators include unary operators and binary operators.
[0010] In one possible design, the model statistics include activation values, gradients, and weights; and the mathematical operators include unary operators and binary operators.
[0011] In a possible design, a symbolic expression tree is constructed according to the model statistical information and the mathematical operator, including: Using the model statistics as leaf nodes and using the model statistics as the bottom layer of a symbolic expression tree; Taking the leaf node as the current node, selecting a specified mathematical operator from the mathematical operators according to the current node, and taking the current node as input data of the specified mathematical operator to generate a new internal node; The new internal node is used as the current node to generate a new internal node again, thereby obtaining a plurality of quantitative metric expressions, and the plurality of quantitative metric expressions constitute a final symbolic expression tree.
[0012] In one possible design, the performance metric uses perplexity.
[0013] In a possible design, any individual in the symbolic expression tree population is evaluated, and a performance index of any individual is calculated, including: Quantizing the specified model according to any one of the individuals to obtain a quantized model of the specified model; Using the quantized model to perform reasoning on a predefined test set to obtain an output result of the quantized model on the test set; According to the output result, the perplexity of the quantized model is calculated, and the perplexity is used as the performance indicator of any individual.
[0014] In a possible design, based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals, and a tournament selection strategy is adopted.
[0015] In one possible design, generating new individuals based on the parent individuals includes: Randomly select a subtree from each of the parent individuals and swap the positions of the two subtrees to generate a new individual; And / or, randomly select any node of an individual from the parent individuals and replace the node with another randomly selected node to generate a new individual.
[0016] In one possible design, generating new individuals based on the parent individuals includes: Randomly select a subtree from each of the parent individuals and swap the positions of the two subtrees to generate a new individual; And / or, randomly select any node of an individual from the parent individuals and replace the node with another randomly selected node to generate a new individual.
[0017] In one possible design, the adversarial operation pairs include square and square root operations, exponential and logarithmic operations, sine and arcsine operations, and cosine and arccosine operations that constitute mutually inverse operations.
[0018] In a second aspect, the present invention provides an electronic device, comprising: a memory for storing computer program instructions; and, A processor is used to execute the computer program instructions to complete the operation of a post-training quantization method based on symbolic metric as described in any one of the above.
[0019] In a third aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements a post-training quantization method based on symbolic metrics as described in any one of the above.
[0020] The beneficial effects of the present invention are: The present invention can alleviate the burden of calculating quantitative indicators in the process of model quantization, significantly reduce the computational and storage requirements of large language models, and enable them to run efficiently on resource-constrained devices. Specifically, during the implementation of the present invention, the mathematical operators and model statistical information of the specified model are first obtained, and a symbolic expression tree population is randomly generated according to a preset initial tree depth and an initial population size; then, each individual in the symbolic expression tree population is evaluated separately, and the performance indicators of each individual are calculated, and then based on the performance indicators of each individual, two individuals with the best performance indicators are selected from the symbolic expression tree population as parent individuals; then, new individuals are generated based on the parent individuals, and the new individuals are added to the symbolic expression tree population, and then the individuals in the symbolic expression tree population are optimized to obtain an optimized symbolic expression tree population; then, the individuals in the symbolic expression tree population are optimized. Each individual in the optimized symbolic expression tree population is evaluated separately, and the performance index of each individual is calculated. According to the performance index of each individual, it is judged whether the optimized symbolic expression tree population reaches the preset target. If so, the optimized symbolic expression tree population is used as the final symbolic expression tree population. If not, the symbolic expression tree population is updated to the optimized symbolic expression tree population. Then, based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals until the optimized symbolic expression tree population reaches the preset target. Finally, the specified model is quantized based on the final symbolic expression tree population. In the present invention, a search framework for model quantization metrics is constructed based on mathematical operators and model statistical information, and symbolic learning is adopted to expand the quantization metric expression of the model through an evolutionary algorithm, so that better quantization indicators can be obtained, which reduces the cost of manual intervention and trial and error, and improves the search efficiency of quantization indicators and the reliability of results in the subsequent model quantization process. In addition, since the present invention does not introduce second-order partial derivatives, it has certain advantages in calculation and memory usage.
[0021] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is a flowchart of the post-training quantization method based on symbolic metric in Example 1; Figure 2 This is a module block diagram of an electronic device in Example 2. DETAILED DESCRIPTION
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.
[0024] Embodiment 1: This embodiment discloses a post-training quantization method based on symbolic metrics, which can be executed by, but is not limited to, a computer device or a virtual machine with certain computing resources, such as a personal computer, a smart phone, a personal digital assistant, or a wearable device, or by a virtual machine.
[0025] like Figure 1 As shown, a post-training quantization method based on symbolic metrics may include but is not limited to the following steps: S1. Obtain mathematical operators and model statistical information of the specified model; it should be noted that in this embodiment, the mathematical operators and model statistical information constitute a search space to cover existing quantitative indicators and expand to more complex quantitative measurement combinations, thereby greatly increasing the breadth and depth of quantitative measurement of the model, so that various complex quantitative measurement results of the model can be generated and evaluated based on this embodiment.
[0026] In step S1, the model statistical information includes activation values, gradients and weights; and the mathematical operators include unary operators and binary operators.
[0027] Specifically, the activation value can be represented by the letter X, which represents the activation value of each layer in the specified model. These activation values X can be obtained by calibrating using Wikitext2 data (data in the Wikitext2 public data set) with a batch size of 128, reflecting the intermediate output of the specified model when processing actual input data; the gradient can be represented by the letter G, which is the gradient calculated by the specified model during the back propagation process, and can also be obtained by calibrating using Wikitext2 data with a batch size of 128, and is used to measure the direction and magnitude of network parameter adjustment; the weight can be represented by the letter W, which is the connection weight of the specified model, which directly determines the output result of the specified model.
[0028] In addition, in this embodiment, the unary operator includes mathematical operators such as absolute value, square, square root, logarithm and exponent, and the unary operator only operates on a single input, such as taking the absolute value or square of a certain weight W; the binary operator includes mathematical operators such as addition, subtraction, multiplication and division, and the binary operator requires two inputs, such as multiplying or adding two weights W.
[0029] S2. Randomly generate a symbolic expression tree population according to a preset initial tree depth and an initial population size; wherein the symbolic expression tree population includes a plurality of individuals, and any individual is a symbolic expression tree constructed according to the model statistical information and the mathematical operator; it should be noted that each symbolic expression tree species in the symbolic expression tree population can be called an individual, and the symbolic expression tree population is randomly generated according to the initial tree depth and the initial population size, that is, a symbolic expression tree population whose number of randomly generated individuals is the same as the number of the initial population, and whose depth of each individual is consistent with the initial tree depth, and in any individual, model statistical information or mathematical operators are randomly selected on each node to construct a complete symbolic expression tree.
[0030] In step S2, the initial tree depth is 3 to 5 layers; the initial population size is 50.
[0031] It should be noted that, in this embodiment, the initial tree depth is set to 3 to 5 layers, which can ensure that the symbol expression trees in the generated symbol expression tree population have sufficient expression complexity and are computationally feasible. Specifically, when generating each symbol expression tree, a depth value d is randomly selected and satisfies 3≤d≤5, and then the tree structure is constructed according to the depth to ensure that the length of all paths in the generated symbol expression tree does not exceed d.
[0032] In addition, in this embodiment, the initial population size is 50, that is, the size of the symbolic expression tree population is 50 individuals. This size can not only ensure the diversity of the population, but also perform effective evolution operations within the scope allowed by computing resources.
[0033] It should also be noted that in the process of generating the symbolic expression tree population, the present embodiment can use the symbolic expression trees in the existing SparseGPT and Wanda schemes. The symbolic expression trees in these schemes can be used as high-quality individuals in the symbolic expression tree population, providing a good starting point. The remaining individuals can be randomly generated according to the model statistical information and the mathematical operators to ensure that there are a variety of different quantitative metric expressions in the symbolic expression tree population, increasing the possibility of subsequent evolution. Specifically, the symbolic expression tree in the SparseGPT scheme is , the symbolic expression tree in the Wanda scheme is .
[0034] Specifically, in step S2, a symbolic expression tree is constructed according to the model statistical information and the mathematical operator, including: S201. Use the model statistical information as leaf nodes and use the model statistical information as the bottom layer of the symbolic expression tree; specifically, in this embodiment, the activation value X, gradient G and weight W are used as leaf nodes and used as the bottom layer of the symbolic expression tree, which represent the basic input data of the quantitative indicators.
[0035] S202. Take the leaf node as the current node, select a specified mathematical operator from the mathematical operators according to the current node, and use the current node as input data of the specified mathematical operator to generate a new internal node; it should be noted that the specified mathematical operator is selected from the mathematical operators and the specified mathematical operator is used as the internal node, and the internal node is used to combine and convert the output of the leaf node or other internal nodes.
[0036] As an example, when the current node is a leaf node, according to the gradient G and weight W in the leaf node, the absolute value operator |∙| is selected from the mathematical operators as the designated mathematical operators, and the gradient G and weight W are used as their input data, that is, the absolute value operator |∙| acts on the gradient G and the weight W, respectively, so as to generate new internal nodes |G| and |W|.
[0037] It should be noted that in this embodiment, mathematical operators can be repeatedly selected according to the current node, and multiple new internal nodes are generated based on the current node. For example, when the current node is the gradient G in the leaf node, the absolute value operator |∙| and the square root operator can be repeatedly selected. , the new internal nodes generated at this time include |G| and .
[0038] S203. Take the new internal node as the current node, that is, return to step S202, so as to generate a new internal node again, and then obtain multiple quantitative metric expressions, which constitute the final symbolic expression tree. As an example, when the current nodes in this step are |G| and |W|, if the multiplication operator × is selected from the mathematical operators as the specified mathematical operator again, the internal node is used as the input data of the currently selected specified mathematical operator, that is, the internal nodes |W| and |G| are combined by the multiplication operator × to generate a new internal node |W|×|G| again; after this, step S202 can be executed again to obtain the final quantitative metric expression, and the number of current nodes can be one or more. As an example, after forming a new internal node |W|×|G|, the square root operator can be further selected from the mathematical operators according to the internal node |G|. , a quantitative measurement expression can be generated , a quantitative index of the specified model can be obtained according to the quantitative measurement expression.
[0039] It should be noted that, in this embodiment, based on steps S201 to S203, the symbolic expression tree is expanded layer by layer, and the leaf nodes are located at the bottom of the tree, and the internal nodes are connected upward step by step, and finally a complete tree structure is formed. Through the symbolic expression tree, this embodiment can flexibly combine different internal nodes and mathematical operators, and then generate a variety of possible quantitative indicators for realizing quantitative measurement of the specified model.
[0040] S3. Evaluate each individual in the symbolic expression tree population separately, and calculate the performance index of each individual.
[0041] In this embodiment, the performance indicator adopts perplexity. Specifically, in step S3, any individual in the symbol expression tree population is evaluated, and the performance indicator of any individual is calculated, including: S301. quantize the specified model according to any one of the individuals to obtain a quantized model of the specified model; S302. Use the quantized model to perform reasoning on a predefined test set to obtain an output result of the quantized model on the test set; S303. According to the output result, the perplexity of the quantized model is calculated, and the perplexity is used as the performance indicator of any individual.
[0042] It should be noted that perplexity is a common indicator for measuring the performance of a large language model. The lower the value, the better the model performance, which means the better the performance of the corresponding individual. In this embodiment, perplexity is used as the performance indicator of the individual, and the calculation method is simple and representative.
[0043] S4. Based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals.
[0044] In step S4, based on the performance indicators of each individual, when two individuals with the best performance indicators are selected from the symbolic expression tree population as parent individuals, the tournament selection strategy is adopted. Specifically, the selection of parent individuals is implemented by adopting the tournament selection strategy, that is, a small number of individuals are randomly selected from the symbolic expression tree population as candidate individuals, and the two individuals with the best performance, that is, the individuals with the lowest perplexity, are selected as parent individuals. This approach can not only ensure that the parent individuals are high-performance individuals, but also meet the diversity requirements to a certain extent.
[0045] S5. Generate a new individual based on the parent individual, and add the new individual to the symbolic expression tree population; based on this step, the individuals in the symbolic expression tree population can be expanded to explore a wider search space.
[0046] Specifically, in step S5, generating a new individual according to the parent individual includes: Randomly select a subtree from each of the parent individuals and swap the positions of the two subtrees to generate a new individual; And / or, randomly select any node of an individual from the parent individuals and replace the node with another randomly selected node to generate a new individual.
[0047] S6. Optimize the individuals in the symbol expression tree population to obtain an optimized symbol expression tree population; it should be noted that in the process of generating and expanding the symbol expression tree population, redundant or adversarial operation pairs may be generated, resulting in reduced availability of individuals in the symbol expression tree population. Therefore, this embodiment further optimizes the symbol expression tree population after expanding it.
[0048] In step S6, the individuals in the symbol expression tree population are optimized to obtain an optimized symbol expression tree population, including: S601. Perform adversarial operation identification processing on each individual in the symbolic expression tree population to obtain adversarial operation pairs in the symbolic expression tree population; specifically, the adversarial operation pairs include square and square root operations, exponential and logarithmic operations, sine and arcsine operations, and cosine and arccosine operations that constitute mutually inverse operations. These adversarial operation pairs are mathematically inverse, and continuous application may lead to redundancy or invalidation of expressions. Specifically, in this embodiment, a recursive traversal method is used to check each node of each individual in the symbolic expression tree population to identify such as exp(log(x)) or sqrt{x 2} and other inverse operations.
[0049] S602. Simplify the adversarial operation pairs to obtain an optimized symbolic expression tree population. Specifically, in this embodiment, after identifying the adversarial operation pairs, these adversarial operation pairs are replaced with simplified forms according to preset simplification rules, such as simplifying exp(log(x)) to x, or replacing sqrt{x 2} is simplified to the absolute value |x|. In this way, the redundant operations in each symbolic expression tree in the symbolic expression tree population are effectively eliminated, making the overall structure more concise, thereby reducing the complexity of the symbolic expression tree population and improving the subsequent quantitative search efficiency. In addition, the simplified symbolic expression tree not only reduces the consumption of computing resources, but also avoids the problem of model performance degradation or behavior uncertainty caused by adversarial operations.
[0050] S7. Evaluate each individual in the optimized symbol expression tree population, calculate the performance index of each individual, and judge whether the optimized symbol expression tree population reaches the preset target based on the performance index of each individual. If yes, use the optimized symbol expression tree population as the final symbol expression tree population. If not, update the symbol expression tree population to the optimized symbol expression tree population, and then select two individuals with the best performance index from the symbol expression tree population as parent individuals based on the performance index of each individual, that is, return to step S4 until the optimized symbol expression tree population reaches the preset target. It should be noted that in this embodiment, the performance index of any individual includes perplexity and its performance on multiple downstream tasks (such as GSM8K and other data sets), so as to comprehensively measure the influence of the quantitative measurement expression corresponding to the symbol expression tree in the optimized symbol expression tree population on the model performance through these multi-dimensional performance test results, and ensure that the quantitative measurement result of the selected symbol expression tree can achieve the improvement of model calculation efficiency while maintaining the model accuracy.
[0051] S8. Quantize the specified model based on the final symbolic expression tree population. It should be noted that in this process, that is, after the specified model is trained using the symbolic expression trees in the final symbolic expression tree population, the trained model is parameter quantized, thereby reducing the size of the model and improving the inference speed while maintaining the performance of the model as much as possible.
[0052] This embodiment can alleviate the burden of calculating quantitative indicators during the model quantization process, significantly reduce the computational and storage requirements of large language models, and enable them to run efficiently on resource-constrained devices. Specifically, during the implementation of this embodiment, the mathematical operators and model statistical information of the specified model are first obtained, and a symbolic expression tree population is randomly generated according to the preset initial tree depth and initial population size; then, each individual in the symbolic expression tree population is evaluated separately, and the performance indicators of each individual are calculated, and then based on the performance indicators of each individual, two individuals with the best performance indicators are selected from the symbolic expression tree population as parent individuals; then, new individuals are generated based on the parent individuals, and the new individuals are added to the symbolic expression tree population, and then the individuals in the symbolic expression tree population are optimized to obtain the optimized symbolic expression tree population; thereafter, Each individual in the optimized symbolic expression tree population is evaluated separately, and the performance index of each individual is calculated. According to the performance index of each individual, it is judged whether the optimized symbolic expression tree population reaches the preset target. If yes, the optimized symbolic expression tree population is used as the final symbolic expression tree population. If not, the symbolic expression tree population is updated to the optimized symbolic expression tree population. Then, based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals until the optimized symbolic expression tree population reaches the preset target. Finally, the specified model is quantized based on the final symbolic expression tree population. In this embodiment, a search framework for model quantization metrics is constructed based on mathematical operators and model statistical information, and symbolic learning is adopted to expand the quantization metric expression of the model through an evolutionary algorithm, so that better quantization metrics can be obtained, which reduces manual intervention and trial and error costs, and improves the search efficiency of quantization metrics and the reliability of results in the subsequent model quantization process. In addition, since the second-order partial derivative is not introduced in this embodiment, it has certain advantages in both calculation and memory usage.
[0053] Embodiment 2: Based on the first embodiment, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a laptop computer, or a desktop computer. The electronic device may be called a user terminal, a portable terminal, a desktop terminal, etc. Figure 2 As shown, the electronic equipment includes: a memory for storing computer program instructions; and, A processor is used to execute the computer program instructions to complete the operation of a post-training quantization method based on symbolic metric as described in any one of Embodiment 1.
[0054] Specifically, the processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen.
[0055] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement the post-training quantization method based on symbolic metrics provided in Example 1 of the present application.
[0056] In some embodiments, the terminal may further optionally include: a communication interface 303 and at least one peripheral device. The processor 301, the memory 302 and the communication interface 303 may be connected via a bus or a signal line. Each peripheral device may be connected to the communication interface 303 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 304, a display screen 305 and a power supply 306.
[0057] The communication interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0058] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with a communication network and other communication devices through electromagnetic signals.
[0059] The display screen 305 is used to display a UI (User Interface). The UI may include any combination of graphics, text, icons, and videos.
[0060] The power supply 306 is used to supply power to various components in the electronic device.
[0061] Embodiment 3: Based on any one of Embodiments 1 to 2, this embodiment discloses a computer program product, including a computer program or an instruction, wherein when the computer program or the instruction is executed by a computer, the post-training quantization method based on symbolic metrics as described in any one of Embodiments 1 is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
[0062] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A post-training quantization method based on symbolic metrics, characterized in that: include: Get mathematical operators and model statistics for a specified model; According to a preset initial tree depth and an initial population size, a symbolic expression tree population is randomly generated; wherein the symbolic expression tree population includes a plurality of individuals, and any individual is a symbolic expression tree constructed according to the model statistical information and the mathematical operator; Evaluate each individual in the symbolic expression tree population and calculate the performance index of each individual; Based on the performance index of each individual, two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals; Generate a new individual according to the parent individual, and add the new individual to the symbolic expression tree population; Optimizing the individuals in the symbol expression tree population to obtain an optimized symbol expression tree population; Evaluate each individual in the optimized symbol expression tree population, calculate the performance index of each individual, and judge whether the optimized symbol expression tree population reaches the preset target according to the performance index of each individual. If yes, use the optimized symbol expression tree population as the final symbol expression tree population. If not, update the symbol expression tree population to the optimized symbol expression tree population, and then select two individuals with the best performance index from the symbol expression tree population as parent individuals based on the performance index of each individual, until the optimized symbol expression tree population reaches the preset target. The specified model is quantized based on the final symbolic expression tree population.
2. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: The model statistics include activation values, gradients, and weights; and the mathematical operators include unary operators and binary operators.
3. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: The initial tree depth is 3 to 5 layers; the initial population size is 50.
4. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: A symbolic expression tree is constructed according to the model statistical information and the mathematical operator, including: Using the model statistics as leaf nodes and using the model statistics as the bottom layer of a symbolic expression tree; Taking the leaf node as the current node, selecting a specified mathematical operator from the mathematical operators according to the current node, and taking the current node as input data of the specified mathematical operator to generate a new internal node; The new internal node is used as the current node to generate a new internal node again, thereby obtaining a plurality of quantitative metric expressions, and the plurality of quantitative metric expressions constitute a final symbolic expression tree.
5. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: The performance indicator adopts perplexity.
6. A post-training quantization method based on symbolic metrics according to claim 5, characterized in that: Evaluate any individual in the symbolic expression tree population and calculate the performance index of any individual, including: Quantizing the specified model according to any one of the individuals to obtain a quantized model of the specified model; Using the quantized model to perform reasoning on a predefined test set to obtain an output result of the quantized model on the test set; According to the output result, the perplexity of the quantized model is calculated, and the perplexity is used as the performance indicator of any individual.
7. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: Based on the performance index of each individual, when two individuals with the best performance index are selected from the symbolic expression tree population as parent individuals, a tournament selection strategy is adopted.
8. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: Generating a new individual according to the parent individual, comprising: Randomly select a subtree from each of the parent individuals and swap the positions of the two subtrees to generate a new individual; And / or, randomly select any node of an individual from the parent individuals and replace the node with another randomly selected node to generate a new individual.
9. The post-training quantization method based on symbolic metrics according to claim 1, characterized in that: Optimizing the individuals in the symbol expression tree population to obtain an optimized symbol expression tree population includes: Performing adversarial operation identification processing on each individual in the symbol expression tree population to obtain adversarial operation pairs in the symbol expression tree population; The adversarial operation pairs are simplified to obtain an optimized symbolic expression tree population.
10. The post-training quantization method based on symbolic metrics according to claim 9, characterized in that: The antagonistic operation pairs include square and square root operations, exponential and logarithmic operations, sine and inverse sine operations, and cosine and inverse cosine operations that constitute mutually inverse operations.