Quantitative perception training method and device, equipment and storage medium

Abstract

The invention provides a quantitative perception training method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, deep learning, quantitative perception training and the like. According to the specific implementation scheme, quantization operation is performed on a fusion weight of a target linear layer of a pre-training model to obtain a quantization weight of the target linear layer; wherein the fusion weight is obtained by fusing the original weight of the target linear layer and the to-be-optimized weight of the low-rank adapter; and based on the quantization weight of the target linear layer, performing quantization perception training on the pre-training model so as to optimize the weight to be optimized under the condition that the original weight of the target linear layer is frozen.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as large models, deep learning, and quantitative perception training. Background Art

[0002] In the actual application and deployment of large models, quantization technology plays a vital role. By implementing this technology, not only can the memory occupied by large models during inference calculations be greatly reduced, but the time required for the inference process can also be effectively shortened, thereby significantly reducing the cost and resource consumption of large model deployment. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for quantitative perceptual training.

[0004] According to one aspect of the present disclosure, a quantization-aware training method is provided, comprising:

[0005] Quantize the fusion weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fusion weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter;

[0006] Based on the quantized weights of the target linear layer, the pre-trained model is trained quantization-aware to optimize the weights to be optimized while freezing the original weights of the target linear layer.

[0007] According to another aspect of the present disclosure, a quantization perception training device is provided, comprising:

[0008] A fusion module is used to quantize the fusion weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fusion weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter;

[0009] An optimization module is used to perform quantization-aware training on a pre-trained model based on the quantized weights of a target linear layer, so as to optimize the weights to be optimized while freezing the original weights of the target linear layer.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0014] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0018] Figure 1 is a flowchart of a quantization-aware training method provided according to an embodiment of the present disclosure;

[0019] Figure 2 It is a schematic diagram of a process of performing quantization-aware training on a pre-trained model according to an embodiment of the present disclosure;

[0020] Figure 3 is a schematic diagram of a framework of a quantization-aware training method provided according to an embodiment of the present disclosure;

[0021] Figure 4 is a structural diagram of a quantization perception training device provided according to an embodiment of the present disclosure;

[0022] Figure 5 It is a block diagram of an electronic device used to implement the quantization-aware training method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0024] In related technologies, the quantization methods commonly used in model quantization training can generally be divided into two methods: post-training quantization (PTQ, Post-Training Quantization) and quantization-aware training (QAT, Quantization-Aware Training).

[0025] Among them, PTQ is a quantization method performed after model training is completed. Based on the floating-point version of the model that has been trained, the weights and activation functions in the model are directly converted from high precision to low precision. The quantization process of PTQ includes scaling and truncation, converting floating-point values ​​to integers and restoring the mapping relationship of precision. Due to direct quantization, the PTQ method may cause high-precision models to lose precision during the quantization process, especially models and tasks that are sensitive to precision, and the screening requirements for the PTQ calibration set are high.

[0026] QAT is a training method that simulates the effects of quantization during training. Quantization operations are introduced during model training, which makes the model "aware" of the existence of quantization during training, and the final trained model itself is quantized. Because the model has taken the effects of quantization into account during training, the accuracy of the quantized model is usually better than PTQ, especially on complex models. However, QAT requires additional simulations during training, and the hardware resources required for model training are much higher than PTQ.

[0027] In view of this, an embodiment of the present disclosure provides a quantization-aware training method. The method effectively reduces the video memory resources required for quantization-aware training and the quantization loss in the quantization process by introducing low-rank adaptation (LoRA) training in large model quantization-aware training.

[0028] like Figure 1 As shown, it mainly includes the following contents:

[0029] S101, performing a quantization operation on the fusion weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fusion weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter.

[0030] In the disclosed embodiment, the pre-trained model is a model pre-trained on large-scale data. It includes multiple layers, and these layers have learned certain "knowledge" after pre-training.

[0031] The pre-trained model can be a large language model. A large language model (LLM) is a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. A large language model can handle a variety of natural language tasks, such as text classification, question answering, and dialogue.

[0032] The pre-trained model can also be a multimodal large model. A multimodal large model (MMLM) is an artificial intelligence model that can process multiple types of data such as text, images, audio, and video, integrate information through cross-modal learning, joint understanding, and generation, and work together in the input layer, fusion mechanism, and output layer. It has been trained in stages such as pre-training, instruction fine-tuning, and alignment fine-tuning. Compared with traditional single-modal models, it has more advantages in terms of task adaptability and other aspects.

[0033] Among them, the linear layer in the pre-trained model is a basic neural network layer structure, which can play the role of linear transformation of input data in the pre-trained model. The weight matrix of the linear layer determines how to transform the input. The target linear layer of quantization-aware training is any linear layer among the multiple linear layers in the pre-trained model. That is, each linear layer in the multiple linear layers is a target linear layer. Since each linear layer is processed in the same way, a target linear layer is used as an example for explanation in the embodiment of the present disclosure.

[0034] The fused weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter. The original weights are the weights learned by the pre-trained model in the pre-training stage, that is, the weights learned by the target linear layer through pre-training. The low-rank adapter is a structure that fine-tunes or improves the pre-trained model. The low-rank structure has a relatively small number of parameters. The weights to be optimized of the low-rank adapter are relatively small parameters that exist in the low-rank adapter during the fine-tuning of the pre-trained model. The model output is adjusted by fusing with the original weights of the target linear layer, so that in the process of quantization-aware training, the parameters of the low-rank adapter are optimized to reduce the hardware resources occupied by quantization-aware training, and at the same time, the accuracy loss after quantization can be minimized.

[0035] S102, based on the quantized weights of the target linear layer, performing quantization-aware training on the pre-trained model to optimize the weights to be optimized while freezing the original weights of the target linear layer.

[0036] In the disclosed embodiment, after the original weights of the target linear layer and the weights to be optimized of the low-rank adapter are fused, the fused weights are then subjected to quantization-aware training. This process retains the original pre-trained weights to the maximum extent, can ensure the accuracy of the pre-trained model and improve the convergence speed of quantization-aware training. By training the weights to be optimized of LoRA to compensate for the quantization loss of the original model, the implementation threshold of quantization-aware training of the pre-trained model is lowered by leveraging the low parameter count characteristics of LoRA. Since the LoRA parameter count is usually much smaller than the weight of the pre-trained model, the memory occupied by the quantization-aware training of the model can be significantly reduced.

[0037] Assume that the pre-trained weight W has d×h parameters and LoRA has (d+h)×r parameters. When d=h=4096, the original weight parameter is 16777216. Generally, r is a small value between 8-64. Assuming that r=32, the LoRA parameter is 262144. When using LoRA quantization training, the model can only train about 1.5% of the original model. As a result, the memory usage during quantization-aware training can be greatly reduced.

[0038] In the disclosed embodiment, since the original weights of the target linear layer and the weights to be optimized of the low-rank adapter are both in the form of matrices, the fused weights can be obtained by performing a matrix addition operation on the original weights of the target linear layer and the weights to be optimized of the low-rank adapter.

[0039] In the disclosed embodiment, by performing a matrix addition operation on the original weights of the target linear layer and the weights to be optimized of the low-rank adapter, the fused weights obtained integrate the information of the weights of the original linear layer and the weights of the low-rank adapter. Thus, the basic feature transformation capability of the original target linear layer is reduced as little as possible, and the low-rank adapter is integrated to offset the quantization loss. While retaining the accuracy of the pre-trained model to a great extent, the memory occupied by quantization-aware training can be reduced, and the convergence speed of quantization-aware training can be improved by relying on the accuracy of the original weights of the target linear layer.

[0040] In the disclosed embodiment, there are multiple linear layers in the pre-trained model. Among the multiple linear layers, each linear layer serves as the target linear layer, and each linear layer performs a quantization operation based on its own preset precision.

[0041] That is, different linear layers may be pre-set with different quantization precisions according to their functions in the pre-trained model, the characteristics of the input and output data, and the requirements for accuracy. For example, for some intermediate linear layers that have little impact on the final output results, lower precision, such as 8-bit integers, can be used for quantization to reduce the amount of calculation and memory usage to a greater extent; while for linear layers close to the output layer or those that are more critical to the accuracy of the results, higher precision, such as 16-bit integers, may be preset.

[0042] Therefore, in the embodiment of the present disclosure, for multiple linear layers, quantization operations are performed with respective preset precisions, and the functions, data relationships, and performance requirements of the entire model of each linear layer can be comprehensively considered to reasonably preset the quantization precision of each linear layer. A balance can be achieved between storage, computing efficiency, and model precision, so that while reducing memory usage, quantization-aware training can be flexibly performed for different needs.

[0043] In the disclosed embodiment, the training loss is used to optimize the weights to be optimized of the low-rank adapter. Figure 2 As shown, including the following:

[0044] S201, based on the quantized weights of the target linear layer, obtain the training loss of the pre-trained model.

[0045] When implementing, it can be achieved through the following steps:

[0046] Step A1, performing a quantization operation on the input information of the target linear layer to obtain a quantized input;

[0047] Step A2, obtaining the output result of the target linear layer based on the quantized input and the quantized weight;

[0048] Step A3: Based on the output result, the training loss of the pre-trained model is obtained.

[0049] That is, first, the quantization operator is inserted into the input information of the target linear layer to calculate the input information containing the quantization loss. Then, the quantization weight containing the simulated quantization loss and the quantization input are matrix multiplied to obtain the final output result. The quantization training logic of each target linear layer is shown in expression (1):

[0050] O=Q(x)Q(W+AB) (1)

[0051] In expression (1), O is the final output result of the target linear layer; Q(x) is the quantized input obtained after the input information x is quantized; Q(W+AB) is the quantized weight obtained by quantizing the fusion weight (W+AB) of the target linear layer of the pre-trained model.

[0052] It is understandable that the quantization operator for quantizing the input information x and the quantization operator for quantizing the fusion weights may be the same or different, and may be flexibly determined according to actual conditions. The quantization accuracy of each quantization operation depends on its own preset value.

[0053] In the pre-trained model, multiple linear layers respectively execute the quantization process of expression (1). Finally, after sequential processing of each network layer of the pre-trained model, the final output result of the pre-trained model is obtained through the forward process. Based on the final output result, the training loss of the pre-trained model can be obtained.

[0054] In the disclosed embodiment, the output result of the target linear layer is obtained by quantizing the input and the weight, and the training loss is calculated based on this result. During the quantization-aware training process, the pre-trained model can learn the quantization loss caused by the quantization input and the quantization fusion weight. Based on this loss, the weights to be optimized of the low-rank adapter can be better optimized, so as to reduce the memory consumption of the pre-trained model as much as possible through quantization-aware training, and improve the inference speed and inference accuracy of its inference process.

[0055] S202, optimizing the weights to be optimized of the target linear layer based on the training loss.

[0056] That is, by minimizing the training loss, the weights to be optimized of the target linear layer are optimized. When implemented, this can be achieved through a gradient descent algorithm or other optimization methods. For example, in each iteration, the gradient of the training loss with respect to the quantized weights is calculated, and then the weights are updated in the negative direction of the gradient. Through multiple iterations, a set of quantized weights is obtained to minimize the training loss.

[0057] In the disclosed embodiment, the training loss of the pre-trained model is obtained based on the quantization weight of the target linear layer, which can be used to measure the difference between the predicted result of the pre-trained model and the actual result. Based on the training loss, the weights to be optimized of the target linear layer are optimized, which can reduce the quantization error generated during the quantization process to a certain extent, thereby achieving the goal of reducing memory usage while maximizing the accuracy of the pre-trained model.

[0058] In some embodiments, since the quantization-aware training of the linear layer will change the output distribution of the linear layer, and the normalization layer in the original pre-trained model is a normalization calculation for the output distribution before quantization, this will undoubtedly bring about certain distribution differences and weaken the role of the normalization layer. Therefore, in order to further improve the effect of quantization-aware training, the original weights of the normalization layer of the pre-trained model can be optimized during the quantization-aware training of the pre-trained model. Thus, by not freezing the normalization layer of the backbone network, it can be trained in coordination with the LoRA parameters to better adapt to the quantized output distribution, thereby accelerating the convergence of the pre-trained model and the final training effect.

[0059] There are many types of normalization layers in the pre-trained model, each of which has its own specific application scenarios and advantages. For example, batch normalization, layer normalization, group normalization, instance normalization, SyncBatchNorm, etc. During implementation, a suitable normalization layer can be selected according to the specific network structure, batch size, and task requirements, which is not limited in the embodiments of the present disclosure.

[0060] Although normalization layers play a vital role in pre-trained models, the number of normalization layers is often relatively limited relative to the number of linear layers in pre-trained models. Therefore, even if the original weights of these normalization layers are involved in the process of quantization-aware training, the amount of parameters that need to be optimized and adjusted in the end is not very significant. Specifically, the performance improvement ratio of the pre-trained model brought about by optimizing the original weights of the normalization layer may only increase from the aforementioned reduction to 1.5% to about 2%. But compared with the original method, this still greatly reduces the memory occupied during quantization-aware training.

[0061] In addition, when a large language model is included in the pre-trained model, during the reasoning process of the large language model, as the context cache increases and the input text becomes longer and longer, the key-value cache (CacheKV) ​​during model reasoning will also become larger and larger, occupying a huge memory space. Therefore, in the disclosed embodiment, during the process of quantization-aware training of the pre-trained model, the key-value pair information to be stored output by the intermediate network layer of the pre-trained model can be quantized to obtain quantized key-value pairs; thereafter, the quantized key-value pairs are stored in the memory for use by other network layers outside the intermediate network layer.

[0062] During implementation, the quantization training logic of the key-value pair is shown in expression (2):

[0063] K ′ =Q(K); V ′ =Q(V) (2)

[0064] In expression (2), K ′ represents the quantized key; K represents the key; V ′ represents the quantized value; V represents the value; Q() represents the simulated quantization process. It can be understood that the quantization accuracy of the key-value pair quantization operation is independent and can be determined according to requirements.

[0065] In the disclosed embodiment, the key-value pair information to be stored output by the intermediate network layer of the pre-trained model is simulated and quantized, so that the pre-trained model can perceive the quantization loss of this part during the training process, and can directly cache the quantized key-value pairs during inference without the need for additional offline quantization operations, thereby reducing the accuracy loss caused by the quantization process, and can reduce the memory occupancy of quantization-aware training by quantizing key-value pairs.

[0066] In summary, the overall process of the quantization perception training method provided by the present disclosure can be as follows: Figure 3 As shown:

[0067] First, we build a pre-trained model with a low-rank adapter by adding a LoRA adapter next to each linear layer of the pre-trained model. Figure 3 Only one linear layer is shown, and the other linear layers are processed in the same way). Among them, the LoRA adapter is a low-rank matrix whose weights to be optimized are much smaller than the original weights of the target linear layer of the pre-trained model. The original weights are frozen during the training process, and only the weights to be optimized of the LoRA adapter are trained.

[0068] Secondly, after building a pre-trained model with a low-rank adapter, quantization training is enabled by inserting quantization nodes in each linear layer. During implementation, the original weights of the linear layer in the pre-trained model and the weights to be optimized in the LoRA adapter can be fused, and then the fused weights obtained after fusion are quantized to obtain quantized weights containing quantization loss. At the same time, the quantization operator is inserted into the input information of the linear layer to calculate the quantized input containing the quantization loss. After that, the quantized weights and the quantized input are matrix multiplied to obtain the output result of the linear layer.

[0069] Finally, by combining the construction of a pre-trained model containing a low-rank adapter with quantization-aware training, the quantization loss of the pre-trained model is compensated by training the weights to be optimized of the LoRA adapter, thereby lowering the implementation threshold of quantization-aware training of the pre-trained model by leveraging the low parameter count of LoRA. Since the weights to be optimized of the LoRA adapter are much smaller than the original weights of the target linear layer of the pre-trained model, the cost of model quantization training can be significantly reduced. Figure 3 In the case shown, if the pre-trained weight W has d×h parameters and LoRA has (d+h)×r parameters, when d=h=4096, the original weight parameter amount is 16777216. Generally, r takes a small value between 8-64. Assuming that r=32, the LoRA parameter amount is 262144. When using LoRA quantization training, the model trainable parameter amount is only about 1.5% of the original model.

[0070] In summary, through the quantization-aware training method provided by the disclosed embodiment, high-precision activation (i.e., linear layer input), linear layer weights, and CacheKV fully quantized models can be produced with fewer training resources, effectively reducing the quantization training cost of large models and ultimately reducing the inference deployment cost of large models.

[0071] Based on the same technical concept, the embodiment of the present disclosure also proposes a quantization perception training device 400, such as Figure 4 As shown, including:

[0072] A fusion module 401 is used to perform a quantization operation on the fusion weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fusion weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter;

[0073] The optimization module 402 is used to perform quantization-aware training on the pre-trained model based on the quantized weights of the target linear layer, so as to optimize the weights to be optimized while freezing the original weights of the target linear layer.

[0074] In some embodiments, among the multiple linear layers in the pre-trained model, each linear layer serves as a target linear layer, and each linear layer performs a quantization operation based on its own preset precision.

[0075] In some embodiments, the optimization module 402 includes:

[0076] A loss subunit 4021, for obtaining a training loss of a pre-trained model based on a quantized weight of a target linear layer;

[0077] The optimization subunit 4022 is used to optimize the weights to be optimized of the target linear layer based on the training loss.

[0078] In some embodiments, the loss subunit 4021 is specifically used to:

[0079] Perform quantization operation on the input information of the target linear layer to obtain quantized input;

[0080] Get the output of the target linear layer based on the quantized input and quantized weights;

[0081] Based on the output results, the training loss of the pre-trained model is obtained.

[0082] In some embodiments, the fusion module 401 is specifically used to:

[0083] Perform a matrix addition operation on the original weights of the target linear layer and the weights to be optimized of the low-rank adapter to obtain the fused weights.

[0084] In some embodiments, the optimization module 402 is further configured to:

[0085] During quantization-aware training of a pre-trained model, the original weights of the normalization layer of the pre-trained model are optimized.

[0086] In some embodiments, a quantization module 403 is further included, which is used to:

[0087] During the quantization-aware training of the pre-trained model, the key-value pair information to be stored output by the intermediate network layer of the pre-trained model is quantized to obtain a quantized key-value pair;

[0088] Store the quantized key-value pairs in memory for use by other network layers besides the intermediate network layer.

[0089] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0090] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0091] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0092] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0093] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0094] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0095] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the quantization-aware training method. For example, in some embodiments, the quantization-aware training method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the quantization-aware training method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the quantization-aware training method in any other appropriate manner (e.g., by means of firmware).

[0096] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0097] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0098] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0099] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0100] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0101] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0102] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0103] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A quantization-aware training method, comprising: Quantizing the fused weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fused weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter; Based on the quantized weights of the target linear layer, quantization-aware training is performed on the pre-trained model to optimize the weights to be optimized while freezing the original weights of the target linear layer.

2. The method according to claim 1, wherein: Among the multiple linear layers in the pre-trained model, each linear layer serves as the target linear layer, and each linear layer performs the quantization operation based on its own preset accuracy.

3. The method according to claim 1, wherein: The performing quantization-aware training on the pre-trained model based on the quantization weight of the target linear layer includes: Obtaining a training loss of the pre-trained model based on a quantized weight of the target linear layer; Based on the training loss, the weight to be optimized of the target linear layer is optimized.

4. The method according to claim 3, wherein: The obtaining the training loss of the pre-trained model based on the quantized weight of the target linear layer includes: Performing a quantization operation on the input information of the target linear layer to obtain a quantized input; Obtaining an output result of the target linear layer based on the quantized input and the quantized weight; Based on the output result, the training loss of the pre-trained model is obtained.

5. The method according to any one of claims 1 to 4, wherein: Obtaining the fusion weight of the target linear layer includes: A matrix addition operation is performed on the original weights of the target linear layer and the weights to be optimized of the low-rank adapter to obtain the fused weights.

6. The method according to any one of claims 1 to 5, further comprising: During the quantization-aware training of the pre-trained model, the original weights of the normalization layer of the pre-trained model are optimized.

7. The method according to any one of claims 1 to 6, further comprising: During the quantization-aware training of the pre-trained model, the key-value pair information to be stored output by the intermediate network layer of the pre-trained model is quantized to obtain a quantized key-value pair; The quantized key-value pairs are stored in a memory for use by other network layers other than the intermediate network layer.

8. A quantitative perception training device, comprising: A fusion module, used to perform a quantization operation on the fusion weights of the target linear layer of the pre-trained model to obtain the quantized weights of the target linear layer; wherein the fusion weights are obtained by fusing the original weights of the target linear layer and the weights to be optimized of the low-rank adapter; An optimization module is used to perform quantization-aware training on the pre-trained model based on the quantization weights of the target linear layer, so as to optimize the weights to be optimized while freezing the original weights of the target linear layer.

9. The device according to claim 8, wherein: Among the multiple linear layers in the pre-trained model, each linear layer serves as the target linear layer, and each linear layer performs the quantization operation based on its own preset accuracy.

10. The device according to claim 8, wherein: The optimization module comprises: A loss subunit, configured to obtain a training loss of the pre-trained model based on a quantized weight of the target linear layer; An optimization subunit is used to optimize the weight to be optimized of the target linear layer based on the training loss.

11. The device according to claim 10, wherein: The loss subunit is specifically used for: Performing a quantization operation on the input information of the target linear layer to obtain a quantized input; Obtaining an output result of the target linear layer based on the quantized input and the quantized weight; Based on the output result, the training loss of the pre-trained model is obtained.

12. The device according to any one of claims 8 to 11, wherein: The fusion module is specifically used for: A matrix addition operation is performed on the original weights of the target linear layer and the weights to be optimized of the low-rank adapter to obtain the fused weights.

13. The device according to any one of claims 8 to 12, wherein the optimization module is further configured to: During the quantization-aware training of the pre-trained model, the original weights of the normalization layer of the pre-trained model are optimized.

14. The apparatus according to any one of claims 8 to 13, further comprising a quantization module, configured to: During the quantization-aware training of the pre-trained model, the key-value pair information to be stored output by the intermediate network layer of the pre-trained model is quantized to obtain a quantized key-value pair; The quantized key-value pairs are stored in a memory for use by other network layers other than the intermediate network layer.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images