A visual language large model hybrid quantization method, device and electronic device

Abstract

The present invention relates to the technical field of hybrid quantization, and in particular to a hybrid quantization method, device and electronic device for a large visual language model. The method comprises: inputting a constructed image encoder calibration set into a pre-established image encoder and a text encoder, selecting a quantization scheme from a pre-defined search space of hybrid precision quantization to quantize the image encoder after training, and calculating a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; inputting a multimodal calibration set into a large visual language model, and selecting a quantization scheme from the search space to perform hybrid precision quantization on the large visual language model, and calculating a second quantization error based on the outputs of the large visual language model before and after quantization; storing each quantization scheme and the corresponding quantization error in a hash table; and using a policy gradient-based reinforcement learning algorithm in combination with the hash table to optimize the strategy for generating a quantization scheme. The hash table is used to accelerate the search process of the quantization scheme and improve the optimization efficiency.

Description

Technical Field

[0001] The present invention relates to the field of hybrid quantization technology, and in particular to a visual language large model hybrid quantization method, device and electronic equipment. Background Art

[0002] With the continuous development of deep learning technology, the application of vision-language models in multiple fields has become more and more extensive. However, the sharp increase in the number of model parameters and the limitation of computing resources have become the main bottlenecks for improving model performance. In recent years, researchers have proposed various model compression techniques, such as pruning, knowledge distillation, and quantization, to reduce the computational complexity and storage requirements of the model. Among them, the mixed precision quantization method, as an effective model compression technology, has received widespread attention and application.

[0003] However, although the mixed-precision quantization method has significant advantages in improving model computing efficiency and resource utilization, it still faces some challenges and problems. First, in practical applications, different parts of the model have different sensitivities to quantization accuracy, and a large number of experiments and tuning are required to determine the optimal quantization scheme, and the search efficiency is low. Secondly, during model training and inference, the mixed-precision quantization method needs to process data and calculations of multiple precisions at the same time, which places high demands on the computing framework and hardware platform. Summary of the invention

[0004] The present invention provides a large-scale hybrid quantization method, device and electronic device for visual language, which are used to solve the defect of low efficiency in searching for quantization schemes in the prior art and realize efficient searching for quantization schemes. The technical solutions proposed by the present invention are as follows:

[0005] In a first aspect, the present invention provides a large-model hybrid quantization method for visual language, comprising:

[0006] Constructing a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set;

[0007] Inputting the image encoder calibration set into a pre-established image encoder and a text encoder, and selecting a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculating a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; wherein different modules in the image encoder and the visual language large model select different quantization bit widths;

[0008] Inputting the multimodal calibration set into a pre-established visual language large model, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculating a second quantization error based on the output of the visual language large model before and after quantization;

[0009] Storing each quantization scheme and the corresponding first quantization error or second quantization error in a hash table;

[0010] In combination with the hash table, a policy gradient-based reinforcement learning algorithm is used to optimize the strategy for generating a quantization scheme to obtain a target quantization scheme.

[0011] Optionally, the step of inputting the image encoder calibration set into a pre-established image encoder and a text encoder, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform post-training quantization on the image encoder, and calculating a first quantization error based on outputs of the image encoder and the text encoder before and after quantization includes:

[0012] A batch of data is randomly selected from the image encoder calibration set, input into the image encoder and the text encoder, and the semantic similarity between each pair of image and text encodings is calculated to obtain a first semantic similarity matrix; wherein a batch of data includes multiple image and text pairs;

[0013] Select a quantization scheme from a predefined mixed precision quantization search space, perform post-training quantization on the image encoder, and obtain a quantized image encoder;

[0014] Input the selected image-text pairs into the quantized image encoder and the original text encoder, calculate the semantic similarity between each pair of image-text encodings, and obtain a second semantic similarity matrix;

[0015] A first quantization error is calculated according to the first semantic similarity matrix and the second semantic similarity matrix.

[0016] Optionally, the step of inputting the multimodal calibration set into a pre-established large visual language model, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the large visual language model, and calculating a second quantization error according to outputs of the large visual language model before and after quantization includes:

[0017] A batch of data is randomly selected from the multimodal calibration set, and the image in the image-text pair is processed using an image encoder. The image encoding and text processed by the image encoder are input into the visual language model to obtain the original output.

[0018] Select a quantization scheme from a predefined mixed precision quantization search space to perform mixed precision quantization on the visual language model, input the image code processed by the image encoder and the text into the quantized visual language model to obtain a quantized output;

[0019] A second quantization error is calculated according to the original output and the quantized output.

[0020] Optionally, the combining the hash table and using a policy gradient-based reinforcement learning algorithm to optimize the strategy for generating a quantization scheme to obtain a target quantization scheme includes:

[0021] Based on the predefined state space, action space and reward function, a reinforcement learning algorithm is used to search for a quantization scheme, and a quantization error of a current quantization scheme is determined based on the hash table; wherein the quantization error of the current quantization scheme includes an image encoder quantization error and a visual language large model quantization error, and the reinforcement learning network includes a policy network and a value network;

[0022] Inputting the image encoder quantization error and the visual language large model quantization error into the reward function to obtain the reward of the current quantization scheme, and calculating the temporal difference target according to the reward of the current quantization scheme as the loss function to update the value network parameters;

[0023] Compute an estimate of the advantage function based on the temporal difference error at each time step of the value network output;

[0024] Calculating an optimization target and updating policy network parameters according to a ratio of a current quantization scheme to an old quantization scheme and an estimated value of the advantage function;

[0025] Repeat the above process of searching for quantization schemes, updating value network parameters, calculating the estimated value of the advantage function, and updating policy network parameters until the policy network converges or reaches the preset training rounds, and obtain the target quantization scheme.

[0026] Optionally, determining a quantization error of a current quantization scheme based on the hash table includes:

[0027] Searching the current quantization scheme in the hash table to determine whether the current quantization scheme exists in the hash table;

[0028] If the current quantization scheme exists in the hash table, obtaining a quantization error corresponding to the current quantization scheme;

[0029] When the current quantization scheme does not exist in the hash table, a quantization error corresponding to a quantization scheme having a minimum edit distance with the current quantization scheme is obtained from the hash table with a preset probability to obtain a quantization error corresponding to the current quantization scheme.

[0030] Optionally, the visual language model includes a self-attention module and a perceptron module; the search space of the mixed precision quantization is defined in the following manner:

[0031] During the quantization process, the search space for defining mixed precision quantization includes multiple quantization bit widths; among them, the Q and K weight matrices in the self-attention module share the same quantization bit width, the V matrix and the self-attention map in the self-attention module share the same quantization bit width, and a quantization bit width is selected for each linear layer in the perceptron module.

[0032] In a second aspect, the present invention further provides a visual language large model hybrid quantization device, comprising the following modules:

[0033] A data set construction module, used to construct a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set;

[0034] A first calculation module is used to input the image encoder calibration set into a pre-established image encoder and a text encoder, and select a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculate a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; wherein different modules in the image encoder and the visual language large model select different quantization bit widths;

[0035] A second calculation module is used to input the multimodal calibration set into a pre-established visual language large model, select a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculate a second quantization error according to the output of the visual language large model before and after quantization;

[0036] A storage module, used for storing each quantization scheme and the corresponding first quantization error or second quantization error into a hash table;

[0037] The strategy optimization module is used to optimize the strategy of generating a quantization scheme in combination with the hash table using a reinforcement learning algorithm based on a policy gradient to obtain a target quantization scheme.

[0038] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large-model hybrid quantization method for visual language as described in the first aspect above is implemented.

[0039] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the visual language large model hybrid quantization method as described in the first aspect above.

[0040] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the visual language large model hybrid quantization method as described in the first aspect above.

[0041] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows:

[0042] The method, device and electronic device for hybrid quantization of a large model of visual language provided by the present invention select a quantization scheme from a predefined search space of hybrid precision quantization to quantize the image encoder after training, and calculate the first quantization error based on the output of the image encoder and the text encoder before and after quantization; and select a quantization scheme from the search space of hybrid precision quantization to perform hybrid precision quantization on the large model of visual language, and calculate the second quantization error based on the output of the large model of visual language before and after quantization. And each quantization scheme and the corresponding first quantization error or second quantization error are stored in a hash table. In the process of optimizing the quantization scheme, the quantization error of the quantization scheme can be searched through the hash table, which significantly accelerates the search process of the quantization scheme. This mechanism allows the model to read the error value directly from the hash table when encountering an evaluated quantization scheme, avoiding repeated calculations, thereby greatly improving the efficiency of the optimization process. This method not only reduces the time cost of searching for quantization schemes, but also accelerates the deployment and practical application of the model, providing strong support for improving the practicality and efficiency of the quantization algorithm.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 This is one of the flow charts of the large-scale hybrid quantization method for visual language provided by the present invention.

[0047] Figure 2This is the second flow chart of the visual language large model hybrid quantization method provided by the present invention.

[0048] Figure 3 It is a structural schematic diagram of the visual language large model hybrid quantization device provided by the present invention.

[0049] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Traditional model quantization techniques mainly include fixed-point quantization and floating-point quantization. Fixed-point quantization can significantly reduce the storage and computing requirements of the model by converting model parameters and calculations from floating-point representation to fixed-point representation. However, while fixed-point quantization reduces computational complexity, it often leads to a significant decrease in model performance. In contrast, floating-point quantization can further reduce the computational complexity of the model with a small performance loss. However, even floating-point quantization still faces the trade-off between maintaining model performance and reducing computing resource requirements.

[0052] As a new quantization technology, the mixed precision quantization method uses different quantization accuracies in the same model to process different parts of the calculation, thereby further reducing the demand for computing resources while ensuring model performance. Specifically, the mixed precision quantization method can select different quantization accuracies at different levels or modules of the model. For example, in a large vision-language model, high-precision quantization can be used for the attention module with higher computational complexity, while low-precision quantization can be used for the feedforward network module with lower computational complexity.

[0053] In the application of large vision-language models, the mixed precision quantization method can effectively improve the computational efficiency and resource utilization of the model. For example, in the quantization process of the image encoder and text decoder, the appropriate quantization accuracy can be selected according to the computational characteristics and performance requirements of different parts, thereby minimizing the demand for computing and storage resources while ensuring model performance. This method can not only improve the computational efficiency of the model, but also significantly reduce the storage space of the model to meet the needs of more practical application scenarios.

[0054] In addition, the mixed precision quantization method has the following advantages in improving the computational efficiency and resource utilization of the visual-language model:

[0055] 1. Flexibility and scalability: The mixed precision quantization method can flexibly adjust the quantization accuracy of different parts according to the needs of different application scenarios, thereby achieving optimal computing efficiency and performance. This flexibility makes the mixed precision quantization method highly scalable and can adapt to models of different scales and complexities.

[0056] 2. Balance between performance and efficiency: By using different quantization precisions in different parts of the model, the mixed precision quantization method can significantly reduce the computational complexity and storage requirements while ensuring the performance of the model. This balance between performance and efficiency makes the mixed precision quantization method have high practical value in practical applications.

[0057] 3. Adapt to different hardware platforms: The mixed precision quantization method can flexibly adjust the quantization scheme according to the computing characteristics and resource limitations of different hardware platforms, so as to give full play to the computing power and resource utilization of the hardware platform. This makes the mixed precision quantization method highly adaptable and can achieve efficient calculation on different hardware platforms.

[0058] In practical applications, the mixed precision quantization method can also be combined with other model compression technologies, such as pruning and knowledge distillation, to further improve the computational efficiency and performance of the model. For example, after model pruning, the mixed precision quantization method can be used to quantize the pruned model, thereby further reducing the computational complexity and storage requirements while ensuring model performance. Similarly, combined with knowledge distillation technology, the performance of the mixed precision quantization model can be further improved by transferring the knowledge of the high-precision teacher model to the low-precision student model.

[0059] However, although the mixed precision quantization method has significant advantages in improving model computing efficiency and resource utilization, it still faces some challenges and problems. First, in practical applications, different model parts have different sensitivities to quantization accuracy, and a lot of experiments and tuning are required to determine the optimal quantization scheme. Secondly, during the model training and reasoning process, the mixed precision quantization method needs to process data and calculations of multiple precisions at the same time, which places high demands on the computing framework and hardware platform. Based on this, after further research and development, the present invention provides a hybrid quantization method, device and electronic device for large visual language models.

[0060] Combine the following Figure 1-Figure 4 The present invention describes the visual language large model hybrid quantization method, device and electronic device.

[0061] Reference Figure 1As shown, the visual language large model hybrid quantization method includes the following:

[0062] Step S101 : constructing a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set.

[0063] The effectiveness of the mixed precision quantization method depends largely on the quality and representativeness of the quantization calibration dataset. Therefore, it is necessary to build a high-quality quantization calibration dataset to ensure the effectiveness of the quantization method. First, a calibration dataset for quantization must be built. The quantization calibration dataset consists of two parts. The first part is the calibration set of the image encoder (hereinafter referred to as the image encoder calibration set), which contains the image-text pairs used in training. These images and texts are semantically matched. This ensures that the image encoder can maintain its original semantic understanding ability during the quantization process.

[0064] The second part is a multimodal calibration set, which contains image-text inputs used in the training of large visual-language models. These texts are usually questions or user instructions related to the image. Through these two parts of the dataset, it can be ensured that the image and language processing capabilities of the model are fully calibrated during the quantization process.

[0065] The image encoder calibration set is mainly used to calibrate and quantize the image encoder separately to ensure its performance after quantization. The multimodal calibration set is used to evaluate the performance of the entire visual language model after mixed precision quantization.

[0066] After building the quantization calibration dataset, we need to define an appropriate mixed-precision quantization search space so that we can flexibly select different quantization bit widths during the quantization process to optimize the performance and accuracy of the model. During the quantization process, the defined mixed quantization space includes multiple quantization bit widths such as INT4, INT8, and FP16.

[0067] Step S102: input the image encoder calibration set into a pre-established image encoder and a text encoder, and select a quantization bit width allocation scheme (hereinafter referred to as the quantization scheme) from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculate a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization.

[0068] After defining the search space, it is necessary to evaluate the effects of different quantization bit width combinations through statistical quantization errors, which will provide necessary data support for the subsequent optimization process. First, the image encoder calibration set is input into the pre-established image encoder and text encoder. Quantization schemes are selected from the pre-defined mixed precision quantization search space, which include different quantization bit widths, quantization methods, etc. The image encoder is quantized after training, and the output difference between the image encoder and the text encoder before and after quantization, i.e., the first quantization error, is calculated.

[0069] Step S103: input the multimodal calibration set into a pre-established visual language large model, select a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculate a second quantization error based on the output of the visual language large model before and after quantization.

[0070] The multimodal calibration set is input into the pre-established visual language model. Similarly, a quantization scheme is selected from the pre-defined mixed precision quantization search space, and the visual language model is quantized with mixed precision. The output difference of the visual language model before and after quantization is calculated, i.e., the second quantization error.

[0071] Step S104: store each quantization scheme and the corresponding first quantization error or second quantization error into a hash table.

[0072] Since the quantization error of the visual language model is determined by the quantization scheme of the image encoder and the quantization scheme of the visual language model, the quantization schemes of the two are used as keys and the quantization error is used as the value to store them in the hash table. Stored in the hash table, the next time the quantization scheme is selected, it is first queried in the hash table. If it already exists, the first quantization error corresponding to the scheme is directly called . The quantization scheme of the current visual language model and the corresponding second quantization error Stored in the hash table, the next time the quantization scheme is selected, the hash table is first searched, if it already exists, the second quantization error corresponding to the scheme is directly called , which can effectively reduce computational overhead and improve search efficiency.

[0073] Step S105 , combining the hash table, using a policy gradient-based reinforcement learning algorithm to optimize the strategy for generating a quantization scheme, and obtaining a target quantization scheme.

[0074] Combined with the hash table, a policy gradient-based reinforcement learning algorithm is used to optimize the quantization scheme. The reinforcement learning algorithm continuously tries and adjusts the quantization scheme to finally obtain the target quantization scheme. The application of the hash table is crucial in the entire quantization process. For different candidate quantization schemes, if the quantization scheme of the image encoder part in the scheme already exists in the hash table, there is no need to recalculate the first quantization error. , the corresponding first quantization error can be directly obtained from the hash table. Similarly, if the quantization scheme of the visual language large model has been recorded in the hash table, the corresponding second quantization error can be directly read from the hash table If it does not exist, the quantization error of the current scheme is calculated and stored in the hash table for the next call, or a most similar quantization scheme is determined from the hash table according to the selected quantization scheme with a certain probability, and the quantization error corresponding to the scheme is called as the quantization error of the current scheme. This step effectively speeds up the search process for the optimal quantization scheme, i.e., the above-mentioned target quantization scheme, and improves the overall efficiency.

[0075] The present invention constructs a standard data set and uses a hash table to store quantization schemes and quantization errors. The quantization error of the quantization scheme can be searched through the hash table, and different quantization schemes can be quickly evaluated and compared, which significantly accelerates the search process of the quantization scheme, thereby improving the quantization efficiency. This mechanism allows the model to directly read the quantization error value from the hash table when encountering an evaluated quantization scheme, avoiding repeated calculations, thereby greatly improving the efficiency of the optimization process. This method not only reduces the time cost of searching for quantization schemes, but also accelerates the deployment and practical application of the model, providing strong support for improving the practicality and efficiency of the quantization algorithm.

[0076] The present invention proposes a method for storing quantization errors based on a hash table. By recording and searching for quantization errors in a hash table, the search process of quantization schemes can be accelerated. Compared with the traditional mixed precision quantization algorithm, the method of the present invention avoids repeated calculations by directly reading the error values ​​in the hash table, thereby greatly improving the efficiency of the optimization process. This method can significantly reduce the time cost of searching for quantization schemes and accelerate the deployment and application of models.

[0077] By using a policy gradient-based reinforcement learning algorithm to optimize the quantization scheme, we can find the quantization scheme with the least impact on the model, thereby reducing the quantization error. By jointly quantizing the image encoder and the large visual language model, the generalization ability of the model can be improved when processing multimodal data. Moreover, by using different quantization schemes for different parts of the image encoder and the large visual language model, different treatments can be performed on the sensitivity of quantization according to different parts of the model, thereby reducing computational complexity and storage requirements while ensuring model performance.

[0078] Optionally, the step S102 of inputting the image encoder calibration set into a pre-established image encoder and a text encoder, selecting a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculating a first quantization error based on outputs of the image encoder and the text encoder before and after quantization includes:

[0079] S1021, randomly select a batch of data from the image encoder calibration set, input the data into the image encoder and the text encoder, calculate the semantic similarity between each pair of image and text encodings, and obtain a first semantic similarity matrix, denoted as .

[0080] A batch of data is randomly selected from the image encoder calibration set. This batch of data contains multiple image-text pairs, each pair consists of an image and an associated text. This batch of data is input into the pre-established image encoder and text encoder. The image encoder converts the image into image encoding, and the text encoder converts the text into text encoding. The semantic similarity between each pair of image-text encoding is calculated to obtain the first semantic similarity matrix mentioned above Each row and column of this matrix represents an image code and a text code, respectively, and the values ​​in the matrix represent the semantic similarity between the corresponding image code and text code.

[0081] S1022. Select a quantization scheme from a predefined mixed precision quantization search space, and perform post-training quantization on the image encoder to obtain a quantized image encoder.

[0082] S1023: Input the selected image-text pair into the quantized image encoder and the original text encoder, calculate the semantic similarity between each pair of image-text encodings, and obtain a second semantic similarity matrix .

[0083] The same image-text pair is input into the quantized image encoder and the original text encoder. The semantic similarity between each pair of image-text encodings is calculated again to obtain the second semantic similarity matrix mentioned above. .

[0084] S1024: According to the first semantic similarity matrix and the second semantic similarity matrix Calculate the change in semantic similarity before and after quantization, that is, the quantization error of the image encoder, and obtain the first quantization error mentioned above .

[0085] The first quantization error It can be measured by comparing the changes in corresponding elements in the two matrices, for example, using indicators such as mean square error (MSE) or cosine similarity change. The following formula is used in the present invention to calculate the first quantization error :

[0086]

[0087] in, Represents the norm.

[0088] The present invention can evaluate the impact of quantization on encoder performance by calculating the change in semantic similarity before and after quantization. This helps to select the optimal quantization scheme, thereby retaining the accuracy of the model to the maximum extent while ensuring the compression rate of the model. During the quantization process, the encoder may encounter problems of numerical instability or reduced accuracy. By calculating the quantization error, the quantization scheme can be discovered and adjusted in time to enhance the robustness of the model. The quantized model has a smaller volume and lower computational complexity, and can run more efficiently on resource-constrained devices. This process provides effective guidance and optimization means for the quantization of the image encoder by accurately calculating and evaluating the quantization error, which helps to achieve efficient model compression and acceleration while maintaining model performance.

[0089] Optionally, the step S103 described above of inputting the multimodal calibration set into a pre-established large visual language model, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the large visual language model, and calculating a second quantization error according to the output of the large visual language model before and after quantization includes:

[0090] S1031. Randomly select a batch of data from the multimodal calibration set, use the image encoder to process the image in the image-text pair, input the image encoding and text processed by the image encoder into the visual language model, and obtain the original output .

[0091] A batch of data is randomly selected from the multimodal calibration set. This batch of data contains multiple image-text pairs, each of which consists of an image and an associated text. The images in the image-text pairs are processed using a pre-established image encoder to convert the images into image encodings. The image encodings and the corresponding texts are input into the pre-established visual language model. After the visual language model processes the input, it outputs a raw output. .

[0092] S1032, select a quantization scheme from the predefined mixed precision quantization search space to perform mixed precision quantization on the visual language model, input the same data into the quantized visual language model, and obtain the quantized output .

[0093] Select a quantization scheme from a predefined search space of mixed precision quantization. This search space contains a variety of possible quantization configurations, such as the quantization bit widths of different modules (such as INT8, INT4, etc.), quantization schemes, and possible post-quantization processing steps. Apply the selected quantization scheme to perform mixed precision quantization on the visual language model. Different parts of the model will use different quantization bit widths to balance model accuracy and compression rate. Use the same image-text pair data as in S1031 and input it into the quantized visual language model. After the quantized model processes the input, it outputs a quantized output .

[0094] S1033: According to the original output And the quantized output , calculate the second quantization error. This error can be measured by comparing the difference between the two outputs. The present invention calculates and The cosine distance between them gives the second quantization error , which is the quantization error of the visual language model. The calculation formula is as follows:

[0095]

[0096] The present invention can significantly reduce the size and computational complexity of the model without significantly reducing the performance of the model through mixed precision quantization. This is particularly important for deploying large visual language models on resource-constrained devices. During the quantization process, by selecting appropriate quantization schemes and parameters, the accuracy and compression rate of the model can be balanced, thereby achieving efficient model compression while ensuring model performance. The quantized model has lower computational complexity and higher inference speed, which can speed up the response time and processing speed of the model, especially when processing large amounts of data. Through quantization, the model can better adapt to different hardware platforms and deployment environments, and improve the flexibility and scalability of the model. Moreover, the quantized model occupies less storage space, which can reduce the storage cost of the model, especially when the model needs to be deployed to multiple devices or cloud servers.

[0097] Optionally, after the quantization error is statistically analyzed, an effective optimization algorithm needs to be designed so that the optimal quantization scheme can be found among a large number of quantization combinations. The patent uses a policy gradient-based reinforcement learning algorithm (such as PPO) to optimize the quantization scheme. The above step S105 combines the hash table and uses a policy gradient-based reinforcement learning algorithm to optimize the strategy for generating the quantization scheme to obtain the target quantization scheme. Figure 2 As shown, including:

[0098] S1051. Based on the predefined state space, action space and reward function, use a reinforcement learning algorithm to search for a quantization scheme, and determine a quantization error of a current quantization scheme based on the hash table.

[0099] In this step, we first define the state space, action space, and reward function, which are the basis of the reinforcement learning algorithm. The state space can include the current quantization bit width selection and the corresponding quantization error; the action space is the operation of adjusting the quantization bit width of different modules. The reward function can be combined with the quantization error data recorded in the hash table to evaluate the effect of different quantization bit width allocation schemes, thereby guiding the reinforcement learning algorithm to search for the optimal solution more efficiently.

[0100] In the present invention, the state space uses the vector Describe the information of the current module to be quantified:

[0101]

[0102] in, Represents the quantization module The state vector of the layer, Indicates that the module to be quantified is layer, Represents the type of module to be quantified, Represents the input and output dimensions of the module, represents the input feature map size, Represents flag bit, weight 1, activated is 0, is the action output of the previous moment. This information together constitutes the input of the reinforcement learning algorithm in the decision-making process, reflecting the specific situation and context of the current quantitative task.

[0103] Action space uses discrete values Describes the quantization bit width of the current module to be quantized , output by the reinforcement learning policy network and minimum and maximum quantization bit width , This method allows the reinforcement learning algorithm to flexibly adjust the bit width of different modules during the quantization process to find the optimal quantization solution.

[0104]

[0105] in, and Represents the minimum quantization bit width and the maximum quantization bit width, Output of the reinforcement learning policy network.

[0106] The design of the reward function should be able to reflect the impact of quantization error on model performance, thereby guiding the reinforcement learning algorithm to find a better quantization solution. Describe the current rewards for the quantization scheme:

[0107]

[0108] in, is the image encoder quantization error, that is, the first quantization error mentioned above, and are the maximum and minimum quantization errors observed during training, is the normalized image encoder quantization error, and They are and The image encoder quantization error and the visual language model quantization error are unified on the same scale by normalization and weighting, and can be adjusted and To control the impact of different quantization errors on the reward function.

[0109] After defining the state space, action space, and reward function, a reinforcement learning algorithm is used to search for quantization solutions. The reinforcement learning network consists of a policy network (Actor) and a value network (Critic). The policy network is responsible for generating actions (i.e., quantization solutions), while the value network is used to evaluate the value of the current state or action-state pair. Taking the Proximal Policy Optimization (PPO) algorithm as an example, by optimizing the target To update the policy parameters :

[0110]

[0111] in, For the current quantitative scheme; Output for the action at the current moment; is the current state; For the old quantitative scheme; It is a small hyperparameter used to control the clipping range, usually between 0.1 and 0.3; is the estimated value of the advantage function, which is used to measure the superiority of the current action. Its calculation formula is:

[0112]

[0113] in, is the TD error at each time step, and the output is directly predicted by the value network in PPO; is the hyperparameter discount factor; Is a smoothing parameter that controls the balance between long-term and short-term, and is usually set between ; T is the end time of an optimization round; From 0 to The integer index of . It is used to traverse from the current time step All time steps between the start and the end of the episode.

[0114] During the search process, the hash table is combined to determine the quantization error of the current quantization scheme, which includes the image encoder quantization error and the visual language large model quantization error. The hash table is used to quickly find and compare the quantization errors under different quantization schemes.

[0115] S1052. Input the image encoder quantization error and the visual language large model quantization error into the reward function to obtain the reward of the current quantization scheme, and calculate the temporal difference target according to the reward of the current quantization scheme as the loss function to update the value network parameters.

[0116] The value network updates its parameters by calculating the temporal difference target as the loss function. Specifically, the temporal difference target The calculation formula is:

[0117]

[0118] In the formula, is the reward for the current quantitative scheme, is the current value network for the next state By comparing the output of the value network and target value , calculate the mean square error loss function :

[0119]

[0120] Based on the mean square error loss function The gradient is calculated by the back-propagation algorithm, and the parameters of the value network are updated using the optimizer .

[0121] In this step, the calculated image encoder quantization error and visual language large model quantization error are input into the reward function defined above to calculate the reward obtained by the current quantization scheme.

[0122] S1053. Calculate an estimated value of the advantage function based on the temporal difference error of each time step output by the value network.

[0123] In this step, the temporal difference error of each time step output by the value network is used to calculate the estimated value of the advantage function. The advantage function is used to measure the advantage of the current action relative to the average action, thereby helping to update the parameters of the policy network.

[0124] S1054: Calculate the optimization target and update the policy network parameters according to the ratio of the current quantization scheme to the old quantization scheme and the estimated value of the advantage function.

[0125] The ratio of the current quantization scheme to the old quantization scheme is used to measure the degree of change in the quantization scheme. In this step, the ratio of the current quantization scheme to the old quantization scheme and the estimated value of the advantage function are combined to calculate the optimal solution based on the above optimization objective. Function calculation optimization goal. The optimization goal is to maximize the long-term reward, thereby guiding the policy network to find a better quantization solution. According to the optimization goal, the parameters of the policy network are updated using methods such as gradient ascent. Through continuous iteration, the policy network will gradually learn a better quantization solution.

[0126] S1055. Repeat the above process of searching for quantization schemes, updating value network parameters, calculating estimated values ​​of advantage functions, and updating policy network parameters until the policy network converges or reaches a preset number of training rounds, and obtains a target quantization scheme.

[0127] In this step, the above steps S1051 to S1054 are repeatedly performed until the policy network converges (i.e., the parameters of the policy network no longer change significantly) or the preset training rounds are reached. Through continuous iteration and optimization, the target quantization solution is finally obtained.

[0128] The present invention introduces a reinforcement learning algorithm to optimize the quantization scheme. Through the design of the reward function and the action space, the latest reinforcement learning algorithm is used to search for the quantization scheme. Compared with the traditional mixed precision quantization algorithm, the present invention can efficiently find the optimal quantization scheme, i.e., the above-mentioned target quantization scheme, among a large number of candidate quantization schemes through reinforcement learning, thereby improving the performance of the model. Compared with the traditional quantization method, the quantization scheme optimized by reinforcement learning can better balance the computing resource requirements and performance of the model, adapt to the needs of more practical application scenarios, and significantly improve the adaptability and generalization ability of the model.

[0129] By defining a reasonable state space, action space and reward function, the algorithm of the present invention can accurately reflect the characteristics and objectives of the quantization task, thereby guiding the search process to move towards the optimal solution. The design of the action space allows the algorithm to flexibly adjust the bit width of different modules during the quantization process. This flexibility enables the algorithm to allocate bit widths according to the importance of different modules and the sensitivity to quantization errors, thereby further improving the quantization effect. The design of the reward function comprehensively considers the quantization error of the image encoder and the quantization error of the multimodal large model, and unifies the two to the same scale through normalization and weighting. This design enables the algorithm to simultaneously consider the influence of multiple quantization errors during the search process, thereby finding a more comprehensive and balanced quantization strategy. By defining a reasonable state space, action space and reward function, and using an efficient search algorithm (such as PPO), a large number of quantization combinations can be efficiently searched to find the optimal quantization strategy.

[0130] Moreover, the quantization scheme optimization process based on hash tables and policy gradient reinforcement learning can automatically search and optimize the quantization scheme, which can significantly improve the quantization efficiency and reduce the time and cost of manual parameter adjustment. Moreover, through the optimization of the policy gradient reinforcement learning algorithm, a better quantization scheme can be found, thereby reducing the quantization error of the image encoder and the visual language large model, and improving the performance and accuracy of the model. Through continuous trial and learning, the reinforcement learning algorithm can make the quantization scheme more robust and generalizable, and can better adapt to different application scenarios and data distributions.

[0131] Figure 2 LayerNorm, FFN, Matmul and Softmax in the figure are components of the visual language model. LayerNorm (Layer Normalization) represents layer normalization, FFN (Feed-Forward Network) represents feed-forward neural network, and Matmul (Matrix Multiplication) represents matrix multiplication.

[0132] Optionally, the step S1051 of determining a quantization error of a current quantization scheme based on the hash table includes:

[0133] S1051. Search the hash table for the current quantization scheme, and determine whether the current quantization scheme exists in the hash table.

[0134] Through the above steps S101-104, a hash table is defined, and the hash table is used to store different quantization schemes and their corresponding quantization errors. The key of the hash table is the unique identifier of the quantization scheme, and the value is the corresponding quantization error.

[0135] When there is a new current quantization scheme, it is first searched in the hash table to see if it has already been computed. This is done by comparing the identity of the current quantization scheme with the key in the hash table.

[0136] S1052 obtains the quantization error corresponding to the current quantization scheme when the current quantization scheme exists in the hash table; and obtains the quantization error corresponding to the quantization scheme with the smallest editing distance with the current quantization scheme from the hash table when the current quantization scheme does not exist in the hash table, to obtain the quantization error corresponding to the current quantization scheme.

[0137] If the current quantization scheme exists in the hash table, it means that the quantization error of this scheme has been calculated before and stored in the hash table. Therefore, the quantization error can be obtained directly from the hash table without recalculation.

[0138] If the current quantization scheme does not exist in the hash table, an approximate calculation can be performed with a certain probability, and the quantization error of the most similar scheme can be called instead. For example, the edit distance or other similarity metrics can be used to find the scheme closest to the current quantization scheme.

[0139] The present invention uses the edit distance to measure the difference between two quantization schemes. The edit distance is the minimum number of edit operations (such as inserting, deleting or replacing characters) required to convert one string into another. Specifically, with a certain probability Calculate the current quantization scheme And the existing quantization scheme in the hash table The edit distance , take the edit distance Minimal quantization scheme The quantization error is used to replace the current quantization scheme The quantization error of is a hyperparameter used to adjust the probability of selecting approximate calculation. If approximate calculation is performed, the current quantization scheme and its approximate quantization error are not included in the hash table. The probability of approximate calculation is adjusted to achieve a balance between calculation efficiency and calculation accuracy, and improve the search efficiency of quantization schemes. Once a stored scheme with the smallest edit distance to the current quantization scheme is found, the quantization error of this scheme can be used as an estimate of the quantization error of the current scheme.

[0140] The present invention can avoid repeated calculation of the same quantization error by using a hash table to store and retrieve quantization schemes and their corresponding quantization errors, thereby significantly improving calculation efficiency. When the current quantization scheme does not exist in the hash table, a relatively accurate estimate can be provided by finding the stored scheme closest to the current scheme to estimate its quantization error. In the process of model quantization, quantization error is an important indicator for evaluating the quality of quantization schemes. By quickly and accurately obtaining the quantization error, the process of model optimization can be accelerated, thereby finding the optimal quantization scheme more quickly. In some cases, it may be necessary to dynamically adjust the quantization scheme according to the input or context of the model. This dynamic quantization can be easily implemented using a hash table because the quantization scheme and its corresponding quantization error can be quickly retrieved and updated.

[0141] Optionally, in the quantization process of the large visual language model, mixed precision quantization is an effective strategy that allows different layers or components in the model to use different quantization bit widths to balance accuracy and performance. Different quantization bit widths are selected for different modules in the large visual language model (such as the visual encoder and the text encoder) to achieve more efficient model compression and performance optimization. Specifically, the quantization bit width is independently selected for each weight matrix in the self-attention module, and the quantization bit width of each linear layer in the feedforward network is also personalized. This flexible quantization scheme can optimize resource utilization according to the computational characteristics of each module. Compared with the traditional mixed precision quantization algorithm, the scheme of the present invention reduces the computational complexity while ensuring the accuracy and performance of the model, and is suitable for a variety of practical application scenarios.

[0142] Specifically, the Q and K weight matrices in the self-attention module of the visual language large model share the same quantization bit width, the V matrix and self-attention map in the self-attention module share the same quantization bit width, and a quantization bit width is selected for each linear layer in the perceptron module.

[0143] In this step, first, multiple bit widths that can be used for quantization are determined, such as 4 bits, 8 bits, 16 bits, etc. The selection of these bit widths should be based on factors such as hardware support, model accuracy requirements, and performance requirements. In the self-attention module, the Q (query) and K (key) weight matrices are usually used to calculate the attention scores, while the V (value) matrix and the self-attention map are used to generate the final output. In order to simplify the quantization process and maintain model performance, the Q and K weight matrices can be set to share the same quantization bit width. Similarly, the V matrix and the self-attention map also share the same quantization bit width. This strategy of sharing quantization bit width helps reduce the number of quantization parameters, thereby reducing the complexity of the quantization process. For each linear layer in the perceptron module, a quantization bit width can be selected separately. This flexibility helps to optimize for the characteristics of different layers, thereby improving the accuracy and performance of the overall model.

[0144] Compared with single precision quantization, the present invention achieves efficient model compression and performance optimization by selecting different quantization bit widths for different modules in the image encoder and visual language large model. The hybrid quantization scheme designed by the present invention can significantly reduce the computational complexity and storage requirements of the model while ensuring the accuracy of the model, and improve the operating efficiency and resource utilization of the model.

[0145] The visual language large model hybrid quantization device provided by the present invention is described below. The visual language large model hybrid quantization device described below and the visual language large model hybrid quantization method described above can be referenced to each other.

[0146] The visual language large model hybrid quantization device provided by the present invention refers to Figure 3 As shown, it includes the following modules:

[0147] A data set construction module 201 is used to construct a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set;

[0148] A first calculation module 202 is used to input the image encoder calibration set into a pre-established image encoder and a text encoder, and select a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculate a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; wherein different modules in the image encoder and the visual language large model select different quantization bit widths;

[0149] A second calculation module 203 is used to input the multimodal calibration set into a pre-established visual language large model, select a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculate a second quantization error according to the output of the visual language large model before and after quantization;

[0150] The storage module 204 is used to store each quantization scheme and the corresponding first quantization error or second quantization error into a hash table;

[0151] The policy optimization module 205 is used to optimize the strategy of generating a quantization scheme in combination with the hash table using a policy gradient-based reinforcement learning algorithm to obtain a target quantization scheme.

[0152] Figure 4An example of a physical structure diagram of an electronic device is provided, and the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the visual language large model hybrid quantization method.

[0153] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the visual language large-model hybrid quantization method provided by the above-mentioned methods.

[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the visual language large-model hybrid quantization method provided by the above-mentioned methods.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large-scale hybrid quantization method for visual language, characterized in that: include: Constructing a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set; Inputting the image encoder calibration set into a pre-established image encoder and a text encoder, and selecting a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculating a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; wherein different modules in the image encoder and the visual language large model select different quantization bit widths; Inputting the multimodal calibration set into a pre-established visual language large model, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculating a second quantization error based on the output of the visual language large model before and after quantization; Storing each quantization scheme and the corresponding first quantization error or second quantization error in a hash table; Based on the predefined state space, action space and reward function, a reinforcement learning algorithm is used to search for quantization schemes, and the quantization error of the current quantization scheme is determined based on the hash table; wherein the quantization error of the current quantization scheme includes the quantization error of the image encoder and the quantization error of the visual language large model, and the reinforcement learning network includes a policy network and a value network; the image encoder quantization error and the visual language large model quantization error are input into the reward function to obtain the reward of the current quantization scheme, and the temporal difference target is calculated according to the reward of the current quantization scheme as the loss function to update the value network parameters; the estimated value of the advantage function is calculated according to the temporal difference error of each time step output by the value network; the optimization target is calculated according to the ratio of the current quantization scheme to the old quantization scheme and the estimated value of the advantage function, and the policy network parameters are updated; the above process of searching for quantization schemes, updating the value network parameters, calculating the estimated value of the advantage function and updating the policy network parameters is repeated until the policy network converges or reaches a preset training round to obtain the target quantization scheme.

2. The visual language large model hybrid quantization method according to claim 1 is characterized in that: The step of inputting the image encoder calibration set into a pre-established image encoder and a text encoder, selecting a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculating a first quantization error based on outputs of the image encoder and the text encoder before and after quantization includes: A batch of data is randomly selected from the image encoder calibration set, input into the image encoder and the text encoder, and the semantic similarity between each pair of image and text encodings is calculated to obtain a first semantic similarity matrix; wherein a batch of data includes multiple image and text pairs; Select a quantization scheme from a predefined mixed precision quantization search space, perform post-training quantization on the image encoder, and obtain a quantized image encoder; Input the selected image-text pairs into the quantized image encoder and the original text encoder, calculate the semantic similarity between each pair of image-text encodings, and obtain a second semantic similarity matrix; A first quantization error is calculated according to the first semantic similarity matrix and the second semantic similarity matrix.

3. The visual language large model hybrid quantization method according to claim 1, characterized in that: The method of inputting the multimodal calibration set into a pre-established visual language large model, selecting a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculating a second quantization error according to the output of the visual language large model before and after quantization includes: A batch of data is randomly selected from the multimodal calibration set, and the image in the image-text pair is processed using an image encoder. The image encoding and text processed by the image encoder are input into the visual language model to obtain the original output. Select a quantization scheme from a predefined mixed precision quantization search space to perform mixed precision quantization on the visual language model, input the image code processed by the image encoder and the text into the quantized visual language model to obtain a quantized output; A second quantization error is calculated according to the original output and the quantized output.

4. The visual language large model hybrid quantization method according to claim 1, characterized in that: The determining a quantization error of a current quantization scheme based on the hash table includes: Searching the current quantization scheme in the hash table to determine whether the current quantization scheme exists in the hash table; If the current quantization scheme exists in the hash table, obtaining a quantization error corresponding to the current quantization scheme; When the current quantization scheme does not exist in the hash table, a quantization error corresponding to a quantization scheme having a minimum edit distance with the current quantization scheme is obtained from the hash table with a preset probability to obtain a quantization error corresponding to the current quantization scheme.

5. The large-scale hybrid quantization method for visual language according to claim 1, characterized in that: The visual language model includes a self-attention module and a perceptron module; the search space of the mixed precision quantization is defined in the following way: During the quantization process, the search space for defining mixed precision quantization includes multiple quantization bit widths; among them, the Q and K weight matrices in the self-attention module share the same quantization bit width, the V matrix and the self-attention map in the self-attention module share the same quantization bit width, and a quantization bit width is selected for each linear layer in the perceptron module.

6. A visual language large model hybrid quantization device, characterized in that: include: A data set construction module, used to construct a quantization standard data set, wherein the quantization standard data set includes an image encoder calibration set and a multimodal calibration set; A first calculation module is used to input the image encoder calibration set into a pre-established image encoder and a text encoder, and select a quantization scheme from a pre-defined mixed precision quantization search space to train and quantize the image encoder, and calculate a first quantization error based on the outputs of the image encoder and the text encoder before and after quantization; wherein different modules in the image encoder and the visual language large model select different quantization bit widths; A second calculation module is used to input the multimodal calibration set into a pre-established visual language large model, select a quantization scheme from a pre-defined mixed precision quantization search space to perform mixed precision quantization on the visual language large model, and calculate a second quantization error according to the output of the visual language large model before and after quantization; A storage module, used for storing each quantization scheme and the corresponding first quantization error or second quantization error into a hash table; A policy optimization module is used to search for quantization schemes based on a predefined state space, action space and reward function using a reinforcement learning algorithm, and determine the quantization error of the current quantization scheme based on the hash table; wherein the quantization error of the current quantization scheme includes an image encoder quantization error and a visual language large model quantization error, and the reinforcement learning network includes a policy network and a value network; the image encoder quantization error and the visual language large model quantization error are input into the reward function to obtain the reward of the current quantization scheme, and a temporal difference target is calculated based on the reward of the current quantization scheme as a loss function to update the value network parameters; an estimated value of the advantage function is calculated based on the temporal difference error of each time step output by the value network; an optimization target is calculated based on the ratio of the current quantization scheme to the old quantization scheme and the estimated value of the advantage function, and the policy network parameters are updated; the above process of searching for quantization schemes, updating the value network parameters, calculating the estimated value of the advantage function and updating the policy network parameters is repeated until the policy network converges or reaches a preset training round to obtain a target quantization scheme.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the visual language large model hybrid quantization method as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the visual language large model hybrid quantization method as described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the visual language large model hybrid quantization method as described in any one of claims 1 to 5 is implemented.

Images