Data quantification method and device, computer equipment, storage medium and program product
By calibrating and quantizing cache key data during the inference process of a large language model, the problem of poor inference performance in existing technologies is solved, and more efficient and accurate model inference is achieved.
Patent Information
- Application Number
- CN202511141699.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-14
- Publication Date
- 2025-11-28
AI Technical Summary
Existing technologies have poor optimization effects on the inference performance of large language models, consume a lot of computing resources and have serious bottlenecks in inference speed. The quantization process introduces accuracy loss and makes it difficult to guarantee the stability of model performance.
During the inference process of the large language model, the cache key data corresponding to the input data is obtained, the data dispersion is reduced by using a preset calibration strategy, and the data is quantized to generate target quantized data. The data distribution is optimized to reduce storage space occupation and improve inference efficiency.
By performing calibration and quantization, the inference performance of the large language model was improved, the stability and accuracy of the quantization results were enhanced, and the inference efficiency and accuracy of the model were increased.
Smart Images
Figure CN121031684A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data quantification method, apparatus, computer equipment, storage medium, and program product. Background Technology
[0002] With the development of artificial intelligence technology, large language models are being used more and more widely in various industries. However, large language models are usually accompanied by huge computational resource consumption and inference speed bottlenecks. Therefore, it is necessary to optimize the inference performance of large language models.
[0003] In related technologies, quantization techniques are used to optimize the inference performance of large language models. Quantization converts the key-value data stored during the inference process of a large language model from high-precision floating-point numbers to low-precision fixed-point numbers, reducing memory usage and computational load, thereby improving inference speed.
[0004] However, the optimization results of the above-mentioned methods for improving the inference performance of large language models are not good. Summary of the Invention
[0005] Therefore, it is necessary to provide a data quantification method, apparatus, computer equipment, storage medium, and program product that can improve the inference performance of large language models in order to address the above-mentioned technical problems.
[0006] Firstly, this application provides a data quantification method. The method includes:
[0007] During the process of reasoning about the input data using the target model, the first cache key data corresponding to the input data is obtained;
[0008] The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0009] The second cache key data is quantized to obtain target quantized data, which is used to update the first cache key data.
[0010] In one embodiment, calibrating the first cache key data using a preset first calibration strategy to obtain the second cache key data includes:
[0011] Mathematical operations are performed on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
[0012] In one embodiment, the step of performing mathematical operations on the first cache key data and the plurality of data calibration parameters included in the first calibration strategy to obtain the second cache key data includes:
[0013] Calculate a first difference between the first cache key data and the median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0014] For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference;
[0015] The average value of each calibration value is determined as the second cache key data.
[0016] In one embodiment, the method further includes:
[0017] By inputting the preset calibration prompt field into the target model, multiple fourth cache key data for each data channel corresponding to the calibration prompt field are obtained;
[0018] For each data channel, the median data corresponding to the data channel is determined based on multiple fourth cache key data of the data channel;
[0019] The first calibration strategy is determined based on multiple fourth cache key data of each data channel and the median data corresponding to each data channel.
[0020] In one embodiment, determining the first calibration strategy based on multiple fourth cache key data for each data channel and the median data corresponding to each data channel includes:
[0021] For each of the data channels, a second difference is calculated between a plurality of fourth cache key data of the data channel and the median data corresponding to the data channel;
[0022] Determine the largest difference from each of the second differences, and calculate the square root value of the largest difference;
[0023] The first calibration strategy is determined based on the median data corresponding to each data channel and the square root value corresponding to each data channel.
[0024] In one embodiment, the method further includes:
[0025] The target quantized data is dequantized to obtain the dequantized target quantized data.
[0026] The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
[0027] Secondly, this application also provides a data quantization device. The device includes:
[0028] The first acquisition module is used to acquire the first cache key data corresponding to the input data during the process of reasoning on the input data using the target model;
[0029] The first calibration module is used to calibrate the first cache key data using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0030] The quantization module is used to quantize the second cache key data to obtain target quantized data, which is used to update the first cache key data.
[0031] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.
[0032] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.
[0033] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.
[0034] In the aforementioned data quantization method, apparatus, computer equipment, storage medium, and program product, during the process of inference on input data using the target model, the server first obtains the first cache key data corresponding to the input data. Then, it calibrates the first cache key data using a preset first calibration strategy to obtain second cache key data. Subsequently, it quantizes the second cache key data to obtain target quantized data. Because the first calibration strategy can reduce the data dispersion of the first cache key data, the data distribution of the calibrated second cache key data is more uniform. Then, quantizing the second cache key data can reduce the differences between multiple second cache key data within the same quantization interval, thereby reducing the quantization interval, reducing the memory space occupied by the target quantized data, and improving the stability of the quantization result. Furthermore, during the inference process, historical results can be quickly and accurately retrieved from the storage space using the target quantized data, improving inference efficiency and accuracy, improving the inference performance of the target model, and thus improving the optimization effect of the inference performance of the target model. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a diagram illustrating the application environment of a data quantization method in one embodiment.
[0037] Figure 2 This is a flowchart illustrating a data quantization method in one embodiment;
[0038] Figure 3 This is a flowchart illustrating the data quantization method in another embodiment;
[0039] Figure 4 This is a flowchart illustrating the data quantization method in another embodiment;
[0040] Figure 5 This is a flowchart illustrating step 403 in one embodiment;
[0041] Figure 6 This is a flowchart illustrating the data quantization method in another embodiment;
[0042] Figure 7 This is a flowchart illustrating the data quantization method in another embodiment;
[0043] Figure 8 This is a structural block diagram of a data quantization device in one embodiment;
[0044] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0046] With the development of artificial intelligence technology, large language models are being used more and more widely in various industries. However, large language models are also usually accompanied by huge computational resource consumption and inference speed bottlenecks, becoming a key issue restricting their further promotion and application. Therefore, how to optimize the performance of large language models and improve quantization accuracy has become a current research hotspot.
[0047] In the training and inference of large language models, the enormous computational resource consumption, especially with the continuous increase in model parameters and computational complexity, leads to higher demands on hardware performance. To alleviate this problem, the industry has explored various acceleration strategies, such as using high-performance GPUs, distributed training, and model parallelization. However, while these techniques have improved training speed to some extent, further optimization is still needed to improve inference performance and quantization accuracy. Among these, quantization technology, as an important model compression and acceleration method, can significantly reduce memory usage and computational load by converting model parameters from high-precision floating-point numbers to low-precision fixed-point numbers, thereby improving inference speed. However, the quantization process often introduces precision loss, leading to a decline in model performance, and it is difficult to guarantee the stability of precision. Therefore, how to optimize quantization accuracy while ensuring model performance has become an important research direction for accelerating quantization of large language models.
[0048] In view of this, this application proposes a data quantization method. During the inference process of the server using the target model on the input data, it first obtains the first cache key data corresponding to the input data. Then, it calibrates the first cache key data using a preset first calibration strategy to obtain second cache key data. Next, it quantizes the second cache key data to obtain target quantized data. Since the first calibration strategy can reduce the data dispersion of the first cache key data, the data distribution of the calibrated second cache key data is more uniform. Furthermore, quantizing the second cache key data reduces the differences between multiple second cache key data within the same quantization interval, and the boundaries between each quantization interval are clear. Moreover, compared to cache key data, target quantized data reduces storage space usage and improves the stability of the quantization results. During inference, the target quantized data can quickly and accurately retrieve historical results from the storage space, improving inference efficiency and accuracy, thereby improving the inference performance of the target model and ultimately enhancing the optimization effect of the target model's inference performance.
[0049] The data quantization method provided in this application embodiment can be applied to, for example... Figure 1 The implementation environment shown includes servers, which can be implemented using standalone servers or server clusters composed of multiple servers. A data storage system stores the data that the servers need to process. The data storage system can be integrated onto the server or hosted in the cloud or on other network servers.
[0050] In other possible implementations, the data quantization method provided in this application embodiment can also be applied to a terminal. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc.
[0051] In one exemplary embodiment, such as Figure 2 As shown, a data quantization method is provided, which is then applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 201-203:
[0052] Step 201: During the process of reasoning about the input data using the target model, obtain the first cache key data corresponding to the input data.
[0053] Here, the target model refers to a large language model related to a specific application scenario, and the input data refers to the text sequence that needs to be processed by the large language model. Optionally, in this embodiment, the input data can be the input data in the first round of inference of the target model, or the input data can be the input data in other rounds of inference of the target model.
[0054] It should be noted that during the multi-round inference process of the large language model, the input data is different in each round. In the first round of inference, the input data is the initial data. In other rounds of inference, the input data includes the initial data and the prediction results from previous rounds. For example, if the current round of inference is the second round, the prediction results and input data from the previous round of inference will be used as the input data for the current round. Therefore, to avoid repeated calculations on the same data, the data used for repeated calculations can be stored as key-value data in each round of inference. In this embodiment, taking a single round of inference as an example, the key data generated in this round of inference is determined as the first cached key data. Optionally, the data structure of the first cached key data can be numerical, vector, etc.
[0055] In this embodiment, the server can store the key-value data generated during the reasoning process using the target model to a storage space, and then retrieve the first cached key data from the key-value data in the storage space. Optionally, the first cached key data can be key data saved in historical reasoning rounds.
[0056] Step 202: The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data.
[0057] The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0058] The first calibration strategy refers to the rules and methods for calibrating the first cache key data. Optionally, the first calibration strategy can be: calculating the data mean of the first cache key data and updating the first cache key data to the data mean; or, the first calibration strategy can also be: if the first cache key data includes multiple data channels, and each data channel includes multiple data, then the data mean of each data channel can be calculated separately, and then the data of each data channel can be updated to the data mean. In this embodiment, after calibrating the first cache key data, the calibration result is used as the second cache key data.
[0059] It should be noted that key-value data in the cache space can be divided into key data and value data. Since value data is evenly distributed, it can be directly quantized during the quantization process. However, key data is unevenly distributed with significant dispersion. Direct quantization might result in different key values being normalized into the same quantization result, leading to either overly coarse quantization granularity (data oversimplification) or overly fine quantization granularity (redundancy in quantization intervals). This could cause the quantized data to fail to accurately map to the original data, resulting in erroneous data readings or missing data. Therefore, it is necessary to process the dispersion of the first cache key data to reduce its overall dispersion.
[0060] Optionally, in this embodiment, the server may calculate the average value of the first cache key data according to the first calibration strategy, update the first cache key data to the average value, and determine the updated first cache key data as the second cache key data. Alternatively, the server may calculate the average value of each data channel separately, update the data of each data channel to the average value, and then determine the updated first cache key data as the second cache key data.
[0061] As one possible implementation, a first calibration strategy can be used to filter the data with large fluctuations in the first cache key data, thereby reducing the overall data dispersion of the first cache key data and obtaining the second cache key data.
[0062] Step 203: Quantize the second cache key data to obtain the target quantized data.
[0063] The target quantization data is used to update the first cache key data.
[0064] Quantization, in this context, refers to the process of converting continuous or high-precision data into discrete or low-precision data. This reduces data complexity, storage space, or computational costs, thereby optimizing data storage efficiency, transmission efficiency, or processing performance, while preserving core information or usability of the data as much as possible within acceptable limits. In this embodiment, quantization can reduce the memory occupied by the second cache key data, thereby improving inference speed during the inference process. Optionally, the quantization type can include int4 quantization or int8 quantization.
[0065] The target quantized data is the result obtained after quantizing the second cache key data. It's understandable that, since the target quantized data occupies less memory, it can be used to update the first cache key data, thereby reducing the memory usage of the key data generated during the target model inference process.
[0066] In this embodiment, the server can use a preset quantization algorithm to quantize the second cache key data, then convert the second cache key data into target quantized data, and then use the target quantized data to update the first cache key data stored in the cache space.
[0067] In the aforementioned data quantization method, during the process of inference on the input data using the target model, the server first obtains the first cache key data corresponding to the input data. Then, it calibrates the first cache key data using a preset first calibration strategy to obtain the second cache key data. Subsequently, the second cache key data is quantized to obtain the target quantized data. Since the first calibration strategy can reduce the data dispersion of the first cache key data, the data distribution of the calibrated second cache key data is more uniform. Then, quantizing the second cache key data can reduce the differences between multiple second cache key data within the same quantization interval, thereby reducing the quantization interval, reducing the memory space occupied by the target quantized data, and improving the stability of the quantization results. Furthermore, during the inference process, the target quantized data can quickly and accurately retrieve historical results from the storage space, improving inference efficiency and accuracy, improving the inference performance of the target model, and thus improving the optimization effect of the inference performance of the target model.
[0068] In one embodiment, based on Figure 2 The illustrated embodiment describes how a server calibrates first cache key data using a preset first calibration strategy to obtain second cache key data. The server can implement this process through the following steps: performing mathematical operations on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to third cache key data corresponding to a preset calibration prompt field.
[0069] The data calibration parameters are used in the first calibration strategy to calibrate the first cache key data. It's important to understand that these parameters are not fixed values and can be dynamically generated based on the third cache key data. The data calibration parameters are a numerical expression of the scenario-based calibration rules. They determine the direction and intensity of the calibration; therefore, it's crucial to ensure that the data calibration parameters match the current inference scenario. For example, the data calibration parameters can be used for denoising, normalization, and feature enhancement.
[0070] Therefore, to improve the accuracy of data calibration parameters, a calibration prompt field can be pre-set, and the target model can be used to infer the calibration prompt field, determining the cache key data in this inference process as the third cache key data. The calibration prompt field reflects the identification information of the current inference scenario; for example, it could be a user group label, scenario type, data source, etc., used to define what calibration rules are needed for the current inference scenario. For example, if the calibration prompt field is "new user" in the user group label, the third cache key data might be the typical distribution range of new user characteristics; in this case, the data calibration parameter could be a normalization coefficient. If the calibration prompt field is "night scene" in the scenario type, the third cache key data might be the noise threshold or baseline value of nighttime sensor data; in this case, the data calibration parameter could be a truncation threshold, for example, correcting values exceeding this threshold to the threshold value.
[0071] The mathematical operation performed on the first cache key data and the multiple data calibration parameters included in the first calibration strategy refers to combining the first cache key data with the data calibration parameters through preset mathematical operations to output calibrated second cache key data. For example, for user feature data, the data calibration parameters can be used to weight the feature dimensions of the first cache key data to enhance the key features of new users; or, for sensor data, the data calibration parameters can be used to subtract abnormal fluctuations in the first cache key data to eliminate scene-specific noise. For example, the mathematical operations can be linear operations, nonlinear operations, statistical operations, vector operations, etc.
[0072] In this embodiment, the server can generate data calibration parameters based on the third cache key data corresponding to the preset calibration prompt field. Then, it determines the mathematical operation based on the data type of the first cache key data. Finally, it substitutes the data calibration parameters and the first cache key data into the mathematical operation to calculate the second cache key data.
[0073] In this embodiment, the server can obtain the second cache key data by performing mathematical operations on the first cache key data and multiple data calibration parameters included in the first calibration strategy. Since the calibration prompt field is information related to the inference scenario, the data calibration parameters obtained according to the third cache key data corresponding to the preset calibration prompt field can improve the matching degree between the data calibration parameters and the inference scenario, thereby improving the accuracy of the obtained second cache key data. In addition, since the process of obtaining the second cache key data through mathematical operations is relatively simple, the efficiency of obtaining the second cache key data can be improved.
[0074] As one possible implementation method, see Figure 3This embodiment relates to the process by which the server performs mathematical operations on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. For example... Figure 3 As shown, step 202 can be steps 301-303.
[0075] Step 301: Calculate the first difference between the first cache key data and the median data included in the first calibration strategy.
[0076] The median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field.
[0077] It should be noted that the multiple data calibration parameters included in the first calibration strategy can be generated based on the calibration prompt field. Specifically, the calibration prompt field is input into the target model for inference, and during the inference process, the key-value data corresponding to the calibration prompt field is determined as the third key-value data. Then, the third key-value data is divided according to the data channel to obtain multiple fourth key-value data, with one fourth key-value data corresponding to one data channel. Optionally, each fourth key-value data may include multiple data.
[0078] The median refers to the middle value among the multiple data points included in each fourth key-value pair. In other words, each fourth key-value pair corresponds to one median. The median can be used to characterize the central tendency of the data points within each fourth key-value pair.
[0079] In this embodiment, the server can divide the first cache key data according to the data channel corresponding to the calibration prompt field to obtain multiple data groups corresponding to the first cache key data, and then calculate the difference between the data in each data group and the median data in the corresponding data channel, and determine them as the first difference.
[0080] Step 302: For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference.
[0081] It should be noted that converting high-order "cumulative quantities" into low-dimensional "basic quantities" can compress the data range and reduce the impact of extreme values. In this embodiment, by calculating the square root value of the median data, the data distribution can be smoothed through nonlinear transformation. For example, if a certain median data is relatively extreme, its dispersion is higher compared to the square root value. Therefore, by calculating the square root value, the dispersion between extreme median data and other median data can be reduced.
[0082] In this embodiment, the server can first calculate the square root value of each median data, and then for each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data, and use the result as the calibration value corresponding to the first difference.
[0083] Step 303: The average value of each calibration value is determined as the second cache key data.
[0084] In this embodiment, the server can calculate the average value of each calibration value and then determine the result as the second cache key data.
[0085] As one possible implementation, different weights can be assigned to each calibration value according to the degree of influence of each data channel corresponding to each calibration value on the first cache key data. Then, the weighted average of each calibration value is calculated, and the result is determined as the second cache key data.
[0086] In this embodiment, the server calculates the first difference between the first cache key data and the median data included in the first calibration strategy. Then, for each first difference, it calculates the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference. Then, the average value of each calibration value is determined as the second cache key data. Since the median data is determined based on multiple fourth cache key data of each data channel corresponding to the calibration prompt field, the median data can characterize the data features of the key value data corresponding to the calibration prompt field. Thus, the median data is used to calibrate the first cache key data, which can suppress outliers in the first cache key data and filter data with large fluctuations, thereby making the data distribution of the calibrated second cache key data more uniform.
[0087] In one embodiment, based on the above embodiments, see [link to embodiment]. Figure 4 This embodiment relates to the process by which the server determines the first calibration strategy, such as... Figure 4 As shown, the server can execute steps 401-403 to achieve this process.
[0088] Step 401: Input the preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field.
[0089] The fourth cached key data refers to the key data corresponding to each data channel after the key data corresponding to the calibration prompt field is divided using data channels. A data channel is an independent path for the target model to receive, transmit, or process data, used to distinguish data of different types, sources, or dimensions. Each data channel includes multiple cached key data. In this embodiment, each data channel corresponding to the calibration prompt field carries cached key data of different dimensions.
[0090] In this embodiment, the server can pre-determine the calibration prompt field based on the application scenario of the target model, then input the calibration prompt field as input data into the target model, perform inference through the target model, and obtain multiple fourth cache key data corresponding to each data channel saved during the inference process.
[0091] Step 402: For each data channel, determine the median data corresponding to the data channel based on the multiple fourth cache key data of the data channel.
[0092] The median data refers to the middle value among multiple fourth cache key data, and can be used as a benchmark reference value for each data channel.
[0093] In this embodiment, taking a single data channel as an example, the multiple fourth cache key data included in the data channel are sorted in order, and then the median data corresponding to the data channel is selected according to the sorting result.
[0094] Step 403: Determine the first calibration strategy based on the multiple fourth cache key data of each data channel and the median data corresponding to each data channel.
[0095] The first calibration strategy can be used to correct the first cache key data of the target model in the subsequent inference process, so as to reduce the data deviation of the first cache key data.
[0096] In this embodiment, taking a single data channel as an example, the server can calculate the deviation between multiple fourth cache key data of the data channel and the median data corresponding to the data channel. Then, the server determines the data dispersion of each data channel based on the deviation, and then formulates a first calibration strategy based on the data dispersion.
[0097] In this embodiment, the server inputs a preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field. Then, for each data channel, the median data corresponding to the data channel is determined based on the multiple fourth cache key data of the data channel. Subsequently, a first calibration strategy can be determined based on the multiple fourth cache key data and the median data corresponding to each data channel. In this process, through the mapping relationship between the calibration prompt field and the data channel, the generated first calibration strategy can accurately match the data distribution characteristics of each data channel, improving the practicality of the first calibration strategy. Moreover, determining the first calibration strategy based on the median data of each data channel allows the determination of the first calibration strategy to set calibration parameters based on the data reference values of different data channels, eliminating the need to manually preset fixed parameters for each data channel, thus improving the flexibility of the first calibration strategy.
[0098] In one embodiment, in Figure 4 Based on the illustrated embodiments, see also Figure 5 This embodiment relates to the process by which the server determines the first calibration strategy based on multiple fourth cache key data for each data channel and the corresponding median data for each data channel. Figure 5 As shown, step 403 includes Figure 5 Steps 501-503 are shown.
[0099] Step 501: For each data channel, calculate the second difference between the multiple fourth cache key data of the data channel and the median data corresponding to the data channel.
[0100] It should be noted that the second difference between each fourth cache key data and the median data in the data channel can characterize the degree of deviation between each data and the median data, thereby characterizing the dispersion of the data distribution in each data channel.
[0101] In this embodiment, taking a single data channel as an example, the server can calculate the second difference between each fourth cache key data of the data channel and the corresponding median data of the data channel.
[0102] Step 502: Determine the largest difference from each of the second differences, and calculate the square root of the largest difference.
[0103] It should be noted that the larger the second difference between the fourth cache key data and the median data corresponding to the data channel, the greater the deviation between this data and the median data. Therefore, in order to reduce the overall data deviation, the square root of the maximum difference can be calculated to reduce the impact of extreme values on the overall data distribution through non-linear mapping. The square root value reflects the data distribution characteristics under non-linear scaling.
[0104] In this embodiment, taking a single data channel as an example, the server can sort the second differences of the data channel, select the largest second difference as the largest difference, and then calculate the square root value of the largest difference to obtain the square root value corresponding to each data channel.
[0105] Step 503: Determine the first calibration strategy based on the median data and the square root value of each data channel.
[0106] In this embodiment, a calibration formula can be established based on the median data and the square root value corresponding to each data channel. Then, the first cache key data to be calibrated is substituted into the calibration formula for calibration. For example, the calibration formula can be expressed as: L=(K-Shift) / Gamma, where L is the calibrated second cache key data, L is the first cache key data, Shift represents the median data corresponding to each data channel, and Gamma represents the square root value corresponding to each data channel.
[0107] For example, if the first cache key data is K, it is necessary to calculate L between K and the median data and the square root value corresponding to each data channel, and then determine the average of multiple L as the second cache key data.
[0108] In this embodiment, the server first calculates the second difference between multiple fourth cache key data of each data channel and the median data corresponding to the data channel. The second difference can intuitively reflect the dispersion of the data from the median data (typical value). Then, the maximum difference is determined from each second difference, and the square root of the maximum difference is calculated. This can reduce the excessive influence of the extreme maximum difference on the first calibration strategy. Then, the first calibration strategy is determined according to the median data and the square root value corresponding to each data channel. This can ensure that the first calibration strategy can both "anchor to the benchmark" (based on the median) and "dynamically adapt to fluctuations" (based on the square root value), avoiding the problem of relying solely on the median or only focusing on fluctuations while ignoring the deviation of the benchmark. This achieves differentiated calibration for each data channel and improves the reliability of the first calibration strategy.
[0109] In one embodiment, based on the above embodiments, see [link to embodiment]. Figure 6 This embodiment describes how a server processes target quantized data. For example... Figure 6 As shown, the server can execute steps 601 and 602 to achieve this process:
[0110] Step 601: Perform dequantization on the target quantized data to obtain the dequantized target quantized data.
[0111] In this process, dequantization is the reverse of quantization, which involves restoring the quantized target data to its original floating-point precision.
[0112] In this embodiment, the server can use the scaling factor and zero offset value in the quantization parameters of the quantization process to perform linear processing on the target quantized data to obtain the target quantized data after inverse quantization.
[0113] Step 602: The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data.
[0114] The target cache key data is used in the next inference process based on the target model.
[0115] The second calibration strategy is the inverse process of the first calibration strategy, and the target cache key data is the cache key data calibrated by the second calibration strategy. It can be understood that by calibrating the target quantized data after dequantization, systematic biases or residual errors introduced during the dequantization process can be corrected. Then, using the target cache key data in the next inference process based on the target model can reduce the impact of data errors on the inference results.
[0116] In this embodiment, the server can determine a second calibration strategy based on a first calibration strategy. Then, it substitutes the dequantized target quantized data into the calibration formula of the second calibration strategy to calculate the target cache key data. For example, the calibration formula of the second calibration strategy can be expressed as: M = N * Gamma + Shift, where M is the target cache key data, N is the dequantized target quantized data, Shift represents the median data corresponding to each data channel, and Gamma represents the square root value corresponding to each data channel.
[0117] In this embodiment, the server performs dequantization on the target quantized data to obtain dequantized target quantized data. Then, it uses a preset second calibration strategy to calibrate the dequantized target quantized data to obtain target cache key data. Since the target cache key data is data after dequantization and calibration by the second calibration strategy, it can compensate for the accuracy loss during the quantization process. Therefore, when the target cache key data is used in the next inference process based on the target model, the inference accuracy of the target model can be improved.
[0118] In one embodiment, a data quantization method is provided for a server, such as... Figure 7 As shown, the method includes the following steps:
[0119] Step 701: Input the preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field.
[0120] Step 702: For each data channel, determine the median data corresponding to the data channel based on the multiple fourth cache key data of the data channel.
[0121] Step 703: Calculate the second difference between the multiple fourth cache key data of the data channel and the corresponding median data of the data channel.
[0122] Step 704: Determine the largest difference from each of the second differences, and calculate the square root of the largest difference.
[0123] Step 705: Determine the first calibration strategy based on the median data and the square root value of each data channel.
[0124] Step 706: During the process of reasoning about the input data using the target model, obtain the first cache key data corresponding to the input data.
[0125] Step 707: Calculate the first difference between the first cache key data and the median data included in the first calibration strategy.
[0126] Step 708: For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference.
[0127] Step 709: The average value of each calibration value is determined as the second cache key data.
[0128] Step 7010: Quantize the second cache key data to obtain the target quantized data.
[0129] Step 7011: Perform inverse quantization on the target quantized data to obtain the inverse quantized target quantized data.
[0130] Step 7012: The target quantization data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used for the next inference process based on the target model.
[0131] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0132] Based on the same inventive concept, this application also provides a data quantization apparatus for implementing the data quantization method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data quantization apparatus embodiments provided below can be found in the limitations of the data quantization method described above, and will not be repeated here.
[0133] In one exemplary embodiment, such as Figure 8 As shown, a data quantization device is provided, comprising:
[0134] The first acquisition module 801 is used to acquire the first cache key data corresponding to the input data during the process of reasoning on the input data using the target model;
[0135] The first calibration module 802 is used to calibrate the first cache key data using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0136] The quantization module 803 is used to quantize the second cache key data to obtain target quantized data, which is used to update the first cache key data.
[0137] In one embodiment, the first calibration module 802 includes:
[0138] The calibration unit is used to perform mathematical operations on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
[0139] In one embodiment, the calibration unit is specifically used for:
[0140] Calculate the first difference between the first cache key data and the median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0141] For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference;
[0142] The average value of each calibration value is used as the second cache key data.
[0143] In one embodiment, the above-described apparatus further includes:
[0144] The second acquisition module is used to input the preset calibration prompt field into the target model and obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0145] The first determining module is used to determine the median data corresponding to each data channel based on multiple fourth cache key data of the data channel.
[0146] The second determining module is used to determine the first calibration strategy based on multiple fourth cache key data of each data channel and the median data corresponding to each data channel.
[0147] In one embodiment, the second determining module includes:
[0148] The first calculation unit is used to calculate, for each data channel, the second difference between multiple fourth cache key data of the data channel and the median data corresponding to the data channel;
[0149] The second calculation unit is used to determine the maximum difference from each of the second differences and to calculate the square root value of the maximum difference;
[0150] The determination unit is used to determine the first calibration strategy based on the median data and the square root value of each data channel.
[0151] In one embodiment, the above-described apparatus further includes:
[0152] The third acquisition module is used to perform dequantization processing on the target quantized data to obtain the dequantized target quantized data.
[0153] The second calibration module is used to calibrate the target quantized data after inverse quantization using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
[0154] Each module in the aforementioned data quantization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the server in hardware form or independent of it, or stored in the server's memory in software form, so that the processor can call and execute the operations corresponding to each module.
[0155] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data used by the target model to infer from the input data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a data quantization method.
[0156] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0157] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0158] During the process of reasoning about the input data using the target model, the first cache key data corresponding to the input data is obtained;
[0159] The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0160] The second cache key data is quantized to obtain the target quantized data, which is then used to update the first cache key data.
[0161] In one embodiment, the processor specifically implements the following steps when executing a computer program:
[0162] Mathematical operations are performed on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
[0163] In one embodiment, the processor specifically implements the following steps when executing a computer program:
[0164] Calculate the first difference between the first cache key data and the median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0165] For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference;
[0166] The average value of each calibration value is used as the second cache key data.
[0167] In one embodiment, the processor specifically implements the following steps when executing a computer program:
[0168] Input the preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0169] For each data channel, the median data corresponding to the data channel is determined based on multiple fourth cache key data of the data channel;
[0170] The first calibration strategy is determined based on the multiple fourth cache key data of each data channel and the corresponding median data of each data channel.
[0171] In one embodiment, the processor specifically implements the following steps when executing a computer program:
[0172] For each data channel, calculate the second difference between multiple fourth cache key data of the data channel and the corresponding median data of the data channel;
[0173] Determine the largest difference from among the second differences, and calculate the square root of the largest difference;
[0174] The first calibration strategy is determined based on the median data and the square root value of each data channel.
[0175] In one embodiment, the processor specifically implements the following steps when executing a computer program:
[0176] The target quantized data is dequantized to obtain the dequantized target quantized data.
[0177] The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
[0178] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0179] During the process of reasoning about the input data using the target model, the first cache key data corresponding to the input data is obtained;
[0180] The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0181] The second cache key data is quantized to obtain the target quantized data, which is then used to update the first cache key data.
[0182] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0183] Mathematical operations are performed on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
[0184] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0185] Calculate the first difference between the first cache key data and the median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0186] For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference;
[0187] The average value of each calibration value is used as the second cache key data.
[0188] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0189] Input the preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0190] For each data channel, the median data corresponding to the data channel is determined based on multiple fourth cache key data of the data channel;
[0191] The first calibration strategy is determined based on the multiple fourth cache key data of each data channel and the corresponding median data of each data channel.
[0192] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0193] For each data channel, calculate the second difference between multiple fourth cache key data of the data channel and the corresponding median data of the data channel;
[0194] Determine the largest difference from among the second differences, and calculate the square root of the largest difference;
[0195] The first calibration strategy is determined based on the median data and the square root value of each data channel.
[0196] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0197] The target quantized data is dequantized to obtain the dequantized target quantized data.
[0198] The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
[0199] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0200] During the process of reasoning about the input data using the target model, the first cache key data corresponding to the input data is obtained;
[0201] The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data.
[0202] The second cache key data is quantized to obtain the target quantized data, which is then used to update the first cache key data.
[0203] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0204] Mathematical operations are performed on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
[0205] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0206] Calculate the first difference between the first cache key data and the median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0207] For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference;
[0208] The average value of each calibration value is used as the second cache key data.
[0209] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0210] Input the preset calibration prompt field into the target model to obtain multiple fourth cache key data for each data channel corresponding to the calibration prompt field;
[0211] For each data channel, the median data corresponding to the data channel is determined based on multiple fourth cache key data of the data channel;
[0212] The first calibration strategy is determined based on the multiple fourth cache key data of each data channel and the corresponding median data of each data channel.
[0213] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0214] For each data channel, calculate the second difference between multiple fourth cache key data of the data channel and the corresponding median data of the data channel;
[0215] Determine the largest difference from among the second differences, and calculate the square root of the largest difference;
[0216] The first calibration strategy is determined based on the median data and the square root value of each data channel.
[0217] In one embodiment, when the computer program is executed by the processor, it specifically implements the following steps:
[0218] The target quantized data is dequantized to obtain the dequantized target quantized data.
[0219] The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
[0220] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0221] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0222] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data quantification method, characterized in that, The method includes: During the process of reasoning about the input data using the target model, the first cache key data corresponding to the input data is obtained; The first cache key data is calibrated using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data. The second cache key data is quantized to obtain target quantized data, which is used to update the first cache key data.
2. The method according to claim 1, characterized in that, The step of calibrating the first cache key data using a preset first calibration strategy to obtain the second cache key data includes: Mathematical operations are performed on the first cache key data and multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data. The data calibration parameters are related to the third cache key data corresponding to the preset calibration prompt field.
3. The method according to claim 2, characterized in that, The step of performing mathematical operations on the first cache key data and the multiple data calibration parameters included in the first calibration strategy to obtain the second cache key data includes: Calculate a first difference between the first cache key data and each median data included in the first calibration strategy, wherein the median data is determined based on multiple fourth cache key data for each data channel corresponding to the calibration prompt field; For each first difference, calculate the ratio between the first difference and the square root value corresponding to the median data to obtain the calibration value corresponding to the first difference; The average value of each calibration value is determined as the second cache key data.
4. The method according to claim 1, characterized in that, The method further includes: By inputting the preset calibration prompt field into the target model, multiple fourth cache key data for each data channel corresponding to the calibration prompt field are obtained; For each data channel, the median data corresponding to the data channel is determined based on multiple fourth cache key data of the data channel; The first calibration strategy is determined based on multiple fourth cache key data of each data channel and the median data corresponding to each data channel.
5. The method according to claim 4, characterized in that, The step of determining the first calibration strategy based on multiple fourth cache key data of each data channel and the median data corresponding to each data channel includes: For each of the data channels, a second difference is calculated between a plurality of fourth cache key data of the data channel and the median data corresponding to the data channel; Determine the largest difference from each of the second differences, and calculate the square root value of the largest difference; The first calibration strategy is determined based on the median data corresponding to each data channel and the square root value corresponding to each data channel.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: The target quantized data is dequantized to obtain the dequantized target quantized data. The target quantized data after inverse quantization is calibrated using a preset second calibration strategy to obtain target cache key data, which is used in the next inference process based on the target model.
7. A data quantization device, characterized in that, The device includes: The first acquisition module is used to acquire the first cache key data corresponding to the input data during the process of reasoning on the input data using the target model; The first calibration module is used to calibrate the first cache key data using a preset first calibration strategy to obtain the second cache key data. The first calibration strategy is used to reduce the data dispersion of the first cache key data. The quantization module is used to quantize the second cache key data to obtain target quantized data, which is used to update the first cache key data.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.