Data processing method, system on chip and data processing system

Abstract

The embodiment of the invention provides a data processing method, a system on chip and a data processing system.The data processing method is applied to the system on chip and comprises the steps that a to-be-processed activation matrix, a weight quantization matrix and weight quantization parameters corresponding to a service processing model are obtained; the to-be-processed activation matrix is generated based on at least one of to-be-processed text information and to-be-processed image information; performing quantization processing on the activation matrix to be processed based on the preset quantization parameter to obtain an activation quantization matrix and an activation quantization parameter; performing element calculation on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain result statistical information, and determining at least one element value, symbol information corresponding to each element value and the number of each element value based on the result statistical information; and determining a target output matrix according to the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value and the number of each element value.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of data processing technology, and in particular to a data processing method, a system-on-a-chip, and a data processing system. Background Technology

[0002] With the development of large model technology, the number of parameters in large models has grown exponentially. Consequently, the computational complexity and memory usage during inference have increased dramatically, leading to increased inference latency and decreased energy efficiency. Therefore, quantization algorithms based on large models have emerged.

[0003] Quantization algorithms typically convert floating-point numbers into low-bit integers to achieve parameter compression and efficient computation. Currently, large model structures contain numerous floating-point matrix multiplication and accumulation calculations, which consume a significant amount of time during inference. While current quantization algorithms can convert floating-point multiplication and accumulation into integer multiplication and accumulation, and then accelerate computation using integer computation units, the performance improvement from this conversion remains limited when the matrix operation scale is enormous. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a system-on-a-chip, a data processing system, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, applied to an on-chip system, comprising: Obtain the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters. The weight quantization parameters are generated based on the initial weight matrix. The activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed. The activation matrix to be processed is quantized based on the preset quantization parameters to obtain the activation quantization matrix and activation quantization parameters. The element information corresponding to the activation quantization matrix and the weight quantization matrix is ​​calculated to obtain statistical information of the results, and at least one element value, the symbol information corresponding to each element value and the number of each element value are determined based on the statistical information of the results. The target output matrix is ​​determined based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

[0006] According to a second aspect of the embodiments of this specification, an on-chip system is provided, comprising: The acquisition module is configured to acquire the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters, the weight quantization parameters are generated based on the initial weight matrix, and the activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed. The quantization module is configured to perform quantization processing on the activation matrix to be processed based on the preset quantization parameters to obtain the activation quantization matrix and the activation quantization parameters. The calculation module is configured to perform element-wise calculations on the element information corresponding to the activation quantization matrix and the weight quantization matrix, obtain result statistics, and determine at least one element value, the sign information corresponding to each element value, and the number of each element value based on the result statistics. The determination module is configured to determine the target output matrix based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

[0007] According to a third aspect of the embodiments of this specification, a data processing system is provided, comprising: Processor core; A system-on-a-chip, wherein the system-on-a-chip implements the steps of the above method when executed.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0010] This specification provides a data processing method according to an embodiment, applied to an on-chip system, comprising: obtaining a pending activation matrix, a weight quantization matrix, and weight quantization parameters corresponding to a service processing model, wherein the weight quantization matrix is ​​generated based on an initial weight matrix and preset quantization parameters, the weight quantization parameters are generated based on the initial weight matrix, and the pending activation matrix is ​​generated based on at least one of pending text information and pending image information; quantizing the pending activation matrix based on the preset quantization parameters to obtain an activation quantization matrix and activation quantization parameters; performing element-wise calculations on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain result statistics, and determining at least one element value, the sign information corresponding to each element value, and the quantity of each element value based on the result statistics; and determining a target output matrix according to the weight quantization parameters, the activation quantization parameters, each element value, the sign information corresponding to each element value, and the quantity of each element value. The method provided in the embodiments of this specification employs a non-uniform quantization approach. By pre-designing preset quantization parameters, it maintains a similar distribution between the quantized data and the original data, thereby reducing quantization loss. After determining the preset quantization parameters, the activation matrix and weight matrix can be quantized simultaneously based on these parameters. This transforms floating-point matrix multiplication operations into matrix addition operations, simplifying the complexity of the calculation process, significantly reducing computation time, and improving the usability of quantization for business processing models. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a data processing method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating the processing procedure of a data processing method applied to text processing tasks, provided in one embodiment of this specification. Figure 3 This is a schematic diagram of the structure of a system-on-a-chip provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0016] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0017] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0018] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0019] Uniform quantization: refers to a quantization method that divides the values ​​of the input signal into equal intervals. It belongs to the linear coding type. Its core feature is that the quantization interval is fixed, and the same step size is used regardless of the signal size.

[0020] Non-uniform quantization is a quantization technique that dynamically adjusts the quantization interval based on the signal amplitude. Its core principle is to adjust the signal-to-noise ratio of small signals through compression and expansion techniques, while also taking into account the dynamic range coverage of large signals.

[0021] Exponential function sampling: a technique for generating random variables from an exponential distribution through random sampling methods, often used in probability statistics and numerical simulation.

[0022] With the development of computer technology, the number of parameters in large language models has grown exponentially. This continuous expansion of parameter size has significantly improved the model's expressive power and task generalization performance. However, the massive number of parameters also leads to a sharp increase in computational complexity and memory consumption during model inference, resulting in increased inference latency and decreased energy efficiency. The computational and storage overhead caused by the large number of parameters places extremely high demands on hardware computing power, memory bandwidth, and energy consumption for model deployment, severely restricting its implementation in edge computing scenarios such as mobile devices and embedded systems. Therefore, model parameter quantization algorithms have emerged.

[0023] Low-bit quantization is a type of quantization algorithm that converts floating-point model parameters into low-precision integer representations to reduce memory usage, accelerate computation, and lower deployment costs. Its core lies in reducing the number of bits (e.g., from FP32 to INT8 or INT4) to achieve more efficient hardware resource utilization. By compressing high-precision weight parameters into a low-bit table, low-bit quantization can significantly reduce model storage space and computational resource consumption. Simultaneously, it addresses the issue of model precision loss through algorithmic adjustments.

[0024] Based on whether the range of quantized values ​​is divided into equally spaced intervals, quantization algorithms can be divided into uniform quantization and non-uniform quantization. Uniform quantization is simple to implement and computationally friendly. However, this method cannot quantize the activation matrix and weight matrix simultaneously. Since the range of values ​​in the activation matrix and weight matrix of a large model usually follows a normal distribution, non-uniform quantization can more closely approximate the distribution of the original data with the same number of bits, adjusting the quantization interval division and reducing quantization errors in dense data regions. However, non-uniform quantization is complex to implement, computationally expensive, and poorly supported by existing hardware.

[0025] Therefore, this specification provides a data processing method. One or more embodiments of this specification also relate to a system-on-a-chip, a data processing system, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0026] See Figure 1 , Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown. The method is applied to a system-on-a-chip and specifically includes the following steps.

[0027] Step 102: Obtain the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters. The weight quantization parameters are generated based on the initial weight matrix. The activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed.

[0028] The data processing methods provided in the embodiments of this specification are applied to a system-on-a-chip (SoC). An SoC can be understood as a system designed for artificial intelligence computing applications, a module used to handle a large number of computational tasks in artificial intelligence applications. Specifically, depending on the task undertaken, it may include a training system for building neural network models, an inference system for using neural network models for inference and prediction, and so on.

[0029] The system-on-a-chip (SoC) can deploy business processing models to execute batch inference tasks within these models. The activation matrix to be processed can be understood as the information-rich intermediate computation results generated at each layer of the business processing model as it processes input data. The weight matrix is ​​a set of adjustable parameters storing the knowledge and skills of the business processing model.

[0030] In a specific embodiment provided in this specification, the encoder in the Transformer model is used as an example to illustrate the business processing model. The encoder includes multiple sequentially connected encoding layers. After the input business data is processed by the embedding layer, it will be sequentially input into each encoding layer for data processing. The matrix input into the encoding layer can be understood as the activation matrix, and the parameters stored in the encoding layer can be understood as the weight matrix. After the weight matrix and the activation matrix are processed by matrix multiplication, the output result of the current encoding layer will be output, and the output result will be input into the next encoding layer for corresponding processing.

[0031] In the methods provided in the embodiments of this specification, the business processing model has been pre-deployed on the system-on-a-chip. To adapt to the methods provided in the embodiments of this specification, the weight matrices in the business processing model deployed on the system-on-a-chip are quantized weight quantization matrices. Therefore, the weight quantization matrix can be understood as the weight matrix generated after the business processing model has been processed by a preset quantization method. Since the business processing model has been deployed on the system-on-a-chip, the weight information of the business processing model after deployment on the system-on-a-chip is the weight quantization matrix.

[0032] The weight quantization matrix is ​​generated based on the initial weight matrix of the business processing model and preset quantization parameters. The initial weight matrix can be understood as the original weight matrix when the business processing model is not deployed on the on-chip system, and the preset quantization parameters can be understood as the quantization parameters used to perform quantization processing on the initial weight matrix to generate the weight quantization matrix. Specifically, in the method provided in the embodiments of this specification, the preset quantization parameters can be understood as the base for quantizing the weight matrix.

[0033] In a specific embodiment provided in this specification, the explanation is based on an example of a business processing model with 12 processing layers, each corresponding to an initial weight matrix. Before the business processing model is deployed on the system-on-a-chip, the initial weight matrices are (A1, A2, A3...A12). With a preset quantization parameter of X, the weight quantization matrices corresponding to each initial weight matrix are obtained as (B1, B2, B3...B12), where A1=X. B1 The elements in matrix A1 are values ​​with base X and exponents of the elements in B1. Correspondingly, the elements in B1 are the logarithms of the true values ​​of all elements in A1, base X, i.e., B1 = logX. A1 .

[0034] The weight quantization parameter can be understood as a standardized parameter that quantizes the values ​​of each element in the initial weight matrix. Each initial weight matrix corresponds to one weight quantization parameter. Furthermore, the weight quantization parameter in the initial weight matrix is ​​the maximum absolute value of each element's value.

[0035] In the method provided in the embodiments of this specification, after the business processing model is deployed to the on-chip system, the parameters are quantized according to the initial weight matrix and preset quantization parameters of the business processing model to obtain the weight quantization parameters corresponding to each parameter value in the initial weight matrix, and the weight quantization matrix corresponding to the initial weight matrix is ​​determined according to the weight quantization parameters.

[0036] It should be noted that in the methods provided in the embodiments of this specification, the activation matrix to be processed is specifically a business matrix related to the target business processed by the business processing model. Specifically, the activation matrix to be processed is generated based on the text information and / or image information input into the business processing model. For example, if the input to the business processing model is text information to be processed, then the activation matrix to be processed is the activation matrix generated based on the text information to be processed; if the input to the business processing model is image information to be processed, then the activation matrix to be processed is the activation matrix generated based on the image information to be processed; if the input to the business processing model is multimodal information composed of text information to be processed and image information to be processed, then the activation matrix to be processed is the activation matrix generated based on the text information to be processed and the image information to be processed.

[0037] In practical applications, there are usually multiple weight quantization matrices in a business processing model, and there is a corresponding relationship between the weight quantization matrices and the weight quantization parameters. In one or more specific embodiments provided in this specification, obtaining the weight quantization matrix and weight quantization parameters corresponding to the business processing model includes: Obtain the initial weight matrix and preset quantization parameters corresponding to the business processing model; The weight quantization parameters are determined based on the values ​​of each element in the initial weight matrix. The weight quantization matrix is ​​calculated based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters.

[0038] In the method provided in the embodiments of this specification, after the business processing model is deployed to the system on chip, its corresponding preset quantization parameters are usually determined, and the initial weight matrices of the business processing model can also be determined. In order to improve the reasoning ability of the system on chip to execute the business processing model, the process of deploying the business processing model to the system on chip can be understood as the process of quantizing the initial weight matrices of the business processing model through preset quantization parameters.

[0039] Based on this, the initial weight matrix and preset quantization parameters of the business processing model can be obtained, and the weight quantization matrix corresponding to the initial weight matrix can be determined according to the values ​​of each element in the initial weight matrix and the preset quantization parameters.

[0040] In practical applications, there may be multiple initial weight matrices corresponding to the business processing model. In order to provide a clear explanation, the method provided in the embodiments of this specification will be explained using a certain initial weight matrix as an example. The same processing method will be used for any initial weight matrix.

[0041] After determining the initial weight matrix, the weight quantization parameters are first determined based on the values ​​of each element in the initial weight matrix. Specifically, determining the weight quantization parameters based on the values ​​of each element in the initial weight matrix includes: determining the absolute value of the weight element corresponding to each weight element in the initial weight matrix; and determining the absolute value of the first weight element from the absolute values ​​of each weight element as the weight quantization parameter.

[0042] The initial weight matrix contains multiple elements, and the value of each element may be positive or negative depending on the actual situation. To ensure uniformity, the method provided in the embodiments of this specification selects the element with the largest absolute value among all weight elements as the weight quantization parameter. For further explanation, the initial weight matrix is ​​denoted as... W To illustrate, for each element value in the initial weights, its sign bit is... sign ( W Then the weight quantization parameter can be written as max( abs ( W )),in, abs ( W ) represents the absolute value of each weight element in the initial weight matrix, and max( abs ( W )) is the absolute value of the first weighted element among the absolute values ​​of all weighted elements.

[0043] After determining the weight quantization parameters, the weight quantization matrix can be calculated based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters. Specifically, calculating the weight quantization matrix based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters includes: The absolute values ​​of each weight element are normalized according to the weight quantization parameters to obtain the normalized values ​​of the weight elements corresponding to the absolute values ​​of each weight element. Using the preset quantization parameters as the base, calculate the weight quantization value corresponding to each weight element with the normalized value of each weight element being true, and determine the weight quantization matrix based on each weight quantization value.

[0044] Specifically, the normalized value of each weight element can be determined using the following formula 1: Formula 1 in, W’ The normalized value of each weight element is the value corresponding to the weight element. abs ( W ) represents the absolute value of each weight element in the initial weight matrix, and max( abs ( W )) is the weight quantization parameter.

[0045] After determining the normalized value of each weight element, the weight quantization value of each weight element is further calculated based on the preset quantization parameters and the normalized value of each weight element. Specifically, the weight quantization value is calculated using the preset quantization parameters as the base and the normalized value of the weight element as the true value. W q = log β W’ ,in, W q Each element in the value set represents the weight quantization value corresponding to that weight element. W q It is a positive integer; β Preset quantization parameters; W’ This represents the normalized value for each weight element.

[0046] The above describes the specific method for determining the weight quantization matrix. In practical applications, after the business processing model is trained, the values ​​of each element in its corresponding initial weight matrix are determined. It can be quantized according to the preset quantization parameters to obtain the weight quantization matrix corresponding to each initial weight matrix, and the business processing model can be deployed to the on-chip system through the weight quantization matrix.

[0047] In practical applications, the preset quantization parameters are determined through prior experimental testing. Specifically, the preset quantization parameters are determined through the following steps: S1002. Obtain sample service data and input the sample service data into the service processing model to obtain multiple reference activation matrices in the service processing model.

[0048] In the method provided in the embodiments of this specification, multiple sample business data are acquired, and the multiple sample business data are sequentially input into the business processing model to obtain multiple reference activation matrices of the business processing model during the processing process.

[0049] The reference activation matrix can be understood as the business data matrix information related to the sample business data generated by the business processing model during the processing of sample business data. Similar to the actual inference task, the sample business data can be either text data or image data.

[0050] In practical applications, taking text data as the sample business data as an example, the text data is input into the business processing model. The embedding layer of the business processing model converts the text data into an initial text activation matrix. The initial text activation matrix is ​​then input into the business processing layer of the business processing model. In the business processing layer, a matrix multiplication operation is performed with the weight matrix of the business processing model to obtain the business processing activation matrix output by the business processing layer. Starting from the second business processing layer, the business processing activation matrix output by the previous business processing layer is received, and a matrix multiplication operation is performed with the weight matrix in the current business processing layer to output the business processing activation matrix output by the current business processing layer. This process continues until the processing is complete. During this process, all activation matrices can be used as reference activation matrices for the business processing model.

[0051] S1004. Determine a reference activation interval based on the information of each element in multiple reference activation matrices, and determine at least one candidate quantization parameter based on the reference activation interval.

[0052] After obtaining multiple reference activation matrices, each of which involves multiple reference matrix element values, the reference activation interval is determined from these multiple reference matrix element values. Specifically, the maximum and minimum activation element values ​​are determined from the multiple reference matrix element values, and the reference activation interval is determined based on the maximum and minimum activation element values.

[0053] To ensure that the information of each element in the activation matrix can be quantized, at least one candidate quantization parameter is further determined based on the reference activation interval.

[0054] Candidate quantization parameters can be understood as the reference base corresponding to the reference activation interval. The specific method of determining them can be understood as the result of the exponential operation based on the candidate quantization parameters covering the reference activation interval.

[0055] S1006. Obtain the set of candidate quantization values ​​corresponding to each candidate quantization parameter based on each candidate quantization parameter and the preset quantization bit width.

[0056] After determining the candidate quantization parameters, the set of candidate quantization values ​​corresponding to each candidate quantization parameter can be further calculated based on the candidate quantization parameters and the preset quantization bit width. The preset quantization bit width can be understood as the precision used for subsequent quantization of matrix element values; a larger preset quantization bit width results in higher quantization precision. In practical applications, the preset quantization bit width can be designed to be 4 bits, 5 bits, 6 bits, 8 bits, etc. When the preset quantization bit width is 4 bits, it corresponds to 16 levels of fineness; when the preset quantization bit width is 8 bits, it corresponds to 256 levels of fineness. To avoid excessive precision loss during subsequent quantization processing, the preset quantization bit width can be designed to be 8 bits.

[0057] For example, given the candidate quantization parameters determined based on the reference activation interval as [0.8, 0.85, 0.9, 0.95], and a preset quantization bit width of 8 bits (representing 0-255 positive integer values), then the candidate quantization value corresponding to the candidate quantization parameter 0.8 can be determined to be [0.8]. 0 0.8 1 0.8 2 ...0.8 255 Similarly, for each candidate quantization parameter, a set of corresponding candidate quantization values ​​will be determined.

[0058] S1008. Perform quantization and dequantization processing on the initial activation matrix based on each candidate quantization value set to obtain the dequantization activation matrix corresponding to each candidate quantization value set.

[0059] After determining each candidate quantization value set, the initial activation matrix in the business processing model can be subjected to quantization and dequantization operations based on the candidate quantization parameters in each candidate quantization value set to obtain the dequantization activation matrix corresponding to each candidate quantization value set.

[0060] In one specific embodiment provided in this specification, quantization and dequantization processing is performed on the initial activation matrix based on each candidate quantization value set to obtain the dequantized activation matrix corresponding to each candidate quantization value set, including: Determine the candidate quantization parameter to be processed and the set of candidate quantization values ​​to be processed corresponding to the candidate quantization parameter to be processed, wherein the candidate quantization parameter to be processed is any one of at least one candidate quantization parameter; Normalize each initial activation value in the initial activation matrix to obtain the initial activation normalized value corresponding to each initial activation value. The initial activation normalization values ​​and the candidate quantization parameters to be processed are quantized to obtain activation quantization information. Dequantization is performed based on the symbol information and activation quantization information corresponding to each initial activation value to obtain the dequantized activation matrix.

[0061] Specifically, following the previous example, among the candidate quantization parameters [0.8, 0.85, 0.9, 0.95], taking 0.9 as an example, the candidate quantization parameter to be processed is 0.9, and the set of candidate quantization values ​​to be processed corresponding to the candidate quantization parameter is as follows: = [0.9 0 0.9 1 0.9 2 ...0.9 255 The initial activation matrix can be understood as the activation matrix that needs to be processed during the business processing model's model processing. In the embodiments of this specification, the initial activation matrix is ​​used as... XFor example, the sign bit is denoted as... sign ( X ).

[0062] Determining the initial activation matrix X Then, based on Formula 1 above, the initial activation matrix is... X The initial activation values ​​in the data are normalized to obtain the initial activation normalized value corresponding to each initial activation value, which is denoted as . .based on Traversing multiple bases Calculate the activation quantification information according to the following formula 2: Formula 2 in, b To preset the quantization bit width, To activate quantized information, each element is between 0 and... Integers.

[0063] In calculation Furthermore, the quantized values ​​are dequantized according to Formula 3 below to obtain the dequantized activation matrix. : Formula 3 in, This is the dequantized activation matrix. For weight quantization parameters, The sign bit is the corresponding bit for each activated element.

[0064] S1010. Calculate the similarity information between the dequantization activation matrix corresponding to each candidate quantization value set and the initial activation matrix, and determine the preset quantization parameter in at least one candidate quantization parameter based on the similarity information.

[0065] After processing in the above manner, the inverse quantization activation matrix corresponding to each candidate quantization value set can be obtained, and the preset quantization parameter can be further determined from at least one candidate quantization parameter by using the similarity information between each inverse quantization activation matrix and the initial activation matrix.

[0066] Specifically, refer to Formula 4 below, which calculates the cosine similarity between the initial activation matrix and the dequantized activation matrix: Formula 4 In one specific embodiment provided in this specification, determining a preset quantization parameter from at least one candidate quantization parameter based on each similarity information includes: The candidate quantization parameter with the highest similarity information is determined as the preset quantization parameter.

[0067] In this embodiment, the cosine similarity between the initial activation matrix after each base processing and the dequantized activation matrix is ​​calculated, and the candidate quantization parameter corresponding to the value with the largest cosine similarity is selected as the preset quantization parameter.

[0068] Step 104: Quantize the activation matrix to be processed based on the preset quantization parameters to obtain the activation quantization matrix and activation quantization parameters.

[0069] In the method provided in the embodiments of this specification, the activation matrix to be processed is used as... The following example will be used for explanation. Based on the preset quantization parameters, the activation matrix to be processed... Each activation element in the matrix is ​​quantized to obtain the activation quantization matrix and activation quantization parameters.

[0070] In one or more specific embodiments provided in this specification, the activation matrix to be processed is quantized based on the preset quantization parameters to obtain an activation quantization matrix and activation quantization parameters, including: The activation quantization parameters are determined based on the values ​​of each element in the activation matrix to be processed. The activation quantization matrix is ​​calculated based on the activation matrix to be processed and the preset quantization parameters.

[0071] In the method provided in the embodiments of this specification, activation quantization parameters are determined based on the values ​​of each element in the activation matrix to be processed. The specific implementation of determining the activation quantization parameters is similar to that of determining the weight quantization parameters, specifically including determining the absolute value of each activation element in the activation matrix to be processed; and determining the absolute value of the first activation element from the absolute values ​​of each activation element as the activation quantization parameter. For the specific implementation of determining the activation quantization parameters, please refer to the relevant description of determining the weight quantization parameters above, and it will not be repeated here.

[0072] Given a known activation matrix to be processed, an activation quantization matrix can be calculated based on the values ​​of each element in the activation matrix and a preset quantization parameter. Calculating the activation quantization matrix based on the activation matrix to be processed and the preset quantization parameter includes: normalizing the absolute value of each activation element to obtain a normalized value corresponding to the absolute value of each activation element; calculating the activation quantization value corresponding to each activation element using the preset quantization parameter as the base and the normalized value of each activation element as a true number; and determining the activation quantization matrix based on each activation quantization value.

[0073] The specific implementation of calculating the activation quantization matrix is ​​similar to that of calculating the weight quantization matrix. For details on the specific implementation, please refer to the relevant description of calculating the weight quantization matrix above, which will not be repeated here.

[0074] In the method provided in the embodiments of this specification, the activation matrix to be processed is... For example, the activation quantization matrix can be obtained. and activation quantization parameters .

[0075] Step 106: Perform element calculations on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain result statistics, and determine at least one element value, the symbol information corresponding to each element value, and the quantity of each element value based on the result statistics.

[0076] When the system-on-chip performs operations on the activation matrix and weight matrix, this refers to performing matrix multiplication on the activation matrix and weight matrix. In the above method, both the activation matrix and weight matrix have been converted into a form with the preset quantization parameter as the base and the quantization matrix as the logarithm. Therefore, the matrix multiplication calculation of the activation matrix and weight matrix is ​​transformed into a matrix addition operation with the preset quantization parameter as the base and the activation quantization matrix and the weight quantization matrix.

[0077] Specifically, see Formula 5 below, which shows how the summation of two floating-point matrices is converted into integer addition: Formula 5 in, a and c These are two floating-point matrices being multiplied. β To preset quantization parameters, a q and c q This is the quantization matrix. for The sign bit of the corresponding element.

[0078] In this embodiment, after the matrix multiplication operation is converted into the addition of quantized matrix elements, the operation result and sign bit can be statistically analyzed to obtain result statistics. Based on the result statistics, at least one element value of the operation result, the sign information corresponding to each element value, and the number of each element value can be determined.

[0079] In one or more specific embodiments provided in this specification, element-wise calculations are performed on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain statistical information of the results, including: Determine the target row of the activation quantization matrix and the target column corresponding to the weight quantization matrix; The element value is obtained by adding the elements at the corresponding positions in the target row and the target column, and the sign information corresponding to the element value is determined according to the sign of the element at the corresponding position. Statistical information is obtained by performing statistics on each element value and its corresponding symbol.

[0080] In practical applications, the target row for computation is determined from the activation quantization matrix, and the target column for computation is determined from the weight quantization matrix. The elements at corresponding positions in the target row and target column are added together to obtain the corresponding element values. It should be noted that in the more methods provided in the embodiments of this specification, element addition specifically refers to adding the absolute values ​​of the corresponding elements to obtain the element value. For example, the absolute values ​​of the target row elements are (12, 23, 11, 21, 9), and the absolute values ​​of the target column elements are (19, 5, 20). The element values ​​obtained by adding the two elements are (31, 28, 31, 46, 31). The sign information corresponding to each element value can be determined as (+1, +1, -1, +1, -1) according to the method of determining the sign bit in Formula 5 above. Based on the element values ​​(31, 28, 31, 46, 31) and the sign information (+1, +1, -1, +1, -1), the statistical results are [+31 (1), +28 (1), -31 (2), +46 (1)].

[0081] In the example above, the statistical results include the element values ​​(+31, +28, -31, +46), of which there is 1 +31, 1 +28, 2 -31, and 1 +46.

[0082] In one or more specific embodiments provided in this specification, determining at least one element value, the symbol information corresponding to each element value, and the quantity of each element value based on the statistical information of the results includes: The element values ​​are canceled out based on each element value and the corresponding symbol information to obtain the cancellation result. The cancellation result includes at least one element value, the symbol information corresponding to each element value, and the number of each element value.

[0083] In practical applications, the above is a statistical analysis based on the calculation results. To facilitate subsequent processing, the elements in the statistical results can be offset.

[0084] Specifically, taking the above example again, the statistical results are [+31(1), +28(1), -31(2), +46(1)], where there is 1 instance of +31 and 2 instances of -31. After canceling each other out, the cancellation result is 1 instance of -31. Therefore, the final cancellation result is [+28(1), -31(1), +46(1)]. Through the cancellation result, it can be determined that the included element values ​​are 28, 31, and 46. The sign information corresponding to 28 is +1, and the quantity is 1; the sign information corresponding to 31 is -1, and the quantity is 1; the sign information corresponding to 46 is +1, and the quantity is 1.

[0085] Step 108: Determine the target output matrix based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

[0086] After the above processing, we can obtain the weight quantization parameters, activation quantization parameters, each element value, the symbol information corresponding to each element value, and the number of each element value. Then, we can accumulate them according to the following formula 6 to obtain the final target output matrix.

[0087] Formula 6 in, To activate the quantization parameters, For weight quantization parameters, The number of element values. The symbol information corresponding to the element value. For element values, To preset quantization parameters, After the operation, the first i row and number j The element value of the column element.

[0088] The method provided in the embodiments of this specification employs a non-uniform quantization approach. By pre-designing preset quantization parameters, it maintains a similar distribution between the quantized data and the original data, thereby reducing quantization loss. After determining the preset quantization parameters, the activation matrix and weight matrix can be quantized simultaneously based on these parameters. This transforms floating-point matrix multiplication operations into matrix addition operations, simplifying the complexity of the calculation process, significantly reducing computation time, and improving the usability of quantization for business processing models.

[0089] Furthermore, when different business processing models are deployed on different on-chip systems, the preset quantization parameters can be adaptively determined instead of using the same preset quantization parameters. This adapts to the distribution of activation and weight matrices in different business processing models, better solving the problem of model accuracy degradation caused by quantization. Thus, both model accuracy and processing speed are improved.

[0090] The following is in conjunction with the appendix Figure 2 Taking the application of the data processing method provided in this specification to a text processing task as an example, the data processing method will be further explained. Figure 2 This specification illustrates a data processing method for text processing tasks according to an embodiment of the present specification, which specifically includes the following steps.

[0091] Step 202: Obtain the text processing model and preset quantization parameters, wherein the text processing model includes multiple initial weight matrices.

[0092] Step 204: Quantize each initial weight matrix according to the preset quantization parameters to obtain a weight quantization matrix, determine the weight quantization parameters corresponding to each initial weight matrix, and deploy the text processing model to the on-chip system based on the weight quantization matrix.

[0093] Step 206: Receive the text to be processed based on the text processing model deployed on the on-chip system, and convert the text to be processed into a text activation matrix.

[0094] Step 208: Quantize the text activation matrix to be processed based on the preset quantization parameters to obtain the text activation quantization matrix and text activation quantization parameters.

[0095] Step 210: In the processing unit of the on-chip system, perform matrix addition operation based on the text activation quantization matrix and the weight quantization matrix to obtain statistical results. The statistical results include at least one element value, the symbol information corresponding to each element value, and the number of each element value.

[0096] Step 212: Determine the text activation output matrix based on the text activation quantization parameters, the values ​​of each element of the weight quantization parameters, the symbol information corresponding to each element value, and the number of each element value.

[0097] The method provided in the embodiments of this specification employs a non-uniform quantization approach. During text information processing, by pre-designing preset quantization parameters, the initial weight matrix in the text processing model is quantized and then deployed to the on-chip system. Based on these preset quantization parameters, the text activation matrix generated from the text information during text processing is quantized to obtain the activation quantized matrix. This transforms matrix multiplication operations into matrix addition operations based on the preset quantization parameters, simplifying the computational complexity, significantly reducing computation time, and improving the usability of quantization in the business processing model.

[0098] By designing preset quantization parameters, the distribution of the quantized data is kept similar to that of the original data, thereby reducing quantization loss.

[0099] When different business processing models are deployed on different on-chip systems, the system can adaptively determine preset quantization parameters instead of using the same preset quantization parameters. This adapts to the distribution of activation and weight matrices in different business processing models, better solving the problem of model accuracy degradation caused by quantization. This results in a dual improvement in model accuracy and processing speed.

[0100] Corresponding to the above method embodiments, this specification also provides an on-chip system embodiment. Figure 3 A schematic diagram of the structure of a system-on-a-chip provided in one embodiment of this specification is shown. Figure 3 As shown, the device includes: The acquisition module 302 is configured to acquire the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters, the weight quantization parameters are generated based on the initial weight matrix, and the activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed. The quantization module 304 is configured to perform quantization processing on the activation matrix to be processed based on the preset quantization parameters to obtain the activation quantization matrix and the activation quantization parameters. The calculation module 306 is configured to perform element calculations on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain result statistics, and determine at least one element value, the symbol information corresponding to each element value, and the number of each element value based on the result statistics. The determination module 308 is configured to determine the target output matrix based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

[0101] In one specific embodiment provided in this specification, the acquisition module 302 is further configured as follows: Obtain the initial weight matrix and preset quantization parameters corresponding to the business processing model; The weight quantization parameters are determined based on the values ​​of each element in the initial weight matrix. The weight quantization matrix is ​​calculated based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters.

[0102] In one specific embodiment provided in this specification, the acquisition module 302 is further configured as follows: Determine the absolute value of each weight element in the initial weight matrix; The absolute value of the first weight element is determined from the absolute values ​​of all weight elements; The absolute values ​​of each weight element are normalized according to the weight quantization parameters to obtain the normalized values ​​of the weight elements corresponding to the absolute values ​​of each weight element. Using the preset quantization parameters as the base, calculate the weight quantization value corresponding to each weight element with the normalized value of each weight element being true, and determine the weight quantization matrix based on each weight quantization value.

[0103] In one specific embodiment provided in this specification, the quantization module 304 is further configured as follows: The activation quantization parameters are determined based on the values ​​of each element in the activation matrix to be processed. The activation quantization matrix is ​​calculated based on the activation matrix to be processed and the preset quantization parameters.

[0104] In one specific embodiment provided in this specification, the quantization module 304 is further configured as follows: Determine the absolute value of each activation element in the activation matrix to be processed; The absolute value of the first active element is determined from the absolute values ​​of all active elements; Normalize the absolute value of each activation element to obtain the normalized value of each activation element corresponding to its absolute value. Using the preset quantization parameters as the base, and with the normalized value of each activation element as a true number, calculate the activation quantization value corresponding to each activation element, and determine the activation quantization matrix based on each activation quantization value.

[0105] In one specific embodiment provided in this specification, the computing module 306 is further configured as follows: Determine the target row of the activation quantization matrix and the target column corresponding to the weight quantization matrix; The element value is obtained by adding the elements at the corresponding positions in the target row and the target column, and the sign information corresponding to the element value is determined according to the sign of the element at the corresponding position. Statistical information is obtained by performing statistics on each element value and its corresponding symbol.

[0106] In one specific embodiment provided in this specification, the computing module 306 is further configured as follows: The element values ​​are canceled out based on each element value and the corresponding symbol information to obtain the cancellation result. The cancellation result includes at least one element value, the symbol information corresponding to each element value, and the number of each element value.

[0107] In one specific embodiment provided in this specification, the on-chip system further includes a quantization parameter determination module, configured as follows: Acquire sample business data and input the sample business data into the business processing model to obtain multiple reference activation matrices in the business processing model; A reference activation interval is determined based on the information of each element in multiple reference activation matrices, and at least one candidate quantization parameter is determined based on the reference activation interval. The set of candidate quantization values ​​corresponding to each candidate quantization parameter is obtained based on each candidate quantization parameter and the preset quantization bit width; Based on each set of candidate quantized values, the initial activation matrix is ​​subjected to quantization and dequantization processing to obtain the dequantization activation matrix corresponding to each set of candidate quantized values. Calculate the similarity information between the dequantization activation matrix corresponding to each candidate quantization value set and the initial activation matrix, and determine the preset quantization parameter from at least one candidate quantization parameter based on the similarity information.

[0108] In one specific embodiment provided in this specification, the quantization parameter determination module is further configured as follows: Determine the candidate quantization parameter to be processed and the set of candidate quantization values ​​to be processed corresponding to the candidate quantization parameter to be processed, wherein the candidate quantization parameter to be processed is any one of at least one candidate quantization parameter; Normalize each initial activation value in the initial activation matrix to obtain the initial activation normalized value corresponding to each initial activation value. The initial activation normalization values ​​and the candidate quantization parameters to be processed are quantized to obtain activation quantization information. Dequantization is performed based on the symbol information and activation quantization information corresponding to each initial activation value to obtain the dequantized activation matrix.

[0109] In one specific embodiment provided in this specification, the quantization parameter determination module is further configured as follows: The candidate quantization parameter with the highest similarity information is determined as the preset quantization parameter.

[0110] The on-chip system provided in the embodiments of this specification adopts a non-uniform quantization method. By pre-designing preset quantization parameters, it maintains a similar distribution between the quantized data and the original data, thereby reducing quantization loss. After determining the preset quantization parameters, the activation matrix and weight matrix can be quantized simultaneously based on these parameters. This converts floating-point matrix multiplication operations into matrix addition operations, simplifying the complexity of the calculation process, greatly reducing computation time, and improving the usability of quantization for business processing models.

[0111] Furthermore, when different business processing models are deployed on different on-chip systems, the preset quantization parameters can be adaptively determined instead of using the same preset quantization parameters. This adapts to the distribution of activation and weight matrices in different business processing models, better solving the problem of model accuracy degradation caused by quantization. Thus, both model accuracy and processing speed are improved.

[0112] The above is an illustrative scheme of a system-on-a-chip (SoC) according to this embodiment. It should be noted that the technical solution of this SoC and the technical solution of the data processing method described above belong to the same concept. Details not described in detail in the SoC technical solution can be found in the description of the data processing method technical solution described above.

[0113] This specification also provides a data processing system according to one embodiment, including: Processor core; A system-on-a-chip, wherein the system-on-a-chip implements the steps of the above-described data processing method when executed.

[0114] The processor core is responsible for logic control, task scheduling, and serial computation. Its functions include generating computation task instructions and allocating tasks to computing nodes on the on-chip system. Its design focuses on low latency and complex instruction processing. The on-chip system is responsible for executing computation tasks for data-parallel processing, achieving high throughput through large-scale thread concurrency.

[0115] The processor core and the system-on-a-chip (SoC) can interact with each other. The processor core sends tasks to the SoC, which executes the tasks in parallel on each computing node and returns the results to the processor core.

[0116] The above is an illustrative scheme of a data processing system according to this embodiment. It should be noted that the technical solution of this data processing system and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the data processing method described above.

[0117] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are described simply because they are substantially similar to the data processing method embodiments; relevant parts can be referred to in the description of the data processing method embodiments.

[0119] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0120] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0121] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0122] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0123] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0124] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method applied to an on-chip system, comprising: Obtain the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters. The weight quantization parameters are generated based on the initial weight matrix. The activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed. The activation matrix to be processed is quantized based on the preset quantization parameters to obtain the activation quantization matrix and activation quantization parameters. The element information corresponding to the activation quantization matrix and the weight quantization matrix is ​​calculated to obtain statistical information of the results, and at least one element value, the symbol information corresponding to each element value and the number of each element value are determined based on the statistical information of the results. The target output matrix is ​​determined based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

2. The method as described in claim 1, wherein obtaining the weight quantization matrix and weight quantization parameters corresponding to the business processing model includes: Obtain the initial weight matrix and preset quantization parameters corresponding to the business processing model; The weight quantization parameters are determined based on the values ​​of each element in the initial weight matrix. The weight quantization matrix is ​​calculated based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters.

3. The method as described in claim 2, wherein determining the weight quantization parameters based on the values ​​of each element in the initial weight matrix includes: Determine the absolute value of each weight element in the initial weight matrix; The absolute value of the first weight element is determined from the absolute values ​​of all weight elements; Accordingly, calculating the weight quantization matrix based on the weight quantization parameters, the initial weight matrix, and the preset quantization parameters includes: The absolute values ​​of each weight element are normalized according to the weight quantization parameters to obtain the normalized values ​​of the weight elements corresponding to the absolute values ​​of each weight element. Using the preset quantization parameters as the base, calculate the weight quantization value corresponding to each weight element with the normalized value of each weight element being true, and determine the weight quantization matrix based on each weight quantization value.

4. The method as described in claim 1, wherein the activation matrix to be processed is quantized based on the preset quantization parameters to obtain an activation quantization matrix and activation quantization parameters, comprising: The activation quantization parameters are determined based on the values ​​of each element in the activation matrix to be processed. The activation quantization matrix is ​​calculated based on the activation matrix to be processed and the preset quantization parameters.

5. The method as described in claim 4, wherein determining the activation quantization parameters based on the values ​​of each element in the activation matrix to be processed includes: Determine the absolute value of each activation element in the activation matrix to be processed; The absolute value of the first active element is determined from the absolute values ​​of all active elements; Accordingly, the activation quantization matrix is ​​calculated based on the activation matrix to be processed and the preset quantization parameters, including: Normalize the absolute value of each activation element to obtain the normalized value of each activation element corresponding to its absolute value. Using the preset quantization parameters as the base, and with the normalized value of each activation element as a true number, calculate the activation quantization value corresponding to each activation element, and determine the activation quantization matrix based on each activation quantization value.

6. The method as described in claim 1, wherein element-wise calculations are performed on the element information corresponding to the activation quantization matrix and the weight quantization matrix to obtain result statistical information, including: Determine the target row of the activation quantization matrix and the target column corresponding to the weight quantization matrix; The element value is obtained by adding the elements at the corresponding positions in the target row and the target column, and the sign information corresponding to the element value is determined according to the sign of the element at the corresponding position. Statistical information is obtained by performing statistics on each element value and its corresponding symbol.

7. The method of claim 6, wherein determining at least one element value, the symbol information corresponding to each element value, and the quantity of each element value based on the statistical information of the result, comprises: The element values ​​are canceled out based on each element value and the corresponding symbol information to obtain the cancellation result. The cancellation result includes at least one element value, the symbol information corresponding to each element value, and the number of each element value.

8. The method according to any one of claims 1-7, wherein the preset quantization parameter is determined by the following steps: Acquire sample business data and input the sample business data into the business processing model to obtain multiple reference activation matrices in the business processing model; A reference activation interval is determined based on the information of each element in multiple reference activation matrices, and at least one candidate quantization parameter is determined based on the reference activation interval. The set of candidate quantization values ​​corresponding to each candidate quantization parameter is obtained based on each candidate quantization parameter and the preset quantization bit width; Based on each set of candidate quantized values, the initial activation matrix is ​​subjected to quantization and dequantization processing to obtain the dequantization activation matrix corresponding to each set of candidate quantized values. Calculate the similarity information between the dequantization activation matrix corresponding to each candidate quantization value set and the initial activation matrix, and determine the preset quantization parameter from at least one candidate quantization parameter based on the similarity information.

9. The method of claim 8, wherein quantization and dequantization processing is performed on the initial activation matrix based on each candidate quantization value set to obtain the dequantization activation matrix corresponding to each candidate quantization value set, comprising: Determine the candidate quantization parameter to be processed and the set of candidate quantization values ​​to be processed corresponding to the candidate quantization parameter to be processed, wherein the candidate quantization parameter to be processed is any one of at least one candidate quantization parameter; Normalize each initial activation value in the initial activation matrix to obtain the initial activation normalized value corresponding to each initial activation value. The initial activation normalization values ​​and the candidate quantization parameters to be processed are quantized to obtain activation quantization information. Dequantization is performed based on the symbol information and activation quantization information corresponding to each initial activation value to obtain the dequantized activation matrix.

10. The method of claim 8, wherein a preset quantization parameter is determined from at least one candidate quantization parameter based on each similarity information, comprising: The candidate quantization parameter with the highest similarity information is determined as the preset quantization parameter.

11. A system-on-a-chip, comprising: The acquisition module is configured to acquire the activation matrix to be processed, the weight quantization matrix, and the weight quantization parameters corresponding to the business processing model. The weight quantization matrix is ​​generated based on the initial weight matrix and the preset quantization parameters, the weight quantization parameters are generated based on the initial weight matrix, and the activation matrix to be processed is generated based on at least one of the text information to be processed and the image information to be processed. The quantization module is configured to perform quantization processing on the activation matrix to be processed based on the preset quantization parameters to obtain the activation quantization matrix and the activation quantization parameters. The calculation module is configured to perform element-wise calculations on the element information corresponding to the activation quantization matrix and the weight quantization matrix, obtain result statistics, and determine at least one element value, the sign information corresponding to each element value, and the number of each element value based on the result statistics. The determination module is configured to determine the target output matrix based on the weight quantization parameter, the activation quantization parameter, each element value, the symbol information corresponding to each element value, and the number of each element value.

12. A data processing system, comprising: Processor core; A system-on-a-chip, wherein the system-on-a-chip, when executed, implements the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-10.

14. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-10.

Images