Data processing method, electronic device, storage medium and program product
By utilizing existing operators in the AI processor of the terminal device to collaboratively complete the normalization operation, the problem of insufficient computing power of the terminal device is solved, the efficient deployment of the Norm operator is realized, and the hardware design dependency is reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAN UNISOC TECH CO LTD
- Filing Date
- 2026-01-05
- Publication Date
- 2026-04-21
AI Technical Summary
Due to limited computing power, AI processors in terminal devices struggle to support the hardware deployment of complex Norm operators, thus restricting the performance requirements of large models in terminal devices.
By leveraging the hardware modules that support the first, second, and third operators commonly found in the AI processors of terminal devices, normalization operations can be completed collaboratively, enabling the normalization of the Norm operator without relying on dedicated hardware modules for transcendental functions.
This significantly reduces the reliance on the computing resources and hardware design of AI processors in terminal devices, providing a feasible path for the efficient deployment of large models in terminal devices.
Smart Images
Figure CN121900720A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method, electronic device, storage medium, and program product. Background Technology
[0002] With the rapid development of artificial intelligence technology, the demand for deploying large models (such as Transformer models) on terminal devices is increasing. Among them, the normalization (Norm) operator can effectively accelerate the training and convergence process of large models.
[0003] In related technologies, high-performance artificial intelligence (AI) processors typically support Norm operator operations by designing dedicated hardware modules.
[0004] However, the Norm operator contains transcendental functions, which are computationally difficult and have complex operational structures. The AI processors of terminal devices are limited by their own computing power and cannot support the hardware deployment of the Norm operator, thus restricting the performance requirements of large models in terminal devices. Summary of the Invention
[0005] This application provides a data processing method, electronic device, storage medium, and program product for implementing Norm operator operations using the computing power of the AI processor in a terminal device.
[0006] In a first aspect, embodiments of this application provide a data processing method, including:
[0007] Get the first data to be processed;
[0008] Based on the first operator, the first data is processed to obtain the mean of the first data;
[0009] Based on the second operator, the distribution differences of the first data are eliminated using the mean as a benchmark, to obtain the second data;
[0010] Based on the first and third operators, and using the mean as a benchmark, the first data is aggregated and transformed to obtain the third data;
[0011] Based on the third operator, the scale bias of the second data is eliminated using the third data as a calibration factor, resulting in the normalized first data.
[0012] In some embodiments, processing the first data to obtain the mean of the first data includes:
[0013] Determine the first quantization scaling factor and weight parameter of the first operator. The first quantization scaling factor is the reciprocal of the total number of data points of the first data, and the weight parameter is a preset value.
[0014] Based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data.
[0015] Multiply the first quantization scaling factor by the fourth data point to obtain the mean of the first data point.
[0016] In some embodiments, based on the first operator and the third operator, and using the mean as a benchmark, the first data is aggregated and transformed to obtain the third data, including:
[0017] Based on the third operator, determine the difference between each data point in the first data and the mean;
[0018] Based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point.
[0019] Based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data.
[0020] In some embodiments, for any data point; based on the third operator, the differences between the data points in the first data are squared to obtain the first parameter corresponding to each data point, including:
[0021] Determine the second quantization scaling factor;
[0022] Based on the third operator, the initial parameters are obtained by squaring the differences between the data points.
[0023] Based on the third operator, the product of the initial parameter and the second quantization scaling factor is used to determine the first parameter corresponding to the data point.
[0024] In some embodiments, based on a first operator, multiple first parameters are aggregated and transformed to obtain third data, including:
[0025] Determine the third quantization scaling factor of the first operator, where the third quantization scaling factor is the ratio of the first quantization scaling factor to the second quantization scaling factor;
[0026] Based on the first operator, multiple first parameters are aggregated to obtain the fifth data.
[0027] Based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data;
[0028] The third data is determined based on the sixth data and the preset mapping table.
[0029] In some embodiments, the scaling bias of the second data is eliminated using the third data as a calibration factor to obtain the normalized first data, including:
[0030] The product of the third data, the second data, and the preset second parameter is determined as the scale calibration value. The second parameter is used to perform scale deviation calibration on the second data.
[0031] The sum of the scale calibration value and the preset third parameter is determined as the normalized first data. The third parameter is used to compensate for the reference offset of the scale calibration value.
[0032] Secondly, embodiments of this application provide a data processing apparatus, comprising: an acquisition module, a first processing module, a second processing module, a third processing module, and a fourth processing module, wherein:
[0033] The acquisition module is used to acquire the first data to be processed;
[0034] The first processing module is used to process the first data based on the first operator to obtain the mean of the first data;
[0035] The second processing module is used to eliminate the distribution differences of the first data based on the second operator and the mean value to obtain the second data;
[0036] The third processing module is used to perform aggregation and transformation processing on the first data based on the first operator and the third operator, with the mean as the benchmark, to obtain the third data;
[0037] The fourth processing module is used to eliminate the scale bias of the second data based on the third operator and using the third data as a calibration factor, so as to obtain the normalized first data.
[0038] In some embodiments, the first processing module is specifically used for:
[0039] Determine the first quantization scaling factor and weight parameter of the first operator. The first quantization scaling factor is the reciprocal of the total number of data points of the first data, and the weight parameter is a preset value.
[0040] Based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data.
[0041] Multiply the first quantization scaling factor by the fourth data point to obtain the mean of the first data point.
[0042] In some embodiments, the third processing module is specifically used for:
[0043] Based on the third operator, determine the difference between each data point in the first data and the mean;
[0044] Based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point.
[0045] Based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data.
[0046] In some embodiments, for any single data point; the third processing module is specifically used for:
[0047] Determine the second quantization scaling factor;
[0048] Based on the third operator, the initial parameters are obtained by squaring the differences between the data points.
[0049] Based on the third operator, the product of the initial parameter and the second quantization scaling factor is used to determine the first parameter corresponding to the data point.
[0050] In some embodiments, the third processing module is specifically used for:
[0051] Determine the third quantization scaling factor of the first operator, where the third quantization scaling factor is the ratio of the first quantization scaling factor to the second quantization scaling factor;
[0052] Based on the first operator, multiple first parameters are aggregated to obtain the fifth data.
[0053] Based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data;
[0054] The third data is determined based on the sixth data and the preset mapping table.
[0055] In some embodiments, the fourth processing module is specifically used for:
[0056] The product of the third data, the second data, and the preset second parameter is determined as the scale calibration value. The second parameter is used to perform scale deviation calibration on the second data.
[0057] The sum of the scale calibration value and the preset third parameter is determined as the normalized first data. The third parameter is used to compensate for the reference offset of the scale calibration value.
[0058] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the data processing method as described in the first aspect and various possible designs of the first aspect.
[0059] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the data processing method described in the first aspect and various possible designs of the first aspect.
[0060] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the data processing method described in the first aspect and various possible designs of the first aspect.
[0061] In a sixth aspect, embodiments of this application provide a chip, the chip including at least one processor, the processor being configured to execute program instructions to implement the data processing methods described in the first aspect above and various possible designs of the first aspect.
[0062] This application provides a data processing method, electronic device, storage medium, and program product. The method includes: acquiring first data to be processed; processing the first data based on a first operator to obtain the mean of the first data; eliminating the distribution differences of the first data based on a second operator and the mean to obtain second data; performing aggregation and transformation processing on the first data based on the first and third operators and the mean to obtain third data; and eliminating the scale bias of the second data based on the third operator and the third data as a calibration factor to obtain normalized first data. In the above method, the hardware modules that support the first, second, and third operators, which are commonly found in the AI processors of terminal devices, are used to collaboratively complete the normalization operation. This eliminates the need for dedicated transcendental function hardware modules, thus significantly reducing the dependence on the computing resources and hardware design of the AI processor in the terminal device and providing a feasible path for the efficient deployment of large models in terminal devices. Attached Figure Description
[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0064] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;
[0065] Figure 2 A flowchart illustrating a method for determining third data provided in an embodiment of this application;
[0066] Figure 3 A flowchart illustrating a method for determining normalized first data provided in an embodiment of this application;
[0067] Figure 4This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;
[0068] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0069] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0071] It should be noted that the data processing method, electronic device, storage medium, and program product provided in this application can be used in the computer field, or in any field other than the computer field. The application field of the data processing method, electronic device, storage medium, and program product in this application is not limited.
[0072] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.
[0073] With the rapid development of artificial intelligence technology, the demand for deploying large models (such as Transformer models) on terminal devices is increasing. Among them, the normalization (Norm) operator can effectively accelerate the training and convergence process of large models.
[0074] In related technologies, high-performance artificial intelligence (AI) processors typically support Norm operator operations by designing dedicated hardware modules.
[0075] However, the Norm operator contains transcendental functions (such as square root functions, reciprocal functions, etc.). These functions are computationally difficult and have complex operational structures. The AI processors of terminal devices are limited by their own computing power and cannot support the hardware deployment of Norm operators, thus restricting the performance requirements of large models in terminal devices.
[0076] To address the aforementioned technical problems, in this embodiment, the following steps are taken: First data to be processed is obtained; the first data is processed based on a first operator to obtain its mean; the distribution differences of the first data are eliminated based on the mean using a second operator, resulting in second data; the first data is aggregated and transformed based on the mean using both the first and third operators, resulting in third data; and the scale bias of the second data is eliminated using the third operator as a calibration factor, resulting in normalized first data. In this method, the hardware modules supporting the first, second, and third operators, commonly found in the AI processors of terminal devices, collaboratively complete the normalization operation. This eliminates the need for dedicated transcendental function hardware modules, significantly reducing the reliance on the computational resources and hardware design of the terminal device's AI processor, and providing a feasible path for the efficient deployment of large models on terminal devices.
[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0078] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Please refer to [link / reference]. Figure 1 As shown, the method may include the following steps:
[0079] S101, Obtain the first data to be processed.
[0080] The execution subject of this application embodiment can be an electronic device or a normalization device installed in an electronic device. The normalization device can be implemented by software or by a combination of software and hardware.
[0081] The electronic device can be a terminal device, which includes a first operator, a second operator, and a third operator. The inclusion of a first operator, a second operator, and a third operator in the terminal device can be understood as the terminal device's AI processor having pre-built hardware modules that support the operations of the first, second, and third operators.
[0082] The first data is feature data in int8 quantization format (such as input features of a large model Transformer, output feature vectors of fully connected layers, etc.), which is a commonly used data format optimized by AI processors of terminal devices for low computing power scenarios, and can effectively reduce data storage overhead and transmission bandwidth usage.
[0083] S102. Based on the first operator, process the first data to obtain the mean of the first data.
[0084] In some embodiments, the first data is processed based on the first operator to obtain the mean of the first data. This can be understood as implementing the following formula 1 based on the first operator:
[0085] Formula 1;
[0086] in, This represents the mean of the first data point. This represents the i-th data point in the first set of data. This represents the total number of data points in the first data set.
[0087] In some embodiments, processing the first data to obtain the mean of the first data includes:
[0088] Determine the first quantization scaling factor and weight parameter of the first operator. The first quantization scaling factor is the reciprocal of the total number of data points of the first data, and the weight parameter is a preset value.
[0089] Based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data.
[0090] Multiply the first quantization scaling factor by the fourth data point to obtain the mean of the first data point.
[0091] For example, when the number of data points in the first data is n, the first quantization scaling factor is 1 / n.
[0092] In some embodiments, the first operator is, for example, the conv operator, and the weight parameters of the conv operator are all 1s. The fact that the weight parameters of the conv operator are all 1s can be understood as follows: when the number of data points in the first data is n, the number of sub-parameters of the weight parameters is n, and all sub-parameters are 1.
[0093] In some embodiments, based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data. This can be understood as follows: when the weight parameters of the first operator are all 1s, based on the first operator, the following formula 2 is implemented:
[0094] Formula 2;
[0095] in, Indicates the fourth data point. This represents the i-th sub-parameter in the weight parameters. .
[0096] In some embodiments, multiplying the first quantization scaling factor by the fourth data to obtain the mean of the first data can be understood as setting the first quantization scaling factor of the first operator to 1 / n, then the following formula 3 can be implemented using the conv operator:
[0097] Formula 3;
[0098] In some embodiments, when the AI processor of the terminal device supports the ReduceMean operator, the first data can also be processed based on the ReduceMean operator to obtain the mean of the first data, that is, Formula 1 can be implemented based on the ReduceMean operator.
[0099] In some embodiments, it can be first determined whether the AI processor of the terminal device contains a hardware module that supports the ReduceMean operator; if the AI processor of the terminal device contains a hardware module that supports the ReduceMean operator, the ReduceMean operator is directly called to implement Formula 1; if the AI processor of the terminal device does not contain a hardware module that supports the ReduceMean operator, the conv operator is used to implement Formula 1.
[0100] S103. Based on the second operator, using the mean of the first data as a benchmark, the distribution differences of the first data are eliminated to obtain the second data.
[0101] In some embodiments, the second operator is, for example, the elementwise_sub operator.
[0102] In some embodiments, the elementwise_sub operator is used to implement the following formula 4:
[0103] Formula 4;
[0104] in, This represents the i-th data point of the second set of data.
[0105] S104. Based on the first operator and the third operator, and taking the mean of the first data as the benchmark, perform aggregation and transformation processing on the first data to obtain the third data.
[0106] In some embodiments, the third operator is, for example, the elementwise_mul operator.
[0107] In some embodiments, based on the first operator and the third operator, and using the mean of the first data as a benchmark, the first data is aggregated and transformed to obtain the third data. This can be understood as implementing the following formula 5 based on the first operator and the third operator:
[0108] Formula 5;
[0109] in, This indicates the third data point.
[0110] S105. Based on the third operator, using the third data as a calibration factor, the scale bias of the second data is eliminated to obtain the normalized first data.
[0111] In some embodiments, based on the third operator and using the third data as a calibration factor, the scale bias of the second data is eliminated to obtain the normalized first data. This can be understood as implementing the following formula 6 based on the third operator:
[0112] Formula 6;
[0113] in, This represents the i-th data point of the first set of data after normalization.
[0114] In this embodiment, first data to be processed is obtained; based on a first operator, the first data is processed to obtain its mean; based on a second operator, the distribution differences of the first data are eliminated using the mean as a benchmark to obtain second data; based on the first and third operators, the first data is aggregated and transformed using the mean as a benchmark to obtain third data; based on the third operator, the scale bias of the second data is eliminated using the third data as a calibration factor to obtain normalized first data. In the above method, the hardware modules that support the first, second, and third operators, commonly found in the AI processors of terminal devices, are used to collaboratively complete the normalization operation. This eliminates the need for dedicated transcendental function hardware modules, significantly reducing the reliance on the computational resources and hardware design of the terminal device's AI processor, and providing a feasible path for the efficient deployment of large models in terminal devices.
[0115] The following, based on any of the above embodiments, combined with Figure 2 The process of aggregating and transforming the first data based on the first and third operators, using the mean as a benchmark, to obtain the third data is explained in detail.
[0116] Figure 2 This is a flowchart illustrating a method for determining third data provided in an embodiment of this application. Please refer to... Figure 2 The method may include:
[0117] S201. Based on the third operator, determine the difference between each data point in the first data and the mean of the first data.
[0118] In some embodiments, determining the difference between each data point in the first data and the mean of the first data based on the third operator can be understood as implementing the following formula 7 based on the third operator:
[0119] Formula 7;
[0120] in, This represents the difference corresponding to the i-th data point in the first data (i.e., the difference between the i-th data point in the first data and the mean of the first data).
[0121] S202. Based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point in the first data.
[0122] In some embodiments, based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point in the first data. This can be understood as implementing the following formula 8 based on the third operator:
[0123] Formula 8;
[0124] in, This represents the first parameter corresponding to the i-th data point in the first data set.
[0125] In some embodiments, for any data point, based on the third operator, the difference between the data points in the first data is squared to obtain the first parameter corresponding to each data point, including:
[0126] Determine the second quantization scaling factor;
[0127] Based on the third operator, the initial parameters are obtained by squaring the differences between the data points.
[0128] Based on the third operator, the product of the initial parameter and the second quantization scaling factor is used to determine the first parameter corresponding to the data point.
[0129] In some embodiments, based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point in the first data. This can also be understood as implementing the following formula 9 based on the third operator:
[0130] Formula 9;
[0131] in, This represents the first parameter corresponding to the i-th data point in the first data set. This represents the second quantization scaling factor.
[0132] For example, the second quantization scaling factor is, for instance, 1 / 168.
[0133] In this embodiment, a second quantization scaling factor is introduced to solve the problems of overflow and uncontrolled numerical range in square operations under the int8 quantization format. Specifically, the difference of the first data is in int8 quantization format (value range -128 to 127). The numerical range of this value will expand dramatically (up to 127² = 16129), far exceeding the storage limit of the int8 quantization format. Direct storage or operation will lead to severe overflow truncation, causing significant precision loss. By introducing a second quantization scaling factor (such as the exemplary 1 / 168), the result can be pre-scaled after the squaring operation, thus reducing the precision loss. Compression to the range that the int8 quantization format can support (e.g., 16129 × (1 / 168) ≈ 96, which is in the range of -128 to 127) effectively avoids data distortion caused by overflow.
[0134] It should be noted that if the first data is in another quantization format, the value of r can be adjusted to avoid data distortion caused by overflow. For example, if the first data is int16 quantization format, r can be set to 1 / 64.
[0135] S203. Based on the first operator, perform aggregation and transformation processing on multiple first parameters to obtain the third data.
[0136] In some embodiments, based on the first operator, multiple first parameters are aggregated and transformed to obtain third data. This can be understood as implementing the following formula 10 based on the first operator:
[0137] Formula 10;
[0138] In some embodiments, based on a first operator, multiple first parameters are aggregated and transformed to obtain third data, including:
[0139] Determine the third quantization scaling factor of the first operator, where the third quantization scaling factor is the ratio of the first quantization scaling factor to the second quantization scaling factor;
[0140] Based on the first operator, multiple first parameters are aggregated to obtain the fifth data.
[0141] Based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data;
[0142] The third data is determined based on the sixth data and the preset mapping table.
[0143] The first quantization scaling factor is The second quantization scaling factor is At that time, the third quantization scaling factor is For example, when the second quantization scaling factor is 1 / 168, the third quantization scaling factor is... .
[0144] In some embodiments, based on the first operator, multiple first parameters are aggregated to obtain the fifth data. This can be understood as implementing the following formula 11 based on the first operator:
[0145] Formula 11;
[0146] in, This represents the fifth data point.
[0147] In some embodiments, based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data. This can be understood as implementing the following formula 12 based on the first operator:
[0148] Formula 12;
[0149] in, Indicates the sixth data point. This represents the third quantization scaling factor.
[0150] In this embodiment of the application, since the second quantization scaling factor r has already... The data is compressed to the range that the int8 quantization format can support. The third quantization scaling factor is used to restore the original true value later, which perfectly offsets the numerical scaling effect brought by the second quantization scaling factor and ensures the accuracy of the sixth data.
[0151] In some embodiments, determining the third data based on the sixth data and a preset mapping table can be understood as retrieving the third data from the preset mapping table based on the sixth data.
[0152] In some embodiments, based on a preset mapping table, the following formula 13 is implemented:
[0153] Formula 13;
[0154] Specifically, the range of data in int8 quantization format is greater than or equal to -128 and less than or equal to 127, and the data in int8 quantization format is an integer. Since this embodiment of the application uses int8 quantization format throughout the process, the sixth data obtained through the aforementioned steps is also int8 quantization format, that is, the sixth data is between -128 and 127 and is an integer. Based on this, the preset mapping table pre-stores 256 int8 integers in the range of -128 to 127, as well as the third data corresponding to each integer. This mapping table is a high-performance lookup table that is pre-built offline and solidified in the terminal device for the transcendental function of formula 13, and can be directly and quickly called by the terminal device.
[0155] Given that the result of the transcendental function in Formula 13 is a small floating-point value, in order to ensure that the third data output from the lookup table has higher precision, this application performs input dequantization and output quantization optimization operations simultaneously during the mapping table pre-fabrication stage, taking into account the output characteristics of the sixth data, so that the precision of the lookup table result fully meets the calculation requirements of the AI processor of the terminal device.
[0156] In some embodiments, the integer and the third data satisfy the following formula 14:
[0157] Formula 14;
[0158] in, This represents the j-th integer among 256 integers. .
[0159] In the embodiments of this application, The calculated result range is in the decimal range of the thousandths place. A numerical magnitude that is too small can easily lead to precision loss. This application has been optimized through experimental testing. If a magnification of 10240 times is performed, then .
[0160] In this embodiment, based on the third operator, the difference between each data point in the first data and the mean is determined; based on the third operator, the difference corresponding to each data point in the first data is squared to obtain the first parameter corresponding to each data point; based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data. In the above method, the variance-related parameters are efficiently solved by squaring the difference and performing aggregation and transformation. At the hardware level, only the basic operators (first operator and third operator) already supported by the AI processor of the terminal device need to be called, without the need to introduce a dedicated transcendental function operation unit, which greatly reduces the computational complexity and hardware implementation overhead. At the same time, it fully utilizes the existing computing power of the terminal device, thereby achieving efficient and feasible normalization processing under resource-constrained conditions, providing strong support for the deployment and application of large models in terminal devices.
[0161] The following, based on any of the above embodiments, combined with Figure 3 The process of eliminating the scaling bias of the second data by using the third data as a calibration factor, and obtaining the normalized first data, is explained in detail.
[0162] Figure 3 This is a flowchart illustrating a method for determining normalized first data as provided in an embodiment of this application. Please refer to... Figure 3 The method may include:
[0163] S301. The product of the third data, the second data, and the preset second parameter is determined as the scale calibration value, wherein the second parameter is used to perform scale deviation calibration on the second data.
[0164] In some embodiments, based on the third operator, the third data, the second data, and the second parameter are multiplied to obtain the scale calibration value. This can be understood as implementing the following formula 15 based on the third operator:
[0165] Formula 15;
[0166] in, This represents the scale calibration value of the i-th data point in the first dataset. , This represents the first vector parameter.
[0167] S302. The sum of the scale calibration value and the preset third parameter is determined as the normalized first data, wherein the third parameter is used to compensate for the reference offset of the scale calibration value.
[0168] In some embodiments, based on the third operator, the scale calibration value is added to a preset third parameter to obtain the normalized first data. This can be understood as implementing the following formula 16 based on the third operator:
[0169] Formula 16;
[0170] Where represents the i-th data point of the first data after normalization. , This represents the second vector parameter. This is the output quantization scaling factor after being amplified by 10240 times, i.e. , This indicates the output quantization zero point, which is the standard offset parameter in int8 quantization format, with a value of, for example, 128.
[0171] In some embodiments, and As learnable vector parameters, the values can be optimized through model training iterations based on the actual business scenarios of terminal devices (such as feature processing of large model Transformers and normalization of feature vectors in fully connected layers) and data distribution characteristics. This adapts to the scale calibration and benchmark offset requirements of different feature data, further improving the accuracy and adaptability of normalization processing. At the same time, γ and β are stored in int8 quantization format, which is consistent with the design logic of full int8 quantization format and avoids additional format conversion overhead.
[0172] In this embodiment, the product of the third data, the second data, and a preset second parameter is determined as the scale calibration value. The second parameter is used to perform scale bias calibration on the second data. The sum of the scale calibration value and the preset third parameter is determined as the normalized first data. The third parameter is used to compensate for the baseline offset of the scale calibration value. In the above method, by introducing trainable second and third parameters, flexible calibration and compensation of the normalization result are achieved, effectively enhancing the adaptability and expressive power to different data distributions. This process is entirely based on the operator (third operator) already supported by the terminal device, without relying on complex transcendental function calculations. While significantly reducing hardware overhead and computational latency, it ensures the efficiency and practicality of normalization in terminal device deployment, thereby supporting the stable operation and performance optimization of large models in resource-constrained environments.
[0173] Figure 4 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. Please refer to... Figure 4 The data processing device 40 includes: an acquisition module 401, a first processing module 402, a second processing module 403, a third processing module 404, and a fourth processing module 405, wherein:
[0174] The acquisition module 401 is used to acquire the first data to be processed;
[0175] The first processing module 402 is used to process the first data based on the first operator to obtain the mean of the first data;
[0176] The second processing module 403 is used to eliminate the distribution differences of the first data based on the second operator and the mean value to obtain the second data;
[0177] The third processing module 404 is used to perform aggregation and transformation processing on the first data based on the first operator and the third operator, with the mean as the benchmark, to obtain the third data;
[0178] The fourth processing module 405 is used to eliminate the scale bias of the second data based on the third operator and using the third data as a calibration factor to obtain the normalized first data.
[0179] The data processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.
[0180] In some embodiments, the first processing module 402 is specifically used for:
[0181] Determine the first quantization scaling factor and weight parameter of the first operator. The first quantization scaling factor is the reciprocal of the total number of data points of the first data, and the weight parameter is a preset value.
[0182] Based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data.
[0183] Multiply the first quantization scaling factor by the fourth data point to obtain the mean of the first data point.
[0184] In some embodiments, the third processing module 404 is specifically used for:
[0185] Based on the third operator, determine the difference between each data point in the first data and the mean;
[0186] Based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point.
[0187] Based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data.
[0188] In some embodiments, for any single data point; the third processing module 404 is specifically used for:
[0189] Determine the second quantization scaling factor;
[0190] Based on the third operator, the initial parameters are obtained by squaring the differences between the data points.
[0191] Based on the third operator, the product of the initial parameter and the second quantization scaling factor is used to determine the first parameter corresponding to the data point.
[0192] In some embodiments, the third processing module 404 is specifically used for:
[0193] Determine the third quantization scaling factor of the first operator, where the third quantization scaling factor is the ratio of the first quantization scaling factor to the second quantization scaling factor;
[0194] Based on the first operator, multiple first parameters are aggregated to obtain the fifth data.
[0195] Based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data;
[0196] The third data is determined based on the sixth data and the preset mapping table.
[0197] In some embodiments, the fourth processing module 405 is specifically used for:
[0198] The product of the third data, the second data, and the preset second parameter is determined as the scale calibration value. The second parameter is used to perform scale deviation calibration on the second data.
[0199] The sum of the scale calibration value and the preset third parameter is determined as the normalized first data. The third parameter is used to compensate for the reference offset of the scale calibration value.
[0200] The data processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.
[0201] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 may include: a transceiver 501, a processor 502, and a memory 503.
[0202] Processor 502 executes computer execution instructions stored in memory, causing processor 502 to perform the scheme in the above embodiments. Processor 502 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0203] The memory 503 is connected to the processor 502 via the system bus and completes communication between them. The memory 503 is used to store computer program instructions.
[0204] Transceiver 501 can be used to obtain the task to be run and its configuration information.
[0205] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.
[0206] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.
[0207] This application also provides a chip for executing instructions, which is used to execute the data processing method described in the above embodiments.
[0208] This application also provides a computer-readable storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform the technical solution of the data processing method described in the above embodiments.
[0209] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the data processing method in the above embodiments.
[0210] This application provides a chip, which includes at least one processor. The processor is used to execute program instructions to implement the technical solution of the data processing method described in the first aspect above and various possible designs of the first aspect.
[0211] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.
[0212] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.
[0213] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.
[0214] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.
[0215] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.
[0216] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.
[0217] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0218] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0219] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.
[0220] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A data processing method, characterized in that, include: Get the first data to be processed; Based on the first operator, the first data is processed to obtain the mean of the first data; Based on the second operator, and using the mean as a benchmark, the distribution differences of the first data are eliminated to obtain the second data; Based on the first operator and the third operator, and using the mean as a benchmark, the first data is aggregated and transformed to obtain the third data; Based on the third operator, and using the third data as a calibration factor, the scale bias of the second data is eliminated to obtain the normalized first data.
2. The method according to claim 1, characterized in that, The first data is processed to obtain the mean of the first data, including: Determine the first quantization scaling factor and weight parameter of the first operator, wherein the first quantization scaling factor is the reciprocal of the total number of data points of the first data, and the weight parameter is a preset value; Based on the weight parameters, the data points in the first data are aggregated to obtain the fourth data; Multiply the first quantization scaling factor by the fourth data to obtain the mean of the first data.
3. The method according to claim 1, characterized in that, Based on the first operator and the third operator, and using the mean as a benchmark, the first data is aggregated and transformed to obtain the third data, including: Based on the third operator, the difference between each data point in the first data and the mean is determined; Based on the third operator, the difference between each data point in the first data is squared to obtain the first parameter corresponding to each data point. Based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data.
4. The method according to claim 3, characterized in that, For any given data point; based on the third operator, the differences between the data points in the first data are squared to obtain the first parameter corresponding to each data point, including: Determine the second quantization scaling factor; Based on the third operator, the difference between the data points is squared to obtain the initial parameters; Based on the third operator, the product of the initial parameter and the second quantization scaling factor is determined as the first parameter corresponding to the data point.
5. The method according to claim 4, characterized in that, Based on the first operator, multiple first parameters are aggregated and transformed to obtain the third data, including: Determine the third quantization scaling factor of the first operator, wherein the third quantization scaling factor is the ratio of the first quantization scaling factor to the second quantization scaling factor; Based on the first operator, the multiple first parameters are aggregated to obtain the fifth data; Based on the first operator, the product of the third quantization scaling factor and the fifth data is determined as the sixth data; The third data is determined based on the sixth data and the preset mapping table.
6. The method according to claim 1, characterized in that, Using the third data as a calibration factor, the scale bias of the second data is eliminated to obtain the normalized first data, including: The product of the third data, the second data, and the preset second parameter is determined as the scale calibration value, and the second parameter is used to perform scale deviation calibration on the second data. The sum of the scale calibration value and the preset third parameter is determined as the normalized first data, and the third parameter is used to perform reference offset compensation on the scale calibration value.
7. A data processing apparatus, characterized in that, include: The module comprises an acquisition module, a first processing module, a second processing module, a third processing module, and a fourth processing module, wherein: The acquisition module is used to acquire the first data to be processed; The first processing module is used to process the first data based on the first operator to obtain the mean of the first data; The second processing module is used to eliminate the distribution differences of the first data based on the second operator and the mean value to obtain the second data; The third processing module is used to perform aggregation and transformation processing on the first data based on the first operator and the third operator, with the mean value as the benchmark, to obtain the third data; The fourth processing module is used to eliminate the scale bias of the second data based on the third operator and using the third data as a calibration factor to obtain the normalized first data.
8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.