Data processing method, computing unit, electronic device, storage medium and program product

Abstract

The embodiment of the invention relates to a data processing method, a computing unit, electronic equipment, a storage medium and a program product. The method is executed by a calculation unit and comprises the steps that first data to be processed are obtained, and the first data correspond to a weight matrix; analyzing the first data into a plurality of data segments, the plurality of data segments including a plurality of first data segments and a second data segment, the plurality of first data segments corresponding to the floating point value of the first precision; based on a preset corresponding relation, multiple basic values corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment are determined, the multiple basic values correspond to floating point values of second precision, the second precision is higher than the first precision, and the product of the multiple basic values and the scaling factor corresponds to multiple weight values in a weight matrix; and performing multiplication calculation of the weight matrix and the input matrix based on the plurality of base values and the scaling factor. In this way, the decoding and calculation efficiency of the data can be effectively improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods for data processing, computing units, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] With the rapid development of computer technology, the parameter scale of deep learning models has expanded rapidly from hundreds of millions to billions or even trillions. However, with the expansion of parameter scale, the computational cost of model training and inference, as well as memory bandwidth, have become the main bottlenecks of deep learning technology. Summary of the Invention

[0003] In a first aspect of this disclosure, a data processing method is provided, executed by a computing unit. The method includes: using a storage module of the computing unit to acquire first data to be processed, the first data corresponding to a weight matrix; using a parsing module of the computing unit to parse the first data into multiple data segments, the multiple data segments including multiple first data segments and a second data segment, the multiple first data segments corresponding to floating-point values ​​of a first precision; using a mapping module of the computing unit, based on a preset correspondence, determining multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment, the multiple base values ​​corresponding to floating-point values ​​of a second precision, the second precision being higher than the first precision, wherein the product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix; and using a multiplier and an accumulator of the computing unit, based on the multiple base values ​​and the scaling factor, performing a multiplication calculation of the weight matrix and an input matrix.

[0004] In a second aspect of this disclosure, a computational unit for data processing is provided. The computational unit includes: a storage module, a parsing module, a mapping module, a multiplier, and an accumulator. The storage module is used to acquire first data to be processed, the first data corresponding to a weight matrix; the parsing module is used to parse the first data into multiple data segments, the multiple data segments including multiple first data segments and one second data segment, the multiple first data segments corresponding to floating-point values ​​of a first precision; the mapping module is used to determine, based on a preset correspondence, multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment, the multiple base values ​​corresponding to floating-point values ​​of a second precision, the second precision being higher than the first precision, wherein the product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix; and the multiplier and accumulator are used to perform multiplication calculations of the weight matrix and the input matrix based on the multiple base values ​​and the scaling factor.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one computing unit; and at least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit. When executed by the at least one computing unit, the instructions cause the electronic device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] The solution provided in this disclosure can parse the first data corresponding to the weight matrix into multiple first data segments and a second data segment. Then, based on a preset correspondence, it determines multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment. The product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix. Thus, it can perform multiplication calculations with the input matrix based on the multiple base values ​​and the scaling factor, effectively improving the data decoding and calculation efficiency.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram is shown of an example environment in which embodiments of the present disclosure may be implemented; Figure 2 A flowchart illustrating an example processing procedure for data processing according to some embodiments of this disclosure is shown; Figure 3 A schematic diagram of the implementation architecture of data processing according to some embodiments of the present disclosure is shown; Figure 4 A flowchart illustrating an example construction process of first data according to some embodiments of this disclosure is shown; Figure 5A schematic structural block diagram of an example computing unit for data processing according to some embodiments of the present disclosure is shown; Figure 6 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0011] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0012] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0013] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0014] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0015] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0016] As mentioned above, with the rapid expansion of parameter scale, the computational cost and memory bandwidth of model training and inference have become the main bottlenecks for deep learning technology. Taking models represented by the Transformer architecture as an example, its core operations are mainly concentrated in matrix multiplication and attention mechanisms, accounting for more than 80% of the computational load on accelerators such as GPUs and NPUs. To reduce memory bandwidth pressure and power consumption, the industry has widely researched low-bit floating-point and mixed-precision integer computing.

[0017] Existing mixed-precision computing and quantization techniques have made significant progress in deep learning inference, but they still have the following obvious technical shortcomings when it comes to model deployment and training scenarios with large-scale data: there is a clear contradiction between compression ratio and numerical accuracy, and the loss of accuracy leads to unstable gradient propagation or training divergence; the granularity of scaling factors is not reasonably designed, making it difficult to balance local adaptability and storage efficiency; the data storage format and computing unit are not integrated, resulting in a path separation between data storage and computing, requiring a caching module to store the decoded data, increasing design complexity, access and storage latency and energy consumption, and creating a memory access bottleneck.

[0018] The embodiments of this disclosure propose a data processing scheme. The scheme includes: using the storage module of a computing unit to acquire first data to be processed, the first data corresponding to a weight matrix; using the parsing module of the computing unit to parse the first data into multiple data segments, the multiple data segments including multiple first data segments and one second data segment, the multiple first data segments corresponding to floating-point values ​​of a first precision; using the mapping module of the computing unit, based on a preset correspondence, determining multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segments, the multiple base values ​​corresponding to floating-point values ​​of a second precision, the second precision being higher than the first precision, wherein the product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix; and using the multiplier and accumulator of the computing unit, based on the multiple base values ​​and the scaling factor, performing a multiplication calculation between the weight matrix and the input matrix.

[0019] In this manner, embodiments of the present disclosure can parse the first data corresponding to the weight matrix into multiple first data segments and a second data segment. Then, based on a preset correspondence, multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment are determined. The product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix. Thus, multiplication calculations can be performed on the input matrix based on the multiple base values ​​and the scaling factor, effectively improving the data decoding and calculation efficiency.

[0020] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0021] Example environment: Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include computing unit 130.

[0022] refer to Figure 1 As shown, the calculation unit 130 can perform calculations on the first matrix 110 and the second matrix 120 to obtain a calculation result 140. As an example, such calculations may include, but are not limited to, at least one of the following: addition calculations, multiplication calculations, subtraction calculations, division calculations, etc.

[0023] The first matrix 110 can be a data matrix of any size, and the second matrix 120 can also be a data matrix of any size; no limitation is made here. The data format and matrix size of the calculation result 140 can be determined based on the data sizes of the first matrix 110 and the second matrix 120, as well as the calculation method performed by the calculation unit 130.

[0024] In some embodiments, the computing unit 130 may be a standalone electronic device or a software or hardware unit embedded in an electronic device. As an example, such an electronic device may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device may also support any type of user-facing interface (such as "wearable" circuitry).

[0025] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0026] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0027] Example process: Figure 2 A flowchart of an example data processing procedure 200 according to some embodiments of the present disclosure is shown. Processing procedure 200 can be implemented at computing unit 130. Reference is made below. Figure 1 To describe the processing procedure 200.

[0028] like Figure 2 As shown in box 210, the computing unit 130 uses the storage module to obtain the first data to be processed, which corresponds to the weight matrix.

[0029] In some scenarios, the storage module can be a component of the computing unit 130.

[0030] As an example, such a weight matrix can correspond to Figure 1 The first matrix 110 or the second matrix 120 in the matrix.

[0031] In some scenarios, the computing unit 130 can obtain the first data to be processed from an external device using the storage module, or it can obtain the first data from the storage module itself. As an example, such an external device can be a device configured on the same device as the computing unit 130, or it can be other devices or components capable of communicating with the device configured with the computing unit 130; this is not limited here.

[0032] In some embodiments, the first data includes data units corresponding to a preset data format, the bit width of the data units being matched with the calculation unit 130, and the preset data format including: multiple weight indices, the multiple weight indices corresponding to the same first bit width; and a scaling index, the scaling index corresponding to a second bit width, the second bit width being greater than the first bit width.

[0033] This embodiment of the disclosure uses a data unit with a bit width matching the computing unit 130, which can perform calculations immediately after decoding the first data without the need for other intermediate storage devices. It can realize simultaneous decoding and calculation of data, and can take into account both data storage and calculation. This effectively reduces the structural complexity of the computing unit, reduces the latency and energy consumption of data access, calculation, storage and other processing processes, and improves data processing efficiency.

[0034] As an example, matching the bit width of the data unit to the computation unit 130 can include having the same bit width as the computation unit 130. For example, both the bit width of the data unit and the bit width of the computation unit 130 may be 32 bits or 64 bits.

[0035] As an example, matching the bit width of the data unit to the computation unit 130 may also include the computation unit 130 having a bit width that is an integer multiple of the bit width of the data unit. For example, if the bit width of the data unit is 32 bits, the bit width of the computation unit 130 may be 32 bits, 64 bits, 128 bits, etc.

[0036] In some embodiments, the bit width of the data unit corresponding to the preset data format is 32 bits, and the multiple weight indices include four weight indices, wherein the first bit width of each weight index is 5 bits, and the second bit width of the scaling index is 12 bits.

[0037] In some implementations of this scheme, the five-bit width of the weight index can include a sign bit, an exponent bit, and a mantissa bit. For example, a five-bit wide weight index can include a one-bit wide sign bit, a two-bit wide exponent bit, and a two-bit wide mantissa bit to balance the dynamic range and precision of the data. In some scenarios, a five-bit wide weight index can also include a three-bit wide exponent bit, a one-bit wide sign bit, and a one-bit wide mantissa bit to provide a larger exponent range, suitable for sparse or high-dynamic-range network layers. In some scenarios, a five-bit wide weight index can also include a one-bit wide exponent bit, a one-bit wide sign bit, and a three-bit wide mantissa bit to improve data precision in low dynamic range regions, suitable for activation functions or normalization layers.

[0038] As an example, the fields of a data unit can be assigned according to the structure in the following table:

[0039] In the table above, a+b+c=5, and a, b, and c are all positive integers.

[0040] As an example, the scaling index can be a fixed point with Q6.5 of log2 (e.g., represented as...). This allows for easy lookup of the corresponding scaling factor. In some scenarios, the scaling index can also use a linear scaling ratio of Q8.4 fixed-point to meet the hardware requirements that do not support exponential lookup. In some scenarios, the scaling index can also use a 10-bit logarithmic index and 2 reserved bits to balance the data compression range and compression accuracy.

[0041] In some implementations, taking the scaling index as an example with a log2 Q6.5 fixed-point representation, the decoding process corresponding to the data unit allocated according to this structure can be represented as: ; In the formula This represents the i-th weight value after decoding the i-th weight index. This represents the i-th base value corresponding to the i-th weight index according to the above structure.

[0042] In this way, the present invention can achieve local logarithmic scaling with minimal granularity by embedding four 5-bit weight indexes and one 12-bit scaling index in a 32-bit storage unit, thus overcoming the problem of excessively coarse data granularity in the prior art and effectively improving data compression accuracy.

[0043] In some embodiments, the four weight indices can represent 32 different quantization levels using a five-bit width. For example, the weight indices can be any value in the symmetric quantization range of -16 to +15, or any value in the asymmetric quantization range of 0 to 31. In the asymmetric quantization range, this scheme reduces accumulated error by retaining 0 to represent precise zero.

[0044] In some embodiments, a 12-bit wide scaling index can be used to index 4096 different scaling factors, enabling high-precision storage of scaling coefficients and effectively improving the data precision corresponding to the weight matrix.

[0045] In this scheme, the computing unit 130 can read 32 bits of data at once, which is exactly a complete quantization weight reassembly, without the need for additional unpacking hardware. The four weight indices share a scaling factor corresponding to a scaling index, which not only ensures the data compression rate but also maintains the data compression accuracy. Furthermore, each 32-bit wide data can be decoded independently, enabling on-demand computation without the need for an additional buffer for data expansion. This reduces the structural complexity of the computing unit, decreases latency and energy consumption in data access, computation, and storage processes, and improves data processing efficiency.

[0046] In box 220, the calculation unit 130 uses the parsing module to parse the first data into multiple data segments, the multiple data segments including multiple first data segments and one second data segment, the multiple first data segments corresponding to floating-point values ​​of first precision.

[0047] In some scenarios, the parsing module used to parse the first data can be a component of the computing unit 130.

[0048] As an example, multiple first data segments correspond to multiple weight indices, and one second data segment corresponds to one scaling index. The calculation unit 130 uses the parsing module to parse the first data into multiple weight indices and one scaling index, where the multiple weight indices share a scaling factor corresponding to the scaling index.

[0049] In some embodiments, such weight indices are used to represent the relative values ​​of weights. Such scaling factors can represent the local dynamic range of the weight matrix, such as the scaling range of a single weight value or multiple weight values ​​corresponding to multiple weight indices in the weight matrix.

[0050] In box 230, the calculation unit 130 uses the mapping module to determine multiple base values ​​corresponding to multiple first data segments and scaling factors corresponding to second data segments based on a preset correspondence. The multiple base values ​​correspond to floating-point values ​​of second precision, which is higher than the first precision. The product of the multiple base values ​​and the scaling factors corresponds to multiple weight values ​​in the weight matrix.

[0051] As an example, the mapping module can be a component of the computation unit 130. The preset correspondence can include a first correspondence between multiple base values ​​and multiple first data segments or multiple weight indices, and can also include a second correspondence between scaling factors and second data segments or scaling indices.

[0052] In some embodiments, the mapping module of the computing unit 130 determines, based on a preset correspondence, multiple base values ​​corresponding to multiple first data segments and scaling factors corresponding to second data segments, including: using a first lookup table maintained by the computing unit 130, the computing unit 130 uses the mapping module to find multiple base values ​​corresponding to multiple first data segments, the first lookup table indicating a first correspondence between different indices and different base values; and using a second lookup table maintained by the computing unit, the mapping module finds scaling factors corresponding to second data segments, the second lookup table indicating a second correspondence between different encoding representations and different scaling factors.

[0053] As an example, the indexes in the first lookup table include the weight indexes corresponding to the first data segments. Each weight index in the first lookup table has a unique corresponding base value. That is, there is a one-to-one correspondence between the multiple weight indices corresponding to the multiple first data segments and the multiple base values ​​in the first lookup table.

[0054] As an example, the encoded representation in the second lookup table may include the scaling index corresponding to the second data segment. The scaling index has a unique encoded representation that has a unique corresponding scaling factor in the second lookup table.

[0055] In this way, the embodiments of this disclosure can quickly and accurately determine multiple base values ​​corresponding to multiple first data segments and scaling factors corresponding to second data segments based on the first lookup table and the second lookup table, realize parallel access and lookup of base values ​​and scaling factors, effectively improve the lookup efficiency of base values ​​and scaling factors, and thus improve data processing efficiency.

[0056] In box 240, computation unit 130 uses multipliers and accumulators to perform multiplication of the weight matrix with the input matrix based on multiple base values ​​and scaling factors.

[0057] As an example, multipliers and accumulators can be components of the computation unit 130.

[0058] As an example, the weight matrix in this scheme can correspond to Figure 1 The first matrix 110 in the input matrix can correspond to... Figure 1 The second matrix 120 in the input matrix; or, the input matrix corresponds to Figure 1 The first matrix 110 in the matrix corresponds to the weight matrix. Figure 1The second matrix in the matrix is ​​120.

[0059] In some embodiments, the calculation unit 130 can use a multiplier to perform multiplication calculations on multiple base values ​​and scaling factors to obtain multiple weight values ​​corresponding to the multiple base values, thereby obtaining a weight matrix corresponding to the first data; then, an accumulator is used to calculate the weight matrix and the input matrix. Thus, the embodiments of this disclosure can realize the process of decoding the first data and calculating with the input matrix, and can decode and calculate the first data simultaneously without intermediate storage, effectively improving data processing efficiency.

[0060] In this manner, embodiments of the present disclosure can parse the first data corresponding to the weight matrix into multiple first data segments and a second data segment. Then, based on a preset correspondence, multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment are determined. The product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix. Thus, multiplication calculations can be performed on the input matrix based on the multiple base values ​​and the scaling factor, effectively improving the data decoding and calculation efficiency.

[0061] It should be noted that in the scheme provided in this disclosure, the first data can correspond to multiple scaling factors to achieve multi-level data compression. For example, the first data can also share other scaling factors with the data corresponding to other weight matrices to achieve two-level data compression, which facilitates maintaining numerical balance in ultra-large models. The calculation process between the first data after multi-level compression and the input matrix can still be performed using the scheme provided in this disclosure.

[0062] In some embodiments, the scaling factor described above is a first scaling factor specific to the first data, and the calculation of multiplication of the weight matrix and the input matrix is ​​performed using a multiplier and an accumulator based on multiple base values ​​and the scaling factor, including: the calculation unit 130 first determines a second scaling factor corresponding to the first data, such second scaling factor corresponds to multiple data units, the multiple data units include at least a first data unit corresponding to the first data and a second data unit corresponding to the second data, and the first data and the second data correspond to the same data structure; then the calculation unit 130 performs the multiplication of the weight matrix and the input matrix using a multiplier and an accumulator based on multiple base values, the first scaling factor and the second scaling factor.

[0063] As an example, the calculation unit 130 performs multiplication calculation based on the product of multiple base values ​​and the first scaling factor and the second scaling factor to obtain multiple weight values, thereby determining the weight matrix corresponding to the first data; then the calculation unit 130 uses an accumulator to calculate the multiple weight values ​​and the input matrix.

[0064] In the data processing scheme provided in this disclosure, the decoding process of the weight matrix can be completed entirely in the register or on-chip cache without the need to add access memory or intermediate cache devices. This can effectively reduce the structural complexity of the computing unit, reduce the latency and energy consumption of data access, calculation, storage and other processing processes, and improve data processing efficiency.

[0065] In some embodiments, the input matrix can be a data matrix of any structure in the prior art. As an example, the input matrix can have the same matrix structure as the weight matrix corresponding to the first data, or it can be a matrix of other structures, such as BF16, FP16, or FP32.

[0066] In some embodiments of this disclosure, after calculating the weight matrix and the input matrix, the output of the calculation unit 130 can be a matrix with a structure such as BF16 or FP32, to facilitate the execution of other corresponding running processes. In some implementations, the output of the calculation unit 130 can also be a data matrix with the same structure as the weight matrix, to facilitate data storage, etc.

[0067] Figure 3 A schematic diagram of an example implementation architecture 300 of a data processing method according to some embodiments of the present disclosure is shown. (Reference) Figure 3 As shown, architecture 300 may include controller 310 and computing unit 130.

[0068] In some scenarios, the controller 310 can perform processes such as model training or inference. When performing model training or inference, the controller 310 issues corresponding calculation instructions to the computing unit 130, so that the computing unit 130 can complete the calculation process corresponding to the calculation instructions. As an example, such calculation processes may include, but are not limited to, matrix multiplication operations.

[0069] As an example, controller 310 may include a DMA (Direct Memory Access) / instruction control module 315. Controller 310 may communicate with computing unit 130 via the DMA / instruction control module 315 to send corresponding calculation instructions to computing unit 130. Such calculation instructions may include at least one first data item corresponding to the weight matrix to be calculated and an input matrix for calculation with the weight matrix.

[0070] Reference Figure 3As shown, the computing unit 130 may include an SRAM / HBM storage module 321. As an example, the SRAM / HBM storage module 321 may be used to store at least one first data item corresponding to the weight matrix; for example, such first data may include at least one data unit of a 32-bit data structure. Upon receiving a computing instruction, the computing unit 130 may obtain the first data to be processed from the SRAM / HBM storage module 321, or from the storage module of the controller 310 via the SRAM / HBM storage module 321, or from an external storage module that can be directly or indirectly communicated with the computing unit 130 via the SRAM / HBM storage module 321.

[0071] Then, the computing unit 130 uses the SRAM / HBM storage module 321 to send the first data to be processed to the BitSplit module 322 for data parsing. Continuing with the 32-bit data structure as an example, the BitSplit module 322 can parse the first data into multiple first data segments and one second data segment. For example, the BitSplit module 322 can parse the first data into four 5-bit first data segments and one 12-bit second data segment to obtain four 5-bit weight indices and one 12-bit scaling index.

[0072] The calculation unit 130 can use the FP5 lookup table module 323 to determine multiple base values ​​corresponding to multiple weight indices from the first lookup table based on the weight indices corresponding to multiple first data segments; and use the Scale lookup table module 324 to determine the scaling factor corresponding to the scaling index from the second lookup table based on the second data segments. As an example, the FP5 lookup table module 323 and the Scale lookup table module 324 can be ROM (read-only) modules, SRAM modules, or other implementable structural modules, which are not limited here.

[0073] As an example, the first lookup table may include a 32-entry value table to represent the mapping relationship between FP5 and FP32. The second lookup table may include 4096 indices to determine the scaling factor that uniquely corresponds to the scaling index.

[0074] Then, the calculation unit 130 can use the multiplier array 325 to perform multiplication calculations on multiple base values ​​and scaling factors respectively to obtain multiple weight values ​​corresponding to the multiple base values. Such multiple weight values ​​are at least a portion of the weight values ​​in the weight matrix corresponding to the calculation instruction.

[0075] As an example, the multiplier array 325 can perform multiple multiplication calculations between base values ​​and scaling factors in parallel to effectively improve the data decoding and computation efficiency of the computing unit 130. For example, the multiplier array 325 may include four parallel FP32 multipliers.

[0076] The multiplier array 325 sends the calculated weight values ​​to the adder array 326 to perform the calculation between the weight matrix and the input matrix.

[0077] As an example, the DMA / instruction control module 315 can directly send the input matrix corresponding to the calculation instruction to the adder array 326 of the calculation unit. The adder array 326 performs the corresponding matrix multiplication operation on the weight matrix and the input matrix, and the output module 327 outputs the corresponding calculation result.

[0078] In some implementations, the accumulator array can support modules that accumulate 32×32 local data blocks.

[0079] In some implementations, output module 327 can output data in BF16 or FP32 structures.

[0080] In architecture 300, the data decoding and computation process performed by computing unit 130 can realize data decoding and computation at the same time without the need for intermediate access storage or buffer modules. This can effectively reduce the structural complexity of computing unit, reduce latency and energy consumption in data access, computation, storage and other processing processes, and improve data decoding and computation efficiency.

[0081] Figure 4 A flowchart of an example construction process 400 for first data according to some embodiments of the present disclosure is shown. As an example, the construction process 400 may be implemented at the computing unit 130, or at other electronic devices or data processing modules. The computing unit 130 is used as the executor of the construction process 400 in the following illustrative description.

[0082] refer to Figure 4 As shown, the first data was constructed based on the following process: In box 410, the calculation unit 130 groups the weight matrix according to a preset dimension to determine multiple weight values ​​corresponding to the same group.

[0083] As an example, such a preset dimension can be either column-oriented or row-oriented.

[0084] In some embodiments, such a preset dimension may be set or adjusted according to the dimensional structure of the weight matrix.

[0085] Based on this preset dimension (e.g., column direction), the calculation unit 130 can group the weight matrix according to the preset data format corresponding to the first data to determine multiple weight values ​​within the same group. Taking a preset data format of four 5-bit weight indices and one 12-bit scaling index as an example, the calculation unit can group the weight matrix into groups of four weight values ​​to complete the grouping of the weight matrix. In some scenarios, if the number of remaining weight values ​​after grouping the weight matrix is ​​insufficient for a group, it can be padded with 0s or 1s to ensure that all weight values ​​in the weight matrix are grouped.

[0086] In box 420, computation unit 130 determines multiple base values ​​and scaling factors by quantizing multiple weight values.

[0087] As an example, such a quantization process can be determined based on a preset calculation method. For instance, such quantization can include any feasible method, such as taking the logarithm.

[0088] In some embodiments, determining multiple base values ​​and scaling factors by quantizing multiple weight values ​​includes: determining multiple logarithmic values ​​with a preset value as the base based on the multiple weight values; determining a reference logarithmic value based on the multiple logarithmic values; determining a scaling factor based on the reference logarithmic value; and determining multiple base values ​​based on the multiple weight values ​​and scaling factors.

[0089] As an example, the preset base can be 2 or 10, etc. For example, taking a preset base of 2 as an example, determining multiple logarithmic values ​​based on multiple weight values ​​can include: for the four weight values ​​in a group, calculating log2 of the absolute value of each weight value.

[0090] In some implementations, determining a reference logarithm based on multiple logarithmic values ​​may include performing a preset operation on the multiple logarithmic values ​​to determine the reference logarithm. For example, such a preset operation may include median calculation or weighted average calculation.

[0091] Continuing with the example of a preset value of 2, the scaling factor determined based on the reference logarithm can be expressed as: .

[0092] After determining the scaling factor, the calculation unit 130 can determine multiple base values ​​corresponding to the multiple weight values ​​based on the ratio of the multiple weight values ​​to the scaling factor. For example, for four weight values ​​within a group, the calculation unit 130 can use the result of dividing each weight value by the scaling factor as the corresponding base value.

[0093] In box 430, computation unit 130 determines multiple weight indices corresponding to multiple base values ​​to construct multiple first data segments.

[0094] As an example, the calculation unit 130 can determine multiple weight indices corresponding to multiple base values ​​based on a preset correspondence or transformation method. Such a correspondence can be a one-to-one mapping between weight indices and base values, and this mapping can be maintained by the calculation unit 130. As an example, such a mapping can be represented as a first lookup table maintained by the calculation unit 130.

[0095] In some embodiments, determining multiple weight indices corresponding to multiple base values ​​includes: converting multiple base values ​​into multiple weight indices based on a preset encoding format.

[0096] In this scheme, such an encoding format can be associated with the bit width corresponding to the weight index. Taking a weight index with a bit width of five bits as an example, such an encoding format can include a sign bit, an exponent bit, and a mantissa bit.

[0097] As an example, a 5-bit wide weight index can include a 1-bit sign bit, a 2-bit exponent bit, and a 2-bit mantissa bit to balance the dynamic range and precision of the data. In some scenarios, a 5-bit wide weight index can also include a 3-bit exponent bit, a 1-bit sign bit, and a 1-bit mantissa bit to provide a larger exponent range, suitable for sparse or high-dynamic-range network layers. In some scenarios, a 5-bit wide weight index can also include a 1-bit exponent bit, a 1-bit sign bit, and a 3-bit mantissa bit to improve data precision in low dynamic range regions, suitable for activation functions or normalization layers.

[0098] For multiple weight values ​​in a group of the weight matrix, after determining the multiple weight indices corresponding to each weight value, the calculation unit 130 can construct multiple corresponding first data segments. For example, the calculation unit 130 can convert a weight index into a corresponding five-bit wide representation to serve as a first data segment.

[0099] In box 440, computation unit 130 constructs a second data segment corresponding to the scaling factor.

[0100] As an example, the computing unit 130 can represent the scaling factor as twelve-bit wide data as a second data segment.

[0101] In some embodiments, the second data segment includes a reference logarithm value determined in block 420. As an example, such a reference logarithm value is a weighted average or median of the plurality of logarithms determined in block 420.

[0102] After constructing multiple first data segments and a corresponding second data segment, the computing unit 130 can package multiple (e.g., 4) first data segments and a corresponding second data segment into a 32-bit wide data to obtain a first data and store it.

[0103] Taking the 32-bit data structure in the table above as an example, this solution utilizes the 32-bit data format to directly map data to the GPU Warp register or ASIC SRAM bus, eliminating the need for additional packing and unpacking logic. Furthermore, this solution employs log2 scaling and 5-bit floating-point encoding to achieve higher compression ratios and dynamic ranges than existing technologies.

[0104] It should be noted that the process of obtaining the first data by weight matrix encoding and the data processing process provided in this disclosure can be applied to a variety of environments, such as model inference platforms, low-bandwidth embedded neural network processors (NPUs) performing matrix multiplication, training acceleration systems (capable of halving the video memory and maintaining accuracy during the training / distillation stage), FPGA / ASIC deployments (utilizing lookup tables and decoding pipelines to implement fixed instruction paths and improve cross-platform data consistency), and multimodal model fusion.

[0105] The solution disclosed herein also supports end-to-end model training and fine-tuning, is compatible with straight-through estimation (STE), and can maintain gradient stability during the model training phase. This solution uses a preset data format combined with STE training, which can improve the computational efficiency in forward propagation, while allowing the model to adapt to the preset data format through training. Compared with post-training quantization (PTQ), the accuracy loss is smaller.

[0106] The solution provided in this disclosure is applicable not only to platforms with a memory bus width of 32 bits, but also to platforms with a vertical width of 64 bits. As an example, the execution entity can package the two sets of first data compressed according to the above solution into a single 64-bit data set for storage or transfer, thereby reducing the number of read instructions. Correspondingly, the execution entity can also package four sets of first data compressed according to the above solution into a single 128-bit data set, or package more sets of first data together, to adapt to the bus width of the data platform.

[0107] Example devices and equipment: Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an example computing unit 130 for processing data according to certain embodiments of the present disclosure is shown. The various modules / components in the computing unit 130 may be implemented by hardware, software, firmware, or any combination thereof.

[0108] like Figure 5 As shown, the calculation unit 130 includes: a storage module 510, a parsing module 520, a mapping module 530, a multiplier 540, and an accumulator 550. The storage module 510 is used to acquire first data to be processed, which corresponds to a weight matrix. The parsing module 520 is used to parse the first data into multiple data segments, each including multiple first data segments and one second data segment. The multiple first data segments correspond to floating-point values ​​of a first precision. The mapping module 530 is used to determine, based on a preset correspondence, multiple base values ​​corresponding to the multiple first data segments and a scaling factor corresponding to the second data segment. The multiple base values ​​correspond to floating-point values ​​of a second precision, which is higher than the first precision. The product of the multiple base values ​​and the scaling factor corresponds to multiple weight values ​​in the weight matrix. The multiplier 540 and the accumulator 550 are used to perform multiplication calculations between the weight matrix and the input matrix based on the multiple base values ​​and the scaling factor.

[0109] In some embodiments, the first data includes data units corresponding to a preset data format, the bit width of the data units being matched with the calculation units, and the preset data format including: multiple weight indices, the multiple weight indices corresponding to the same first bit width; and a scaling index, the scaling index corresponding to a second bit width, the second bit width being greater than the first bit width.

[0110] In some embodiments, the data unit has a bit width of 32 bits, and the multiple weight indices include four weight indices, with the first bit width being 5 bits and the second bit width being 12 bits.

[0111] In some embodiments, the mapping module 530 is configured to: look up multiple base values ​​corresponding to multiple first data segments based on a first lookup table maintained by the computing unit, wherein the first lookup table indicates a first correspondence between different indices and different base values; and look up scaling factors corresponding to second data segments based on a second lookup table maintained by the computing unit, wherein the second lookup table indicates a second correspondence between different encoded representations and different scaling factors.

[0112] In some embodiments, the scaling factor is a first scaling factor specific to the first data, and the multiplier 540 and the accumulator 550 are configured to: determine a second scaling factor corresponding to the first data, the second scaling factor corresponding to a plurality of data units, the plurality of data units including at least a first data unit corresponding to the first data and a second data unit corresponding to the second data, the first data and the second data corresponding to the same data structure; and perform a multiplication calculation of the weight matrix and the input matrix based on a plurality of base values, the first scaling factor and the second scaling factor using the multiplier and the accumulator of the computation unit.

[0113] In some embodiments, the first data is constructed based on the following process: grouping the weight matrix according to a preset dimension to determine multiple weight values ​​corresponding to the same group; determining multiple base values ​​and scaling factors by quantizing the multiple weight values; determining multiple weight indices corresponding to the multiple base values ​​to construct multiple first data segments; and constructing second data segments corresponding to the scaling factors.

[0114] In some embodiments, determining multiple base values ​​and scaling factors by quantizing multiple weight values ​​includes: determining multiple logarithmic values ​​with a preset value as the base based on the multiple weight values; determining a reference logarithmic value based on the multiple logarithmic values; determining a scaling factor based on the reference logarithmic value; and determining multiple base values ​​based on the multiple weight values ​​and scaling factors.

[0115] In some embodiments, the second data segment includes a reference logarithm, which is a weighted average or median of multiple logarithms.

[0116] In some embodiments, determining multiple weight indices corresponding to multiple base values ​​includes: converting multiple base values ​​into multiple weight indices based on a preset encoding format.

[0117] like Figure 6 As shown, electronic device 600 is in the form of a general-purpose electronic device. Components of electronic device 600 may include, but are not limited to, at least one processor 610 or processing unit, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processor 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 600.

[0118] Electronic device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 600.

[0119] Electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0120] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 600 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0121] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 600, or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interfaces (not shown).

[0122] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0123] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0124] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0125] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0127] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method of data processing, performed by a computing unit, characterized in that, The method comprises: acquiring, by a storage module of the computing unit, first data to be processed, the first data corresponding to a weight matrix; parsing, by a parsing module of the computing unit, the first data into a plurality of data segments, the plurality of data segments comprising a plurality of first data segments and a second data segment, the plurality of first data segments corresponding to floating-point values of a first precision; determining, by a mapping module of the computing unit, a plurality of base values corresponding to the plurality of first data segments and a scaling factor corresponding to the second data segment based on a preset correspondence, the plurality of base values corresponding to floating-point values of a second precision, the second precision being higher than the first precision, wherein a product of the plurality of base values and the scaling factor corresponds to a plurality of weight values in the weight matrix; and performing, by a multiplier and an accumulator of the computing unit, multiplication calculation of the weight matrix and an input matrix based on the plurality of base values and the scaling factor.

2. The method of claim 1, wherein, The first data comprises a data unit corresponding to a preset data format, a bit width of the data unit matches the computing unit, and the preset data format comprises: a plurality of weight indexes corresponding to a same first bit width; a scaling index corresponding to a second bit width, the second bit width being greater than the first bit width.

3. The method of claim 2, wherein, The bit width of the data unit is thirty-two bits, the plurality of weight indexes comprises four weight indexes, the first bit width is five bits, and the second bit width is twelve bits.

4. The method of claim 1, wherein, The determining, by the mapping module of the computing unit, the plurality of base values corresponding to the plurality of first data segments and the scaling factor corresponding to the second data segment based on the preset correspondence comprises: finding, by the mapping module, the plurality of base values corresponding to the plurality of first data segments based on a first lookup table maintained by the computing unit, the first lookup table indicating a first correspondence between different indexes and different base values; and finding, by the mapping module, the scaling factor corresponding to the second data segment based on a second lookup table maintained by the computing unit, the second lookup table indicating a second correspondence between different encoding representations and different scaling factors.

5. The method of claim 1, wherein, The scaling factor is a first scaling factor specific to the first data, and the performing, by the multiplier and the accumulator of the computing unit, the multiplication calculation of the weight matrix and the input matrix based on the plurality of base values and the scaling factor comprises: determining a second scaling factor corresponding to the first data, the second scaling factor corresponding to a plurality of data units, the plurality of data units at least comprising a first data unit corresponding to the first data and a second data unit corresponding to second data, the first data and the second data corresponding to a same data structure; and performing, by the multiplier and the accumulator of the computing unit, the multiplication calculation of the weight matrix and the input matrix based on the plurality of base values, the first scaling factor, and the second scaling factor.

6. The method of claim 1, wherein, The first data is constructed based on the following process: grouping the weight matrix according to a preset dimension to determine the plurality of weight values corresponding to the same group; determining the plurality of base values and the scaling factor by quantizing the plurality of weight values; determining a plurality of weight indexes corresponding to the plurality of base values to construct the plurality of first data segments; and constructing the second data segment corresponding to the scaling factor. The determining the plurality of base values and the scaling factor by quantizing the plurality of weight values comprises:

7. The method of claim 6, wherein, determining a plurality of logarithm values with a preset value as a base based on the plurality of weight values; determining a reference logarithm value based on the plurality of logarithm values; determining the scaling factor based on the reference logarithm value; and determining the plurality of base values based on the plurality of weight values and the scaling factor. The second data segment includes the reference logarithm value, which is a weighted average or a median value of the plurality of logarithm values.

8. The method of claim 7, wherein, The determining the plurality of weight indexes corresponding to the plurality of base values comprises:

9. The method of claim 6, wherein, converting the plurality of base values to the plurality of weight indexes based on a preset encoding format. The computing unit comprises a storage module, an analysis module, a mapping module, a multiplier, and an accumulator, wherein:

10. A computing unit for data processing, characterized in that, The storage module is configured to obtain first data to be processed, the first data corresponding to a weight matrix; The analysis module is configured to parse the first data into a plurality of data segments, the plurality of data segments including a plurality of first data segments and a second data segment, the plurality of first data segments corresponding to floating-point values of a first precision; The mapping module is configured to determine, based on a preset correspondence, a plurality of base values corresponding to the plurality of first data segments and a scaling factor corresponding to the second data segment, the plurality of base values corresponding to floating-point values of a second precision, the second precision being higher than the first precision, wherein the product of the plurality of base values and the scaling factor corresponds to a plurality of weight values in the weight matrix; and The multiplier and the accumulator are configured to perform multiplication calculation of the weight matrix and an input matrix based on the plurality of base values and the scaling factor. The first data includes data units corresponding to a preset data format, the bit width of the data units matches the computing unit, and the preset data format includes:

11. The computing unit according to claim 10, characterized in that, a plurality of weight indexes corresponding to the same first bit width; a scaling index corresponding to a second bit width, the second bit width being greater than the first bit width. The bit width of the data units is thirty-two bits, the plurality of weight indexes includes four weight indexes, the first bit width is five bits, and the second bit width is twelve bits.

12. The computing unit according to claim 11, characterized in that, The mapping module is configured to:

13. The computing unit according to claim 10, characterized in that, find the plurality of base values corresponding to the plurality of first data segments based on a first lookup table maintained by the computing unit, the first lookup table indicating a first correspondence between different indexes and different base values; and find the scaling factor corresponding to the second data segment based on a second lookup table maintained by the computing unit, the second lookup table indicating a second correspondence between different encoding representations and different scaling factors. ​ ​ 14. The computing unit according to claim 10, characterized in that, The scaling factor is a first scaling factor specific to the first data, and the multiplier and the accumulator are configured to: determine a second scaling factor corresponding to the first data, the second scaling factor corresponding to a plurality of data units including at least a first data unit corresponding to the first data and a second data unit corresponding to second data, the first data and the second data corresponding to a same data structure; and perform, with the multiplier and the accumulator of the computing unit, a multiplication computation of the weight matrix and the input matrix based on the plurality of base values, the first scaling factor, and the second scaling factor.

15. The computing unit of claim 10, wherein, The first data is constructed based on a process comprising: grouping the weight matrix according to a preset dimension to determine the plurality of weight values corresponding to a same group; determining the plurality of base values and the scaling factor by quantizing the plurality of weight values; determining a plurality of weight indices corresponding to the plurality of base values to construct the plurality of first data segments; and constructing the second data segment corresponding to the scaling factor. The determining the plurality of base values and the scaling factor by quantizing the plurality of weight values comprises:

16. The computing unit according to claim 15, characterized in that, determining a plurality of logarithm values based on the plurality of weight values; determining a reference logarithm value based on the plurality of logarithm values; determining the scaling factor based on the reference logarithm value; and determining the plurality of base values based on the plurality of weight values and the scaling factor. The second data segment includes the reference logarithm value, which is a weighted average or a median value of the plurality of logarithm values.

17. The computing unit according to claim 16, characterized in that, The determining the plurality of weight indices corresponding to the plurality of base values comprises:

18. The computing unit of claim 15, wherein, converting the plurality of base values to the plurality of weight indices based on a preset encoding format. The electronic device comprises:

19. An electronic device, comprising: at least one computing unit; and at least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, the instructions, when executed by the at least one computing unit, causing the electronic device to perform the method according to any one of claims 1-9. The computer-executable instructions are executable by a processor to implement the method according to any one of claims 1-9.

20. A computer-readable storage medium having stored thereon computer- executable instructions, wherein, The computer-executable instructions, when executed by a device, cause the device to perform the method according to any one of claims 1-9.

21. A computer program product, the computer program product being tangibly stored in a computer storage medium and comprising computer-executable instructions, the computer program product being characterized in that, ​

Images