Weight quantization method and device based on hybrid segment coding and server

Abstract

The invention provides a weight quantification method and device based on mixed segment coding and a server, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a to-be-processed floating point weight and an activation value; based on the Laplacian distribution of the floating point weights, carrying out hybrid segmented coding processing on the floating point weights to obtain a coding weight set in a quantization weight coding format; and determining a multiplication coefficient and a shift value of the coding weight based on the identification bit, and carrying out multiplication and addition calculation processing on the activation value and the weight information in the data bit based on the multiplication coefficient and the shift value to obtain an output activation value corresponding to the coding weight. According to the method, the quantization precision of the neural network can be remarkably improved, and the quantization error is reduced while the compression ratio is not changed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a weight quantization method, apparatus and server based on hybrid segmented coding. Background Technology

[0002] With the rapid popularization of artificial intelligence in mobile and edge computing scenarios, weight quantization has become a key technology for ensuring the inference efficiency and deployment cost of deep neural networks. Currently, related technologies propose quantization methods such as tensor-by-tensor quantization, channel-by-channel quantization, and group quantization. However, tensor-by-tensor quantization easily introduces significant quantization errors when there are large differences in the numerical distribution between channels. Channel-by-channel quantization, when introducing significant numerical comparisons in the output channels, can easily lead to a large range of weight values ​​within each channel, thus reducing quantization accuracy. Group quantization has low accuracy when there are large differences in channel characteristics within a group. Therefore, the aforementioned existing technical solutions suffer from low quantization accuracy and large quantization errors. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a weight quantization method, apparatus and server based on hybrid segmented coding, which can significantly improve the quantization accuracy of neural networks and reduce quantization error.

[0004] In a first aspect, embodiments of the present invention provide a weight quantization method based on hybrid segmented coding. The method includes: obtaining floating-point weights and activation values ​​to be processed; performing hybrid segmented coding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of coded weights in a quantized weight coding format, wherein each coded weight in the set includes: data bits and identifier bits; determining the multiplication coefficients and shift values ​​of the coded weights based on the identifier bits, and performing multiplication and addition calculations on the activation values ​​and weight information in the data bits based on the multiplication coefficients and shift values ​​to obtain the output activation value corresponding to the coded weights.

[0005] In one implementation, after obtaining the floating-point weights and activation values ​​to be processed, the method includes: performing statistical analysis on the floating-point weights to obtain a data distribution histogram corresponding to the floating-point weights, and fitting the floating-point weights to the data based on a Laplace model to obtain a fitted probability density function; and combining the data distribution histogram with the probability density function to obtain the Laplace distribution of the floating-point weights.

[0006] In one implementation, the step of performing mixed segmented encoding on floating-point weights based on the Laplace distribution of floating-point weights to obtain a set of encoded weights in a quantized weight encoding format includes: performing segmented quantization on floating-point weights based on the Laplace distribution of floating-point weights, dividing the weight values ​​of the floating-point weights into a set of sub-intervals; mapping the weight values ​​within each sub-interval to fixed-point representation values ​​of data bits, and constructing weight grouping information for the identifier bits using the index identifiers corresponding to the sub-intervals; and combining the fixed-point representation values ​​with the corresponding weight grouping information to obtain the set of encoded weights.

[0007] In one implementation, the step of performing segmented quantization on the floating-point weights and dividing the weight values ​​of the floating-point weights into a set of sub-intervals includes: using the percentile segmentation method, with the weight value as the threshold, dividing the weight values ​​of the floating-point weights into a main peak region, a medium sparse region, and a positive value interval, and determining the main peak region, the medium sparse region, and the positive value interval as a set of sub-intervals.

[0008] In one implementation, the step of combining fixed-point representation values ​​with corresponding weight grouping information to obtain an encoded weight set includes: determining the corresponding weight grouping information based on the sub-interval to which the weight values ​​of each fixed-point representation value belong before mapping, and inserting the weight grouping information to the end of the corresponding fixed-point representation value to obtain an encoded weight set, wherein the weight grouping information corresponding to the weight values ​​in the same sub-interval is the same.

[0009] In one implementation, the step of determining the multiplication coefficient and shift value of the encoded weight based on the identifier bit includes: determining the multiplication coefficient and shift value of the encoded weight by performing a lookup table on the weight grouping information corresponding to any encoded weight.

[0010] In one implementation, the step of performing multiply-add calculations on the activation value and the weight information in the data bits to obtain the output activation value corresponding to the encoded weight includes: substituting the activation value and weight information into the multiply-add calculation model, and performing fixed-point conversion on the floating-point proportional terms in the multiply-add calculation model to convert the floating-point proportional terms into fixed-point form, so as to solve the multiply-add calculation model and obtain the output activation value.

[0011] Secondly, embodiments of the present invention also provide a weight quantization device based on hybrid segmented coding. The device includes: an information acquisition module for acquiring floating-point weights and activation values ​​to be processed; a hybrid segmented coding module for performing hybrid segmented coding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of coded weights in a quantized weight coding format, wherein each coded weight in the set includes: data bits and identifier bits; and a weight quantization module for determining the multiplication coefficients and shift values ​​of the coded weights based on the identifier bits, and performing multiplication and addition calculations on the activation value and the weight information in the data bits based on the multiplication coefficients and shift values ​​to obtain the output activation value corresponding to the coded weight.

[0012] Thirdly, embodiments of the present invention also provide a server, including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0014] The embodiments of the present invention bring the following beneficial effects: This invention provides a weight quantization method, apparatus, and server based on hybrid segmented coding. After obtaining the floating-point weights and activation values ​​to be processed, the method performs hybrid segmented coding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of coded weights in the quantized weight coding format. Then, based on the identifier bits, it determines the multiplication coefficients and shift values ​​of the coded weights. Based on the multiplication coefficients and shift values, it performs multiplication and addition calculations on the activation values ​​and the weight information in the data bits to obtain the output activation value corresponding to the coded weights. This invention can improve the quantization accuracy of neural networks and enhance the adaptability to non-uniform weight distributions within channels.

[0015] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a weight quantization method based on hybrid segmented coding provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating the specific process of a weight quantization method based on hybrid segmented coding provided in an embodiment of the present invention; Figure 3 A schematic diagram of a neural network weight distribution provided in an embodiment of the present invention; Figure 4 A schematic diagram of a 10-bit quantization weight bit division provided for an embodiment of the present invention; Figure 5 A schematic diagram of a weight quantization device based on hybrid segmented coding provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, with the popularization of artificial intelligence in mobile and edge computing scenarios, weight quantization has become a key technology to ensure the inference efficiency and deployment cost of deep neural networks. In the model deployment process, after the floating-point weights are loaded into the compiler's buffer, the quantization module first converts them into integer reference values ​​and writes the encoded weights into the device storage. Then, these integer weights are sent to the multiplication unit to participate in the operation. After accumulation and scaling, the dequantization module restores them to the approximate floating-point result and passes them to the subsequent layers. The entire process must balance the dynamic range and resolution under strict bandwidth and bit width budgets. Currently, the mainstream quantization methods mainly include perTensor quantization, perChannel quantization, and Group quantization strategies.

[0021] Related technologies propose that perTensor quantization uniformly assigns scaling and zeros to the entire tensor. While simple to implement and low in overhead, it easily introduces significant quantization errors when there are large differences in numerical distribution between channels. To alleviate this problem, perChannel quantization allows each channel to use quantization parameters independently, improving its adaptability to distribution differences. However, during neural network training, to enhance model expressiveness and improve the separability between features, significant numerical contrasts are often introduced in each output channel. This results in weight values ​​exhibiting extremely wide distributions even within a single channel. This high dynamic range compresses the effective precision range, making it difficult for perChannel quantization to achieve accurate approximation within that channel. Compared to perTensor, perChannel offers greater flexibility, but its accuracy improvement depends on the density and concentration of data within the channel. If the original distribution itself is scalable and non-uniform, the quantization effect remains limited. Furthermore, Group... Quantization, as a compromise, attempts to divide multiple channels into fixed groups to share quantization parameters, thereby improving accuracy while reducing complexity. However, grouping strategies are usually based on static structure settings and lack dynamic adaptation to differences in data distribution within the group. If the characteristics of the channels within the group are significantly different, it will still affect the final inference effect, limiting its versatility and stability.

[0022] Based on this, the weight quantization method, apparatus and server based on hybrid segmented coding provided by the present invention can significantly improve the quantization accuracy of neural networks and reduce quantization error while maintaining the same compression ratio.

[0023] See Figure 1 The diagram shows a flowchart of a weight quantization method based on hybrid segmented coding. This method mainly includes the following steps S102 to S106: Step S102: Obtain the floating-point weights and activation values ​​to be processed.

[0024] In one implementation, the floating-point weights are the original model parameters obtained during the training of the neural network model, and the activation values ​​are the values ​​input to the model. For example, there is a trained model in the time clock, and the information entered by each person when they clock in is the activation value.

[0025] Step S104: Based on the Laplace distribution of floating-point weights, perform mixed segmented encoding processing on the floating-point weights to obtain a set of encoded weights in the quantized weight encoding format. Each encoded weight in the set includes: data bits and identifier bits.

[0026] In one implementation, a segmented fixed-point quantization encoding method can be used to represent each neural network weight as a combination of an 8-bit quantization value and a 2-bit segment identifier. That is, the data bits are 8 bits and the identifier bits are 2 bits. First, based on the histogram distribution characteristics of the overall weight values, the weights in the same channel are divided into multiple sub-intervals (such as the main distribution area, the extended distribution area, and the outlier distribution area). Quantization parameters are configured independently for each sub-interval to achieve segmented linear quantization. Then, all weights are uniformly encoded to obtain 10-bit encoded fixed-point weights, where 8 bits are used to represent the quantization value of the weight in its respective segment, and 2 bits are used to identify the quantization segment identifier to which the weight belongs.

[0027] Furthermore, the above embodiment uses 8 bits to represent fixed-point values ​​and 2 bits to represent segment indexes, forming a 10-bit structure. In practical applications, it can also be extended to other bit-width structures such as 6+2 bits and 7+1 bits, adjusting the weight encoding precision and compression rate according to the target hardware resources.

[0028] Step S106: Determine the multiplication coefficient and shift value of the encoding weight based on the identifier bit, and perform multiplication and addition calculations on the activation value and the weight information in the data bits based on the multiplication coefficient and shift value to obtain the output activation value corresponding to the encoding weight.

[0029] In one implementation, the multiplication coefficient and shift value of the encoded weight can be determined by looking up the weight grouping information corresponding to any encoded weight in a table. In other words, the corresponding decoding parameters can be selected by reading the 2-bit identifier, thereby restoring each weight.

[0030] The weight quantization method based on hybrid segmented coding provided in this embodiment of the invention can significantly improve the quantization accuracy of neural networks and reduce quantization error while maintaining the same compression ratio.

[0031] This invention, through histogram distribution studies of the weights in YOLO series networks and Transformer series networks, found that the weight distribution of most neural networks has the following characteristics: 1. Highly concentrated near zero: The vast majority of weight values ​​are distributed in a narrow range centered on 0, forming a distinct dense main peak, which conforms to the central concentration characteristic of the Laplace distribution.

[0032] 2. Exhibits a long-tail distribution structure: Although the main peak area accounts for a very high proportion, there are still a small number of large positive and negative extreme weight values ​​that extend to both ends, forming the tail of the distribution, which is a typical long-tail feature.

[0033] 3. Significant consistency across layers: Similar weight distribution patterns can be observed in different network structures (YOLO and Transformer) and different layers (including convolutional layers and fully connected layers), indicating that this statistical characteristic is universal.

[0034] 4. Large differences in dynamic range within segments: Even within a single channel or block, there is often a coexistence of small values ​​and outlier large values, making it difficult to achieve uniform quantization accuracy while taking into account both the main dense area and the tail features.

[0035] 5. Insufficient sparsity but close to sparsity: Although most weights are not zero, the number of low-amplitude weights far exceeds that of high-amplitude weights, which has the potential for sparsity compression and segmented quantization optimization.

[0036] Based on this, see Figure 2 The diagram shows a specific flowchart of a weight quantization method based on hybrid segmented coding. This embodiment of the invention also provides an implementation method for determining the set of coding weights, as detailed in (1) to (3) below: (1) By performing statistical analysis on the floating-point weights, the data distribution histogram corresponding to the floating-point weights is obtained. The floating-point weights are then fitted with the Laplace model to obtain the fitted probability density function. The data distribution histogram and the probability density function are then combined to obtain the Laplace distribution of the floating-point weights.

[0037] In one implementation, see Figure 3 The diagram illustrates a neural network weight distribution. It shows that the weights exhibit a peak-shaped symmetrical distribution, with the main dense region concentrated near zero. Simultaneously, a certain proportion of outlier weights exist, forming a long-tailed distribution. The probability density function is fitted using a Laplace model, and the fitting results show that the weight distribution can be well represented by... , The Laplace distribution with parameter is characterized by the fact that most weights are concentrated in the main region close to 0, but there are still non-negligible outliers with high amplitude. If a uniform scaling parameter is used, it will be difficult to balance accuracy and range.

[0038] (2) Based on the Laplace distribution of floating-point weights, the floating-point weights are segmented and quantized, and the weight values ​​of the floating-point weights are divided into a set of sub-intervals. That is, by using the percentile segmentation method, the weight values ​​of the floating-point weights are divided into the main peak region, the medium sparse region and the positive value interval by using the weight value as the threshold, and the main peak region, the medium sparse region and the positive value interval are determined as a set of sub-intervals.

[0039] In one implementation, a segmented quantization method can be used to divide the weights into multiple sub-intervals according to their numerical ranges, and to linearly encode each sub-interval using independent quantization parameters. This achieves a refined representation of different distribution segments, reducing quantization errors and improving model accuracy. Based on the Laplace distribution, the fitted distribution function of the weights is assumed to be:

[0040] in, The distribution center (axis of symmetry, usually 0); The scaling parameter can be obtained through maximum likelihood estimation:

[0041] Based on the cumulative distribution properties of this distribution function, this invention uses the percentile segmentation method for segmentation, specifically (a) to (c): (a) Main peak region: Contains approximately 99% of the weight values ​​(in actual YOLO and Transformer weight measurements, over 99% of the values ​​are concentrated in this range). For the Laplace distribution, because:

[0042] For the Laplace distribution with scale parameter b (probability density function is...), ), by its probability density in the interval Integrating upwards (using the symmetry of the distribution to simplify the calculation), we can obtain... .because ,therefore Approximately 99.3%.

[0043] Therefore, the threshold selection is as follows:

[0044] The value density is highest within this interval, and the weight accuracy significantly affects the model output. Minimum quantization step is used to ensure accuracy. Since piecewise quantization is employed, each segment has a different numerical range, thus each segment has its own independent step value. A smaller step results in higher accuracy but a narrower range; a larger step results in a wider range but lower accuracy. In other words, the step represents the floating-point difference represented by each integer unit during quantization; a smaller step indicates higher precision, and a larger step indicates lower precision. The quantization parameters are calculated as follows:

[0045] (b) Moderately sparse region: This segment contains approximately 0.6% weight, with a magnitude between the main region and the tail. Threshold selection:

[0046] The corresponding cumulative probability is approximately 99.3%–99.96%. Data in this region is sparse, but not yet considered an outlier. Separating it into a separate segment can prevent incorrect compression at the tail. The quantization parameters are calculated as follows:

[0047] (c) Positive value interval: This segment contains a very small number of outlier values ​​(approximately 0.04%), has the largest numerical range, and the threshold selection is:

[0048] Although these values ​​occur with extremely low probability, they significantly broaden the uniform quantization scale; without separation, they would greatly dilute the precision of the main region. The quantization parameters are calculated as follows:

[0049] in, This represents the maximum absolute value of all values ​​in this weight group.

[0050] (3) Map the weight values ​​in each sub-interval to the fixed-point representation values ​​of the data bits, and use the index identifiers corresponding to the sub-intervals to construct the weight grouping information of the identifier bits, and combine the fixed-point representation values ​​with the corresponding weight grouping information to obtain the encoded weight set.

[0051] In one implementation, see Figure 4 The diagram shows a 10-bit quantization weight partitioning method. Based on the sub-interval to which the weight values ​​of each fixed-point representation belong before mapping, the corresponding weight grouping information is determined, and the weight grouping information is inserted at the end of the corresponding fixed-point representation value to obtain the encoded weight set. The weight values ​​in the same sub-interval have the same weight grouping information.

[0052] In other words, according to the above , , The quantization parameters of the three segments map the weights within their respective intervals to 8-bit fixed-point representations. Then, these 8-bit fixed-point values ​​are shifted left by 2 bits, and a 2-bit segment index identifier is inserted at the end, thus forming a representation as shown below. Figure 4 The 10-bit quantization weight encoding format shown can be extended to 2, 4 or more segments in practical applications, and the segment index bit width can be adjusted to 1 bit, 3 bits, etc.

[0053] The segmented weight quantization method described above encodes the weights as 8-bit quantized values ​​and 2-bit segment identifiers, improving accuracy while maintaining compact storage, and offering the following advantages: 1. Reduce quantization error without changing the compression ratio: By dividing the weight into up to 4 typical numerical distribution regions, each region adopts an independent quantization interval and step setting to achieve high-precision segmented quantization. The weight is identified by the last 2 bits of its segment, avoiding the precision loss caused by uniform parameters. This is especially suitable for scenarios where the dynamic range of values ​​in the channel varies greatly.

[0054] 2. Each weight carries independent quantization information: Compared to perTensor and perChannel quantization, which can only use one quantization parameter, this scheme allows each weight to use the corresponding quantization parameter according to its distribution segment, realizing adaptive quantization of weight granularity and greatly improving the feature representation capability and model expressiveness within the channel.

[0055] 3. Excellent flexibility and adaptability: The scheme supports configurable modeling for weight segmentation strategies, and can flexibly divide different quantization regions according to specific models, layer types or training statistical features to adapt to different structures, taking into account accuracy, resources and versatility. It is suitable for the efficient deployment of various network modules such as convolutional layers and fully connected layers.

[0056] Furthermore, this embodiment of the invention also provides a specific method for calculating the output activation value, as detailed in (1) to (3) below: (1) Fixed-point calculation of segmented quantization: Since the weight adopts a segmented quantization structure, that is, each quantized weight consists of an 8-bit main quantization value. With 2-bit segment index In this example, the activation value uses 8-bit asymmetric quantization, and the weights use 8-bit symmetric quantization. The multiplication and addition process of the activation value and weights can be represented as follows:

[0057] in, , These are the quantization scales for the activation value and the output, respectively. The scale is selected based on the weight segment index; , The zero point for both the activation value and the output; These are the quantization scales for the activation value and the output, respectively. This represents the activation value.

[0058] (2) Fixed-point conversion of floating-point multiplication: Substitute the activation value and weight information into the multiply-accumulate calculation model, and convert the floating-point proportional term in the multiply-accumulate calculation model into a fixed-point form to solve the multiply-accumulate calculation model and obtain the output activation value. Specifically, in order to achieve efficient fixed-point inference, the floating-point proportional term needs to be converted into a fixed-point form.

[0059] Convert to fixed-point form. Pre-calculate the fixed-point coefficients for each segment. With the corresponding shift amount Approximate it as:

[0060] Therefore, the entire reasoning and calculation are transformed into the following form:

[0061] in, It is a fixed-point integer (usually 16~32 bits). This is a displacement scaling term (which can be a constant or a lookup table value), avoiding runtime floating-point division.

[0062] (3) Hardware implementation optimization: In order to adapt to the pipeline structure, the multiplication coefficients of each segment can be pre-calculated in the actual calculation. With displacement Write to the hardware lookup table (LUT) and look up the segment index. Perform fast mapping. Considering hardware resource limitations, a maximum of four 32-bit accumulators can be designed to accommodate segmented quantization with a maximum of four segments. Each segment independently accumulates the multiplication and addition result within its corresponding segment, i.e.: 1. For each weight Select a group based on its 2-bit segment index. .

[0063] 2. Performing fixed-point multiplication: .

[0064] 3. Perform a right shift operation The result is accumulated into the register of the corresponding segment.

[0065] 4. When At that time, the results of the four segment registers are summed to obtain:

[0066] in, Indicates belonging to a segment The set of indices.

[0067] In summary, the above-described solution of the present invention has the following characteristics: 1. The weight quantization encoding method based on the 8+2 bit structure represents each neural network weight as a 10-bit binary format, where 8 bits represent the quantization value within the segment to which the weight belongs, and 2 bits are used to identify the segment number to which the weight belongs.

[0068] 2. A quantization method based on the statistical distribution of weights to divide the data into three segments. By fitting the Laplace distribution of the weight data, the range of weight values ​​is divided into three sub-intervals: the negative tail region, the main dense region, and the positive tail region using a fixed threshold (e.g., ±5b). Different quantization steps and scales are configured for each sub-interval. In addition to using a fixed threshold of ±5b as the segment boundary, the boundary can also be adaptively selected according to the statistical distribution of the actual network layer using methods such as percentile, histogram density threshold, or mean ± standard deviation.

[0069] 3. By setting independent linear quantization parameters within each segment, the scale and zero-point used in each segment are calculated independently based on the dynamic range of each segment interval, thereby improving the accuracy and adaptability of quantization for each segment.

[0070] 4. By combining the fixed-point multiply-add decoding structure with segment index lookup table, the multiplication coefficients and shift parameters of the corresponding segment are obtained through the lookup table (LUT) based on the 2-bit segment identifier carried by each weight during the inference stage, replacing floating-point scaling and achieving efficient fixed-point multiply-add calculation.

[0071] 5. By optimizing the hardware implementation of the segmented accumulator structure, the maximum number of supported segments is limited to 4, and 4 independent accumulators are configured to independently accumulate the multiplication and addition results of each segment, and finally merge and output them in a unified manner.

[0072] 6. Based on the weight restoration mechanism of encoding and decoding matching, the approximate floating-point weight value is restored by fixed-point multiplication and shift calculation according to the quantization parameter set mapped by the 2-bit segment index and the corresponding 8-bit valid value, so as to participate in the inference calculation.

[0073] 7. A general segmented quantization method applicable to modules such as convolutional layers and fully connected layers in neural networks, supporting weight encoding generation and decoding execution during static quantization and quantization-aware training (QAT) processes.

[0074] Therefore, in order to improve the quantization accuracy of neural networks and enhance their adaptability to non-uniform weight distribution within channels, this invention encodes each weight as a combination of an 8-bit data value and a 2-bit segment identifier. The 8 bits represent the value after local linear quantization, and the 2 bits identify the data distribution segment to which the weight belongs. By modeling the actual distribution of weights within a channel in a three-segment manner, different quantization intervals and step sizes can be used to effectively cover the main distribution area and outlier areas, balancing accuracy and range. This method breaks through the limitation of perTensor and perChannel quantization, which can only use uniform quantization parameters within a channel. It allows each weight to adaptively select the most suitable quantization strategy according to its local distribution characteristics, significantly reducing quantization errors without significantly increasing storage overhead, and improving the robustness and performance of the model under low-bit deployment.

[0075] Regarding the weight quantization method based on hybrid segmented coding provided in the foregoing embodiments, this invention provides a weight quantization device based on hybrid segmented coding, see [link to previous document]. Figure 5 The diagram shows a structural schematic of a weight quantization device based on hybrid segmented coding. The device includes the following parts: Information acquisition module 502 acquires the floating-point weights and activation values ​​to be processed; The hybrid segmented coding module 504 performs hybrid segmented coding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of coded weights in the quantized weight coding format. Each coded weight in the set includes: data bits and identifier bits. The weight quantization module 506 determines the multiplication coefficient and shift value of the encoded weight based on the identifier bit, and performs multiplication and addition calculations on the activation value and the weight information in the data bits based on the multiplication coefficient and shift value to obtain the output activation value corresponding to the encoded weight.

[0076] The weight quantization device based on hybrid segmented coding provided in this application embodiment can significantly improve the quantization accuracy of neural networks and reduce quantization error while maintaining the same compression ratio.

[0077] In one embodiment, after obtaining the floating-point weights and activation values ​​to be processed, the information acquisition module 502 is further configured to: perform statistical analysis on the floating-point weights to obtain a data distribution histogram corresponding to the floating-point weights, and perform data fitting on the floating-point weights based on the Laplace model to obtain a fitted probability density function; and combine the data distribution histogram with the probability density function to obtain the Laplace distribution of the floating-point weights.

[0078] In one embodiment, when performing a Laplace distribution based on floating-point weights and performing mixed segmented encoding processing on the floating-point weights to obtain a set of encoded weights in a quantized weight encoding format, the aforementioned mixed segmented encoding module 504 is further configured to: perform segmented quantization processing on the floating-point weights based on the Laplace distribution of floating-point weights, dividing the weight values ​​of the floating-point weights into a set of sub-intervals; map the weight values ​​within each sub-interval to fixed-point representation values ​​of data bits, and construct weight grouping information of the identifier bits using the index identifiers corresponding to the sub-intervals; combine the fixed-point representation values ​​with the corresponding weight grouping information to obtain a set of encoded weights.

[0079] In one embodiment, when performing segmented quantization of the floating-point weights and dividing the weight values ​​of the floating-point weights into a set of sub-intervals, the hybrid segmented encoding module 504 is further configured to: divide the weight values ​​of the floating-point weights into a main peak region, a medium sparse region, and a positive value interval using a percentile segmentation method with the weight value as a threshold, and determine the main peak region, the medium sparse region, and the positive value interval as a set of sub-intervals.

[0080] In one embodiment, when performing the step of combining the fixed-point representation value with the corresponding weight grouping information to obtain the encoded weight set, the above-mentioned hybrid segmented encoding module 504 is further configured to: determine the corresponding weight grouping information according to the sub-interval to which the weight value of each fixed-point representation value belongs before mapping, and insert the weight grouping information to the end of the corresponding fixed-point representation value to obtain the encoded weight set, wherein the weight grouping information corresponding to the weight values ​​in the same sub-interval is the same.

[0081] In one embodiment, when performing the step of determining the multiplication coefficient and shift value of the coding weight based on the identifier bit, the weight quantization module 506 is further configured to: determine the multiplication coefficient and shift value of the coding weight by performing a lookup table on the weight grouping information corresponding to any coding weight.

[0082] In one embodiment, when performing the step of multiplying and adding the activation value with the weight information in the data bits to obtain the output activation value corresponding to the encoded weight, the weight quantization module 506 is further configured to: substitute the activation value and weight information into the multiply-add calculation model, and perform fixed-point conversion on the floating-point proportional term in the multiply-add calculation model to convert the floating-point proportional term into a fixed-point form, so as to solve the multiply-add calculation model and obtain the output activation value.

[0083] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0084] This invention provides a server, specifically, the server includes a processor and a storage device; the storage device stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0085] Figure 6 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. The server 100 includes: a processor 60, a memory 61, a bus 62, and a communication interface 63. The processor 60, the communication interface 63, and the memory 61 are connected through the bus 62. The processor 60 is used to execute executable modules, such as computer programs, stored in the memory 61.

[0086] The memory 61 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 63 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0087] Bus 62 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0088] The memory 61 is used to store programs. After receiving an execution instruction, the processor 60 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0089] Processor 60 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 60 or by instructions in software form. The processor 60 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 61, and the processor 60 reads the information from memory 61 and, in conjunction with its hardware, completes the steps of the above method.

[0090] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0091] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A weight quantization method based on hybrid segmented coding, characterized in that, The method includes: Obtain the floating-point weights and activation values ​​to be processed; Based on the Laplace distribution of the floating-point weights, the floating-point weights are subjected to mixed segmented encoding processing to obtain a set of encoded weights in the quantized weight encoding format, wherein each encoded weight in the set of encoded weights includes: data bits and identifier bits; The multiplication coefficient and shift value of the encoded weight are determined based on the identifier bit. Based on the multiplication coefficient and shift value, the activation value is multiplied and added with the weight information in the data bit to obtain the output activation value corresponding to the encoded weight.

2. The weight quantization method based on hybrid segmented coding according to claim 1, characterized in that, After obtaining the floating-point weights and activation values ​​to be processed, the process includes: By performing statistical analysis on the floating-point weights, a histogram of the data distribution corresponding to the floating-point weights is obtained, and the floating-point weights are fitted based on the Laplace model to obtain the fitted probability density function. By combining the data distribution histogram with the probability density function, the Laplace distribution of the floating-point weights is obtained.

3. The weight quantization method based on hybrid segmented coding according to claim 1, characterized in that, The step of performing mixed segmented encoding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of encoded weights in a quantized weight encoding format includes: Based on the Laplace distribution of the floating-point weights, the floating-point weights are segmented and quantized to divide the weight values ​​of the floating-point weights into a set of sub-intervals. The weight values ​​within each sub-interval are mapped to the fixed-point representation values ​​of the data bits, and the weight grouping information of the identifier bits is constructed using the index identifiers corresponding to the sub-intervals. The fixed-point representation value is combined with the corresponding weight grouping information to obtain the encoded weight set.

4. The weight quantization method based on hybrid segmented coding according to claim 3, characterized in that, The step of performing segmented quantization on the floating-point weights, dividing the weight values ​​of the floating-point weights into a set of sub-intervals, includes: Using the percentile segmentation method, with the weight value as the threshold, the weight values ​​of the floating-point weights are divided into a main peak region, a medium sparse region, and a positive value interval, and the main peak region, the medium sparse region, and the positive value interval are determined as the set of sub-intervals.

5. The weight quantization method based on hybrid segmented coding according to claim 3, characterized in that, The step of combining the fixed-point representation value with the corresponding weight grouping information to obtain the encoded weight set includes: Based on the sub-interval to which the weight value of each fixed-point representation value belongs before mapping, the corresponding weight grouping information is determined, and the weight grouping information is inserted at the end of the corresponding fixed-point representation value to obtain the encoded weight set, wherein the weight grouping information corresponding to the weight values ​​in the same sub-interval is the same.

6. The weight quantization method based on hybrid segmented coding according to claim 1, characterized in that, The step of determining the multiplication coefficients and shift values ​​of the encoded weights based on the identifier bits includes: The multiplication coefficient and shift value of the encoded weight are determined by looking up the weight grouping information corresponding to any encoded weight in a table.

7. The weight quantization method based on hybrid segmented coding according to claim 1, characterized in that, The step of performing a multiplication and addition calculation on the activation value and the weight information in the data bits to obtain the output activation value corresponding to the encoded weight includes: The activation value and the weight information are substituted into the multiply-accumulate calculation model, and the floating-point proportional term in the multiply-accumulate calculation model is converted into a fixed-point form to solve the multiply-accumulate calculation model and obtain the output activation value.

8. A weighted quantization device based on hybrid segmented coding, characterized in that, The device includes: The information acquisition module acquires the floating-point weights and activation values ​​to be processed. The hybrid segmented encoding module performs hybrid segmented encoding on the floating-point weights based on the Laplace distribution of the floating-point weights to obtain a set of encoded weights in the quantized weight encoding format. Each encoded weight in the set of encoded weights includes: data bits and identifier bits. The weight quantization module determines the multiplication coefficient and shift value of the encoded weight based on the identifier bit, and performs multiplication and addition calculations on the activation value and the weight information in the data bits based on the multiplication coefficient and the shift value to obtain the output activation value corresponding to the encoded weight.

9. A server, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images