Command word recognition learning network layer weight quantization, inference methods and devices

Abstract

This invention relates to the field of artificial intelligence technology, and discloses a method and device for quantizing and inferring command word recognition learning network layer weights. The method for quantizing command word recognition learning network layer weights includes: acquiring all weights in the same layer of the network; dividing them into a first category of weights (within the range) and a second category of weights (outside the range) based on their precision range; performing fixed-point quantization on the first category of weights using a low-precision integer format; splitting the second category of weights into multiple sub-weights (all within the low-precision range) and quantizing them, recording their position information and sub-weight combination rules; and storing the quantized data according to a preset structure. This method avoids the precision loss of traditional quantization by splitting out and storing the out-of-range weights completely, significantly reducing storage resource consumption while improving the accuracy of command word recognition inference, and is suitable for low-resource platforms such as smart speakers.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and device for weight quantization and inference of command word recognition learning network layers. Background Technology

[0002] In low-resource embedded platforms such as smart speakers and wearable devices, command word recognition is a core interactive function, and its performance directly determines the user experience. To achieve fast and accurate command word recognition, deep learning networks (such as convolutional neural networks and recurrent neural networks) are typically used, employing trained weight matrices to accurately map speech features.

[0003] However, the weights of deep learning networks are typically stored and computed in 32-bit floating-point format, which poses two major challenges to low-resource platforms: on the one hand, floating-point weights occupy a large amount of storage space, far exceeding the storage limitations of embedded devices; on the other hand, floating-point operations require high computing power, which can lead to increased inference latency and fail to meet the needs of real-time interaction.

[0004] To adapt to low-resource platforms, existing technologies generally employ weight quantization methods to convert floating-point weights into low-precision formats such as 8-bit integers (int8), which can reduce storage usage by 75% and improve computational efficiency. However, the weight distribution of command word recognition networks has special characteristics: some weights (especially those corresponding to edge features) exceed the representation range of int8 (e.g., the floating-point interval [-128, 127]). Existing quantization methods typically truncate or round these out-of-range weights, resulting in the loss of weight information.

[0005] This information loss significantly reduces command word recognition accuracy, especially in scenarios with low signal-to-noise ratios and similar pronunciations (such as "turn on the air conditioner" and "turn on the desk lamp"), where recognition accuracy drops, severely impacting device usability. Therefore, how to fully preserve excess weight information while maintaining low-precision quantization, achieving a balance between storage efficiency and recognition accuracy, has become a pressing technical problem to be solved in the field of command word recognition for low-resource platforms. Summary of the Invention

[0006] The main objective of this invention is to provide a method and device for weight quantization and inference of command word recognition learning network layers, aiming to solve the technical problem that information loss significantly reduces the accuracy of command word recognition in the prior art.

[0007] To achieve the aforementioned objective, the first aspect of this invention proposes a method for quantizing the weights of a command word recognition learning network layer, the method comprising:

[0008] Obtain all values ​​of the command word recognition learning network at the same layer;

[0009] Based on the weight accuracy range of the same layer of command word recognition learning network, all weights are divided into a first type of weights that are within the weight accuracy range and a second type of weights that are outside the weight accuracy range.

[0010] For the first type of weights, a fixed-point quantization method corresponding to a preset low-precision integer format is used for quantization;

[0011] Each of the second type of weights is split into at least two sub-weights according to a preset rule. The preset rule includes that each sub-weight falls within the numerical range of the low-precision integer format, and the sub-weights can be restored to the original second type of weights through a preset combination rule.

[0012] The sub-weights after splitting are quantized using the low-precision integer format, and the position information of each second type of weight in the weight matrix and the corresponding sub-weight combination rules are recorded.

[0013] The quantized first type of weights, the quantized sub-weights, the location information, and the combination rules are stored according to a preset storage structure.

[0014] Further, the step of dividing all weights into a first class of weights within the weight accuracy range and a second class of weights exceeding the weight accuracy range based on the weight accuracy range of the same layer of the command word recognition learning network includes:

[0015] Obtain the actual distribution characteristics of all weights in the same layer of the command word recognition learning network. The distribution characteristics include the numerical concentration range of the weights, the extreme value range, and the proportion of weights that exceed the preset initial weight accuracy range.

[0016] Based on the distribution characteristics and the range of numerical representation in the preset low-precision integer format, the initial weight precision range is dynamically adjusted to obtain the weight precision range.

[0017] Using the weight precision range as the classification criterion, weights falling within the weight precision range are classified as first-class weights, and weights exceeding the weight precision range are classified as second-class weights.

[0018] Further, the step of dynamically adjusting the initial weight precision range based on the distribution characteristics and the numerical representation range of a preset low-precision integer format to obtain the weight precision range includes:

[0019] Based on the numerical representation range of the preset low-precision integer format, the theoretical precision range of the weight quantization under this format is determined;

[0020] Extract the proportion of weights that exceed the accuracy range of the initial weights from the actual distribution characteristics of all weights, and compare this proportion with a preset reasonable proportion range;

[0021] If the percentage is higher than the upper limit of the reasonable percentage range, then the upper and lower limits of the initial weight accuracy range are expanded towards the boundary of the theoretical accuracy range, so that more weights fall into the adjusted weight accuracy range, until the percentage of weights exceeding the initial weight accuracy range drops to within the reasonable percentage range; or,

[0022] If the percentage is lower than the lower limit of the reasonable percentage range, then the upper and lower limit thresholds of the initial weight accuracy range are reduced in the direction of the weight value concentration range. While ensuring that most weights are still within the range, the quantization accuracy is optimized until the percentage of weights exceeding the initial weight accuracy range rises to the reasonable percentage range.

[0023] If it is confirmed that the adjusted weight accuracy range does not exceed the theoretical accuracy range, the adjusted range that meets this condition is determined as the weight accuracy range.

[0024] Furthermore, the preset combination rules include:

[0025] The at least two sub-weights obtained after splitting are divided into high-position sub-weights and low-position sub-weights;

[0026] The high-order sub-weights are processed by a preset bitwise operation logic to obtain the high-order processed value. The preset bitwise operation logic is to shift the high-order sub-weights to the left by a preset number of bits.

[0027] The high-order processed value and the low-order sub-weight value are numerically superimposed to obtain the original second-type weight value after restoration.

[0028] Further, storing the quantized first-class weights, the quantized sub-weights, the position information, and the combination rules according to a preset storage structure includes:

[0029] The preset combination rules, the quantized first-class weights, the quantity information of the quantized sub-weights, the quantized sub-weights, and the position information of each second-class weight in the weight matrix are stored sequentially in a preset order; wherein, the storage format of the position information is adaptively determined according to the row and column dimensions of the weight matrix.

[0030] If the row dimension or column dimension of the weight matrix is ​​less than or equal to a preset dimension threshold, the position information is stored in a first preset storage format. The first preset storage format is to store the row index and column index in a low-byte integer format, and the total number of bytes stored for a single position information does not exceed a preset low-byte upper limit.

[0031] If the row dimension or column dimension of the weight matrix is ​​greater than the preset dimension threshold, the location information is stored in a second preset storage format. The second preset storage format is to store the index that exceeds the threshold in an integer format adapted to the dimension exceeding the threshold, and to store the index that does not exceed the threshold in a low-byte integer format. The total number of bytes stored for a single location information is adapted to the dimension of the weight matrix and meets the storage requirements of the low-resource platform.

[0032] Furthermore, the step of splitting each of the second type of weights into at least two sub-weights according to a preset rule includes:

[0033] Obtain the specific value of a single second-class weight, and determine the upper and lower limits of the numerical representation of the value in a preset low-precision integer format;

[0034] Based on the upper and lower limits of the numerical representation, the second type of weight is divided into high-position sub-weight and low-position sub-weight, wherein the absolute value of the low-position sub-weight does not exceed the numerical representation range of the preset low-precision integer format, and the high-position sub-weight is used to characterize the numerical magnitude of the second type of weight that exceeds the low-position sub-weight.

[0035] Verify whether the high-order and low-order sub-weights after splitting both fall within the numerical range of the preset low-precision integer format. If both are satisfied, the splitting of the second type of weights is completed; if not, the splitting method is readjusted until all the sub-weights obtained from the splitting meet the numerical range requirements.

[0036] A second aspect of this invention proposes an inference computation method for a command word recognition learning network layer, the method being implemented based on stored data obtained from the weight quantization method for the command word recognition learning network layer described in any of the preceding claims, comprising:

[0037] Obtain the activation value data corresponding to the speech of the command word to be processed, as well as the quantized first type weight, quantized sub-weight, second type weight position information and sub-weight combination rules stored according to the preset storage structure;

[0038] Based on the sub-weight combination rules, determine the high-position sub-weights and low-position sub-weights corresponding to the second type of weights, and restore the calculation logic of the second type of weights;

[0039] Perform a conventional matrix multiplication operation on the quantized first type of weights and the activation value data to obtain the first inference calculation result;

[0040] Based on the second type of weight position information, the target activation value corresponding to the second type of weight position is extracted from the activation value data, and sparse calculation is performed in combination with the quantized sub-weights: first, the high-position sub-weights are processed according to the preset bit operation logic and multiplied with the target activation value, then the low-position sub-weights are multiplied with the target activation value, and the two product results are superimposed to obtain the second inference calculation result;

[0041] The first inference calculation result and the second inference calculation result are added together to obtain the final command word recognition inference result.

[0042] Further, the step of performing a conventional matrix multiplication operation on the quantized first type of weights and the activation value data to obtain the first inference calculation result includes:

[0043] Accelerate the matrix multiplication calculation of the first type of weight and activation value data after quantization by calling the adapted hardware API interface or by using assembly instructions;

[0044] During the calculation process, the computational data is kept in a preset low-precision integer format for numerical interaction to avoid additional computing power consumption due to data format conversion.

[0045] After completing the matrix multiplication operation, the result is numerically normalized to ensure that the result value falls within the preset valid range, thus obtaining the first inference calculation result.

[0046] Further, the step of extracting the target activation value corresponding to the second type of weight position from the activation value data based on the second type of weight position information, and performing sparse computation in combination with the quantized sub-weights, includes:

[0047] Read the stored location information of the second type of weights, and determine the row and column indices of the second type of weights in the weight matrix based on this location information;

[0048] The corresponding target activation value is located and extracted from the activation value data to be processed based on the column index, and the position of the target activation value in the calculation result is determined based on the row index.

[0049] The high-order sub-weights and low-order sub-weights in the quantized sub-weights are called. First, the high-order sub-weights are processed according to the preset bit operation logic and then multiplied with the target activation value to obtain the high-order operation result. Then, the low-order sub-weights are multiplied with the target activation value to obtain the low-order operation result.

[0050] The high-order operation result is superimposed with the low-order operation result to obtain a sparse computation sub-result corresponding to a single second-type weight. This sub-result is then added to the value corresponding to its assigned position until the sparse computation of all second-type weights is completed, thus obtaining the second inference computation result.

[0051] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the command word recognition learning network layer weight quantization method and / or the inference calculation method of the command word recognition learning network described in any of the preceding claims.

[0052] Beneficial effects:

[0053] The command word recognition learning network layer weight quantization and inference method and device of the present invention, by splitting the second type of weights exceeding the range of low-precision integer format into multiple quantizable sub-weights, completely preserves the numerical information of the original weights (including magnitude and details), avoiding the precision loss caused by truncation or rounding of over-limit weights in traditional quantization methods, and achieving "no loss of over-limit weights and no information loss in the quantization process", realizing lossless quantization and storage of weights. Since the weight information is completely preserved, the calculation contribution of the original weights can be accurately restored through the sub-weight combination rules during the inference stage, effectively avoiding the model performance degradation caused by weight distortion, especially for command words in easily confused scenarios such as low volume and similar pronunciation, the recognition accuracy is significantly improved. The weights and sub-weights are quantized using a preset low-precision integer format (such as int8), and the data layout is optimized through structured storage. Compared with floating-point format storage, the weight storage amount is reduced, perfectly adapting to embedded devices with low storage resources such as smart speakers. The first type of weights can be directly quantized efficiently using conventional low-precision matrix multiplication. The splitting and restoration of the second type of weights relies only on simple bit operations and addition, with minimal additional computing power consumption. This ensures accuracy while meeting the real-time requirements of low-resource platforms. It can adapt to command word recognition learning network layers with different structures, achieving differentiated quantization based on the weight distribution characteristics of each layer. Deployment is possible without modifying the network topology, reducing the difficulty of engineering applications. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a method for quantizing the weights of a command word recognition learning network layer according to an embodiment of the invention;

[0055] Figure 2 A flowchart illustrating the inference computation method of a command word recognition learning network layer according to an embodiment of the invention;

[0056] Figure 3 This is a schematic block diagram of a computer device according to an embodiment of the invention.

[0057] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0060] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0061] Reference Figure 1 This application provides a method for quantizing the weights of a command word recognition learning network layer, the method comprising:

[0062] S11: Obtain all values ​​of the command word recognition learning network at the same layer.

[0063] Command word recognition learning network: refers to a deep learning model (such as CNN, RNN) used to recognize speech commands (such as "turn on the lights" or "play music"), which consists of a multi-layer network structure, with each layer containing a weight matrix for calculation.

[0064] Weights: Connection parameters between neurons in a network, which determine the weight of input data transmission and directly affect the model's recognition accuracy.

[0065] For example, in Project A, the third convolutional layer of the command word recognition model is selected. All weights of this layer are extracted through the model parameter reading interface, forming a 128×64 weight matrix (a total of 8192 weights). The weight values ​​of this layer are distributed between [-1.2, 0.8], including a large number of regular values ​​concentrated in [-0.5, 0.5] and a small number of extreme values ​​outside this range (such as -1.1, 0.75).

[0066] This step provides the foundational data for subsequent weight classification and quantization, ensuring coverage of all computational parameters at this layer.

[0067] S12: Based on the weight accuracy range of the same layer command word recognition learning network, all weights are divided into a first type of weights that are within the weight accuracy range and a second type of weights that are outside the weight accuracy range.

[0068] Weight precision range: A preset range of values ​​used to distinguish whether weights can be directly quantized, determined by the representation capability of low-precision integer format.

[0069] For example, Project A has a preset weight precision range of [-0.5, 0.5]. Weights falling within this range are classified into Category 1 (which can be directly quantified), and those outside the range are classified into Category 2 (which need to be split and quantified). For instance, out of 8192 weights, 7680 fall within [-0.5, 0.5] (Category 1), and 512 are outside the range (Category 2, such as 0.7, -0.8).

[0070] This step achieves differentiated processing through classification, avoiding the loss of accuracy caused by directly quantifying weights that are outside the range.

[0071] S13: For the first type of weights, quantize them using a fixed-point quantization method corresponding to a preset low-precision integer format.

[0072] Preset low-precision integer format: refers to the integer type (such as int8, with a value range of [-128, 127]) used to store quantized weights, which can save a lot of storage space compared to floating-point format (such as float32).

[0073] Fixed-point quantization: The process of converting floating-point weights into integers through linear mapping. The formula is: quantized value = round(floating-point value / quantization step size + zero offset).

[0074] For example, Project A uses the int8 format with a quantization step size of 0.00390625 (i.e., 1 / 256). The first type of weight (e.g., 0.3) is quantized to round(0.3 / 0.00390625) = 77 (int8 type). A weight of 0.25 is quantized to 64 (0.25 / 0.00390625 = 64), and a weight of -0.4 is quantized to -102 (-0.4 / 0.00390625 ≈ -102.4 → round is -102).

[0075] This step converts floating-point values ​​to low-precision integers, reducing storage by 75% (float32 → int8), making it suitable for the low storage resources of smart speakers and other devices.

[0076] S14: For each of the second type of weights, split it into at least two sub-weights according to a preset rule. The preset rule includes: each sub-weight falls within the numerical range of the low-precision integer format, and the sub-weights can be restored to the original second type of weights through a preset combination rule.

[0077] Preset rule: Ensure that sub-weights can be represented in low-precision integer format and that the original weight splitting logic can be restored.

[0078] For the processing of the second type of weights, they need to be mapped to a high-precision integer field (such as int16) before being split. For example, in Project A, the second type of weight 0.7 is first quantized using the formula quantized_value=round(0.7 / 0.00390625) to obtain the int16 value 179. Since 179 exceeds the representation range of int8 [-128, 127], it needs to be split into two sub-weights according to the preset rules. Similarly, the weight -0.8, after quantization, yields the int16 value -204, which also needs to be split.

[0079] The specific splitting and restoration mechanism is as follows: For the int16 quantized value 179, it can be split into a high-order sub-weight of 1 and a low-order sub-weight of -77. Verification of restoration: (1<<8)+(-77)=256-77=179, completely lossless. Similarly, the int16 value -204 can be split into a high-order sub-weight of -1 and a low-order sub-weight of 52, because (-1<<8)+52=-256+52=-204.

[0080] Furthermore, the splitting of the second type of weights adheres to the principle of "precise splitting and unambiguous restoration," employing an integer splitting mechanism based on bitwise operations and two's complement compatibility. Taking the int8 range as an example: first, the second type of weights are mapped to integers within the int16 range through quantization scaling; then, bitwise operations are used to split them into two sub-weights within the int8 range (lower-order sub-weight and higher-order sub-weight). During restoration, the original value can be completely recovered through the inverse operation. Specifically:

[0081] I. Splitting Mechanism: From int16 to Double int8

[0082] Range validation: First, it checks whether the int16 value to be split is within the valid range (-32768~32767). If it exceeds the range, an exception is thrown to ensure the validity of the input data.

[0083] Bitwise operations extract the high and low unsigned bits:

[0084] Extract the lower 8 unsigned bits (range 0~255) of an int16 using the AND operation num&0xFF.

[0085] By right-shifting by 8 bits and then performing a bitwise AND operation (num>>8)&0xFF, the high 8 unsigned bits of the int16 value (range 0~255) are extracted.

[0086] Converting two's complement to signed int8: Convert the extracted high and low unsigned values ​​to signed int8 (range -128~127). The conversion rules are: if the value is ≥128, subtract 256 to get the corresponding negative value (e.g., the unsigned value 200 is converted to -56); if the value is <128, keep the original value.

[0087] Adjustment for two's complement scenarios: If the lower-order sub-weight is negative (meaning it is a negative number in two's complement form), then the higher-order sub-weight is increased by 1 to ensure the accuracy of the value during subsequent recovery.

[0088] II. Recovery Mechanism: From Double int8 to int16

[0089] Perform the inverse operation on the low-order int8 sub-weights and high-order int8 sub-weights obtained from the splitting: shift the high-order sub-weight to the left by 8 bits (equivalent to multiplying by 256), and then add it to the low-order sub-weight to restore the original int16 value.

[0090] In a specific instance, if a quantized weight corresponds to an int16 value of 200, using the above splitting mechanism, after bit operations and two's complement conversion, it is finally split into a high-bit sub-weight of 1 and a low-bit sub-weight of -56 (because 1×256+(-56)=200, and both 1 and -56 are within the range of int8).

[0091] If the quantized weight corresponds to the int16 value -200 (which is outside the range of int8 and needs to be split), the above splitting mechanism is also used to split it into a high-position sub-weight of -1 and a low-position sub-weight of 56 (because -1×256+56=-200, and both -1 and 56 are within the range of int8).

[0092] During recovery, the higher-order sub-weights are shifted left by 8 bits and then added to the lower-order sub-weights using the recovery mechanism described above (e.g., 1 << 8 + (-56) = 200, -1 << 8 + 56 = -200). This completely restores the original int16 value, ensuring no precision loss during the splitting and recovery process. For weights already within the int8 range (e.g., -100), no splitting is required; they can be stored directly as a single int8 value.

[0093] This step splits the values ​​so that out-of-range weights can be processed by a low-precision format, avoiding the loss of precision caused by direct truncation.

[0094] S15: Quantize each sub-weight after splitting using the low-precision integer format, and record the position information of each second type of weight in the weight matrix and the corresponding sub-weight combination rules.

[0095] Location information: The row and column index of the second type of weight in the original weight matrix (e.g., (3,5) represents the 3rd row and 5th column), used for location during inference.

[0096] Combination rules: The operation logic of restoring sub-weights to their original weights (e.g., "high-position sub-weight + low-position sub-weight").

[0097] For example, in Project A, the sub-weights after splitting (such as 0.5 and 0.1) are quantized into int8 (128 and 26) respectively, and their original positions (10, 25) and combination rules "high bit + low bit" are recorded. For example, the sub-weight of weight 0.6 is quantized into 128 (0.5) and 26 (0.1), the position information is stored as (10, 25), and the combination rule is marked as "addition".

[0098] This step records the location and rules to provide a basis for weight restoration during subsequent reasoning, ensuring calculation accuracy.

[0099] S16: Store the quantized first type of weights, the quantized sub-weights, the location information, and the combination rules according to a preset storage structure.

[0100] Preset storage structure: Specifies the storage order and format of quantized data to optimize reading efficiency.

[0101] For example, Project A stores data in the following order: "Combination rule → First type of weight after quantization → Number of sub-weights → Sub-weights after quantization → Position information". For instance, it stores the "addition" rule first, then 7680 int8 quantized values, and finally 512 sets of position indices. Specifically, in the storage file, the first byte marks the combination rule, the following 30720 bytes (7680 × 4 bytes int8) store the first type of weight, and the following 2048 bytes store the sub-weights and positions.

[0102] This step involves structured storage to reduce data redundancy and improve data retrieval speed during inference.

[0103] In this embodiment, low-precision storage is achieved through "classification quantization + over-range splitting". Taking the command word recognition model of the smart speaker in Project A as an example, for the weights of a certain convolutional layer, they are first divided into a first type of weights that can be directly quantized and a second type of weights that need to be split. The former is directly converted to int8 format, and the latter is split into multiple int8 sub-weights and then quantized. At the same time, the position and restoration rules are recorded, and finally stored in a structured manner.

[0104] The command word recognition learning network layer weight quantization method in this embodiment avoids the truncation or rounding of over-limit weights in traditional quantization methods by splitting the second type of weights that exceed the low-precision range into quantizable sub-weights and storing them completely. This fully preserves all numerical information of the original weights (including magnitude and details), achieving "no loss of over-limit weights and no precision loss during quantization." Because the weight information is completely preserved, the computational contribution of the original weights can be fully restored during the inference stage, avoiding model performance degradation caused by weight distortion. While maintaining accuracy, the weight storage size is significantly reduced compared to floating-point format through low-precision integer quantization and structured storage, and there is no need to store redundant information, making it perfectly compatible with low-storage devices such as smart speakers. Direct quantization of the first type of weights ensures computational efficiency, and the splitting-restoration logic of the second type of weights is simple (only bitwise operations and addition), resulting in a very small proportion of additional computational power consumption during inference, thus resolving the contradiction between "high precision and low resources."

[0105] In one embodiment, step S12, which divides all weights into a first class of weights within the weight accuracy range and a second class of weights outside the weight accuracy range based on the weight accuracy range of the same layer of the command word recognition learning network, includes:

[0106] S121: Obtain the actual distribution characteristics of all weights in the same layer of the command word recognition learning network. The distribution characteristics include the numerical concentration range of the weights, the extreme value range, and the proportion of weights that exceed the preset initial weight accuracy range.

[0107] Actual distribution characteristics: The statistical properties of the weights in terms of numerical values, including the concentration interval (the range in which most weights are clustered), the extreme value range (maximum / minimum value), and the proportion of weights exceeding the initial range (the proportion of weights that exceed the preset initial range).

[0108] For example, in Project A, statistical analysis of the 8192 weights in the third convolutional layer revealed the following: the concentration range is [-0.4, 0.4], the extreme value range is [-1.2, 0.8], the initial weight precision range is preset to [-0.5, 0.5], and the percentage of weights exceeding this range is 6.25% (512 / 8192). Specifically, histogram analysis showed that 80% of the weights are concentrated in [-0.3, 0.3], with only 512 weights exceeding [-0.5, 0.5].

[0109] This step provides data for subsequent precision range adjustments, avoiding classification bias caused by unreasonable initial range settings.

[0110] S122: Based on the distribution characteristics and the numerical representation range of the preset low-precision integer format, the initial weight precision range is dynamically adjusted to obtain the weight precision range.

[0111] Initial weight precision range: The preset interval before optimization (e.g., [-0.5, 0.5]), which needs to be adjusted according to the actual distribution.

[0112] The preset range of low-precision integer format values ​​is: for example, the range of quantized values ​​corresponding to int8 ([-128,127]), and the corresponding floating-point range is determined by the quantization step size (e.g., [-0.5,0.5]).

[0113] For example, in Project A, the maximum representation range of int8 corresponds to floating-point [-0.5, 0.5] (quantization step size 0.00390625). However, statistics showed that the proportion of weights exceeding the initial range was 6.25%, higher than the preset reasonable proportion (5%). Therefore, the initial range was expanded to [-1, 1]. After the adjustment, the number of weights exceeding the range decreased to 256 (accounting for 3.125%), falling into the reasonable range.

[0114] This step dynamically adjusts the weights so that more weights can be directly quantified, reducing the number of second-class weights that need to be split and lowering computational complexity.

[0115] S123: Using the weight precision range as the classification basis, weights that fall within the weight precision range are classified as first-class weights, and weights that exceed the weight precision range are classified as second-class weights.

[0116] For example, Project A uses the adjusted range [-1, 1] as the standard, dividing 7936 weights into the first category (direct quantification) and 256 into the second category (requiring splitting). For instance, the original weight of 0.65 was outside the initial range, but after adjustment it falls into [-1, 1] and is classified into the first category, avoiding splitting.

[0117] This step categorizes the data more closely to the actual weight distribution, reduces unnecessary splitting operations, and improves quantification efficiency.

[0118] In this embodiment, the core is to optimize the classification results by dynamically adjusting the precision range of the weights. For example, in Project A, the distribution characteristics of the weights are first statistically analyzed (such as concentrated intervals and out-of-range proportions), and then the initial range is adjusted based on the representation capability of the int8 format. Finally, the first and second categories of weights are divided based on the optimized range. This method solves the problem of unreasonable classification caused by a fixed precision range, making the quantization process more compatible with the actual distribution of weights. Specifically, dynamic adjustment makes the precision range of weights match the actual distribution. For example, in Project A, the number of weights that need to be split is reduced by 50% (from 512 to 256); the splitting and restoration operations of the second category of weights are reduced, and the computation time in the quantization stage is significantly shortened; it adapts to different network layers and can independently adjust the range according to the weight distribution characteristics of each layer, improving the versatility of the method.

[0119] In one embodiment, step S122, which involves dynamically adjusting the initial weight precision range based on the distribution characteristics and the numerical representation range of a preset low-precision integer format to obtain the weight precision range, includes:

[0120] S1221: Based on the numerical representation range of the preset low-precision integer format, determine the theoretical precision range of the weight quantization under this format.

[0121] Theoretical precision range: The range of floating-point weights that can be accurately represented by a low-precision integer format through quantization mapping. It is determined by the numerical range of the integer format and the quantization step size. Quantization of weights outside this range will result in out-of-bounds errors.

[0122] For example, if Project A uses the int8 format (numerical range [-128, 127]) and a quantization step size of 0.00390625, then the direct quantization reference interval is [-128 × 0.00390625, 127 × 0.00390625], which is [-0.5, 0.49609375] (approximately [-0.5, 0.5]). This interval defines the range of floating-point weights that can be directly represented in int8 format without splitting.

[0123] This step clarifies the adjustment boundary of the weight precision range to avoid the adjusted range exceeding the representation capability of the low-precision format, which could lead to quantization exceeding the limit.

[0124] S1222: Extract the weight percentage that exceeds the accuracy range of the initial weight from the actual distribution characteristics of all weights, and compare the percentage with a preset reasonable percentage range.

[0125] Reasonable proportion range: The preset weight proportion range (e.g., 1%-3%) that exceeds the weight precision range. The proportion within this range can balance the quantization precision and computational complexity.

[0126] For example, Project A has an initial weight precision range of [-0.5, 0.5]. After counting 8192 weights, 655 weights exceed this range, accounting for 8% (655 / 8192). The preset reasonable percentage range is 1%-3%, and 8% is higher than the upper limit of 3%. Specifically, a Python script is used to count the weight distribution, iterating through all 8192 weights, counting the number of weights exceeding [-0.5, 0.5], calculating the percentage as 8%, and comparing it with the reasonable range (1%-3%) to determine that the initial range needs to be expanded.

[0127] S1223: If the percentage is higher than the upper limit of the reasonable percentage range, then the upper and lower limit thresholds of the initial weight accuracy range are expanded in the direction of the boundary of the theoretical accuracy range, so that more weights fall into the adjusted weight accuracy range, until the percentage of weights exceeding the initial weight accuracy range is reduced to the reasonable percentage range.

[0128] For example, in Project A, the percentage of out-of-range weights is 8%, which is higher than 3%. Therefore, the initial range needs to be expanded towards the boundary of the theoretical accuracy range ([-0.5, 0.5]). The upper limit is adjusted to 1 and the lower limit is adjusted to -1. The percentage of out-of-range weights is reduced to 2.8%, which falls into the reasonable range (1%-3%).

[0129] This step reduces the number of out-of-bounds weights by gradually expanding the scope, thereby reducing the workload of splitting and calculating, while ensuring the effectiveness of quantization without exceeding the theoretical accuracy range.

[0130] S1224: If the percentage is lower than the lower limit of the reasonable percentage range, then the upper and lower limit thresholds of the initial weight accuracy range are reduced in the direction of the weight value concentration range. While ensuring that most weights are still within the range, the quantization accuracy is optimized until the percentage of weights exceeding the initial weight accuracy range rises to the reasonable percentage range.

[0131] Suppose that the weights of another network layer in Project A are concentrated in the range [-0.25, 0.25]. The percentage of weights outside the initial range [-0.5, 0.5] is 0.8% (less than 1%). By narrowing the range to [-0.25, 0.25], the percentage of weights outside the range increases to 1.5%, falling within a reasonable range. Specifically, after narrowing, the original weight 0.45 changes from "within the range" to "outside the range," and the number of weights outside the range increases from 65 to 123, accounting for 1.5%. This ensures that most weights remain within the range while improving quantization accuracy.

[0132] This step improves the utilization rate of the quantization step size and optimizes the quantization accuracy of the first type of weights by narrowing the range, provided that most weights do not need to be split.

[0133] S1225: Confirm that the adjusted weight accuracy range does not exceed the theoretical accuracy range, and determine the adjusted range that meets this condition as the weight accuracy range.

[0134] The goal of dynamic adjustment is to find an optimal weight precision range that allows the vast majority of weights to be efficiently and directly quantized, while ensuring that weights exceeding the range are fully preserved through a splitting mechanism. For example, in Project A, if the initial adjusted weight precision range is [-1, 1], causing the proportion of out-of-range weights to drop to a reasonable level (e.g., 3%), then this range is the reasonable weight precision range. Although 1 exceeds the direct quantization benchmark range [-0.5, 0.5], the splitting mechanism of this invention can handle these out-of-range weights without loss, thus eliminating the need for a callback. The final determined weight precision range is [-1, 1].

[0135] This step avoids adjusting the range beyond the representation capabilities of the low-precision format, ensuring that weights within all ranges can be directly quantized, while weights exceeding the range can be processed by splitting.

[0136] In this embodiment, the core principle is to optimize the weight precision range based on the theoretical capabilities of weight distribution characteristics and low-precision format through a logical process of "proportion judgment - direction adjustment - boundary verification." For example, in Project A, for the weights of a certain layer in the smart speaker command word recognition model, the theoretical precision range of the int8 format is first determined. Then, by comparing the proportion of weights exceeding the range with the reasonable range, the initial range is dynamically expanded or reduced, ultimately obtaining a weight precision range that both fits the weight distribution and does not exceed the capabilities of the low-precision format. This method solves the problem of unreasonable proportions caused by a fixed range, making weight classification more scientific. It should be noted that if the proportion is within the reasonable proportion range, there is no need to adjust the initial weight precision range.

[0137] The command word recognition learning network layer weight quantization method in this embodiment dynamically adapts the weight distribution, stabilizing the proportion of out-of-range weights at, for example, 1%-3%, thus balancing quantization accuracy and computational complexity. It avoids the risk of quantization exceeding limits by verifying the theoretical accuracy range to ensure that the adjusted range can be supported by low-precision formats without quantization distortion. It also improves the versatility of the method, adapting to the differences in weight distribution of different network layers. Different convolutional layers can all obtain the optimal accuracy range through this method.

[0138] In one embodiment, the aforementioned preset combination rule includes:

[0139] 1401: Divide the at least two sub-weights obtained after splitting into high-position sub-weights and low-position sub-weights.

[0140] High-order sub-weights refer to the sub-weights used to represent the magnitude of numerical values ​​after the second type of weights are decomposed. They can be amplified to the original magnitude through bitwise operations.

[0141] Low-position sub-weights refer to the sub-weights used to represent the remaining details after the second type of weights are split, and they directly participate in the superposition calculation.

[0142] For example, in Project A, the second type of weight 0.7 (outside the weight precision range [-0.5, 0.5]) corresponds to an int16 value of 179 after quantization scaling. This is broken down into a high-order sub-weight of 1 (corresponding to an int8 quantized value of 1) and a low-order sub-weight of -77 (corresponding to an int8 quantized value of -77). The high-order sub-weight of 1 represents the magnitude, while the low-order sub-weight of -77 represents the detail. In other embodiments, for example, a weight of -0.8 corresponds to an int16 value of -204, which is broken down into a high-order sub-weight of -1 (an int8 quantized value of -1) and a low-order sub-weight of 52 (an int8 quantized value of 52), both conforming to the int8 range requirements.

[0143] This step, by clearly defining the functional positioning of sub-weights, provides a logical basis for subsequent bitwise operations and superposition, ensuring the accuracy of the restoration.

[0144] 1402: The high-order sub-weights are processed by a preset bit operation logic to obtain the high-order processed value. The preset bit operation logic is to shift the high-order sub-weights to the left by a preset number of bits.

[0145] Bitwise operation logic refers to binary-based arithmetic. Left shift operation means moving the binary bits of a number to the higher bits by a specified number of bits, which is equivalent to multiplying the number by a specified power of 2 (e.g., left shifting by 8 bits is equivalent to multiplying by 256).

[0146] The preset bit width is determined based on the bit width of the low-precision integer format (e.g., 8 bits for int8). The purpose of the preset bit width is to restore the magnitude represented by the high-order sub-weights in the integer field.

[0147] For example, in Project A, the second type of weight 0.7 corresponds to an int16 quantized value of 179. This value is split into a high-order sub-weight of 1 (int8) and a low-order sub-weight of -77 (int8). During inference, to restore the original quantized value, the high-order sub-weight of 1 needs to be shifted left by 8 bits, i.e., the 1<<8 operation is performed, resulting in 256 (int16). This 256 is added to the low-order sub-weight of -77 to accurately restore the original int16 quantized value of 179.

[0148] This step replaces floating-point multiplication with integer left shift operations, improving computational efficiency and avoiding the precision loss of floating-point operations, thus adapting to the integer arithmetic units of low-resource platforms.

[0149] S1403: Perform numerical superposition operation on the high-order processed value and the low-order sub-weight value to obtain the operation result, which is the restored original second type weight value.

[0150] For example, in Project A, the logic for restoring the original second-type weights is: Original weight = (high-order processed value + low-order sub-weight quantization value) × quantization step size, where:

[0151] The high-order processed value is the quantized value of the high-order sub-weights after left shift (e.g., 32768).

[0152] The quantized value of the low-order sub-weight is an int8 integer (e.g., 51), which corresponds to a floating-point value of 51 × 0.00390625 ≈ 0.2;

[0153] The sum of the two is obtained by adding them together (32768+51=32819), and then multiplying it by the quantization step size (0.00390625) to restore the original weights.

[0154] This step uses a combination of logic—"integer left shift + superposition + single-step quantization step size multiplication"—to restore the original weights without complex floating-point operations. The restoration error is small, and the computational efficiency is improved compared to traditional floating-point restoration methods, making it perfectly suited to the computing power limitations of low-resource platforms.

[0155] In this embodiment, the core of the preset combination rule for the second type of weight sub-weights is to achieve accurate restoration of weights beyond the range through the logic of "high and low bit division - left shift amplification - superposition restoration". For example, in Project A, for the second type of weights in the smart speaker command word recognition model, the split sub-weights are first divided into high bits (magnitude) and low bits (details). Then, an 8-bit left shift operation is performed on the high-bit sub-weights to amplify their magnitude. Finally, they are superimposed with the low-bit sub-weights, and the original weights are restored by combining the quantization step size. This rule avoids complex restoration algorithms and is implemented only through basic bit operations and addition, adapting to the computing power requirements of low-resource platforms.

[0156] In one embodiment, step S16, which stores the quantized first type of weights, the quantized sub-weights, the location information, and the combination rules according to a preset storage structure, includes:

[0157] S161: Store the preset combination rule, the quantized first type of weights, the quantity information of the quantized sub-weights, the quantized sub-weights, and the position information of each second type of weight in the weight matrix in a preset order, wherein the storage format of the position information is adaptively determined according to the row and column dimensions of the weight matrix:

[0158] If the row dimension or column dimension of the weight matrix is ​​less than or equal to a preset dimension threshold, the position information is stored in a first preset storage format. The first preset storage format is to store the row index and column index in a low-byte integer format, and the total number of bytes stored for a single position information does not exceed a preset low-byte upper limit.

[0159] If the row dimension or column dimension of the weight matrix is ​​greater than the preset dimension threshold, the location information is stored in a second preset storage format. The second preset storage format is to store the index that exceeds the threshold in an integer format adapted to the dimension exceeding the threshold, and to store the index that does not exceed the threshold in a low-byte integer format. The total number of bytes stored for a single location information is adapted to the dimension of the weight matrix and meets the storage requirements of the low-resource platform.

[0160] The preset storage order is a data storage order designed to optimize read efficiency, ensuring that the required data can be quickly retrieved in sequence during inference without the need to search across regions.

[0161] The preset dimension threshold is a critical value used to select the storage format for location information. When the value is less than or equal to this value, the low-byte format is used for storage.

[0162] In this embodiment, the step of storing the quantized first-class weights, the quantized sub-weights, the position information, and the combination rules according to a preset storage structure is essentially about optimizing storage on a low-resource platform through "sequential storage + adaptive position format." The specific process is as follows:

[0163] Data is stored in a preset order: "Preset combination rules → quantized first-class weights → number of sub-weights → quantized sub-weights → location information". Among these:

[0164] The preset combination rules are marked with 1 byte (e.g., "01" means "left shift 8 bits + superposition", "02" means "left shift 16 bits + superposition"), to ensure that the logic can be quickly parsed and restored during inference;

[0165] After quantization, the first type of weights are stored consecutively in the row and column order of the weight matrix to avoid I / O latency caused by cross-region lookup;

[0166] The number of sub-weights is stored as a 2-byte unsigned integer, which clarifies the length of the subsequent sub-weight array and facilitates boundary judgment during data parsing.

[0167] The storage format of the aforementioned location information is adaptively determined based on the row and column dimensions of the weight matrix. The core principle is to select either a "low-byte integer format" or an "integer format adapted to dimensions exceeding the threshold" based on the matrix dimensions, ensuring that the number of bytes stored matches the storage capacity of low-resource platforms. Specifically, this is divided into the following two scenarios:

[0168] Scenario 1: First preset storage format when the dimension of the weight matrix is ​​less than or equal to a preset dimension threshold:

[0169] When the row or column dimension of the weight matrix is ​​less than or equal to a preset dimension threshold (e.g., 255, which can be dynamically adjusted according to the storage chip capacity of a low-resource platform; for example, it can be set to 127 for a platform with 8KB of storage capacity and 255 for a platform with 16KB of storage capacity), the location information is stored using a first preset storage format. The first preset storage format stores the row and column indices in low-byte integer format, and the total number of bytes stored for a single location information does not exceed a preset low-byte upper limit (e.g., 2 bytes or 3 bytes, specifically determined by the platform's storage bandwidth). This embodiment uses two typical implementation methods as examples:

[0170] Example 1: Preset dimension threshold = 255, preset low byte limit = 2 bytes.

[0171] At this point, the maximum value of both the row and column indices does not exceed 255. A 1-byte unsigned integer (uint8) is used as the low-byte integer format to store the row and column indices respectively. A single location information occupies 2 bytes (1 byte for the row + 1 byte for the column). For example, if the second type of weight is located in the 10th row and 25th column of a 128×64 matrix, the row index 10 is stored as 0x0A in uint8 format, the column index 25 is stored as 0x19 in uint8 format, and the entire location information is stored as 0x0A19 (2 bytes), saving 50% of storage compared to the fixed 4-byte format.

[0172] Example 2: Preset dimension threshold = 127, preset low byte limit = 2 bytes.

[0173] At this point, the maximum value of the row index and column index does not exceed 127. A 1-byte signed integer (int8) can be used as the low-byte integer format (the value range of int8 [-128, 127] can cover the index requirements). The row index and column index are stored separately, with each position information occupying 2 bytes. For example, the position information of row 50, column 80 is stored as 0x3250 (in int8 format, 50 corresponds to 0x32, and 80 corresponds to 0x50), achieving the same storage efficiency as the uint8 format, and adapting to the hardware characteristics of some platforms that only support signed integer operations.

[0174] Scenario 2: Second preset storage format when the dimension of the weight matrix is ​​greater than the preset dimension threshold.

[0175] When the row or column dimension of the weight matrix is ​​greater than a preset dimension threshold (e.g., 255), the location information is stored using a second preset storage format. This second preset storage format stores indices exceeding the threshold in an integer format adapted to the threshold dimension, and stores indices not exceeding the threshold in a low-byte integer format. The total number of bytes stored for a single location information is adapted to the dimension of the weight matrix. This embodiment uses two typical dimension scenarios as examples:

[0176] Example 1: Single row dimension exceeds the threshold (e.g., matrix dimension = 300×200, row dimension 300>255, column dimension 200≤255).

[0177] At this point, the maximum row index value of 300 exceeds the threshold of 255. Therefore, a 2-byte unsigned integer (uint16) is used to store the row index in an integer format adapted to the threshold dimension. The maximum column index value is 200 ≤ 255, so a 1-byte uint8 is used to store the column index in an integer format with a low byte count. A single location information occupies 3 bytes (2 bytes for the row + 1 byte for the column). For example, in the location information of row 280, column 150, the row index 280 is stored as 0x0118 using uint16, and the column index 150 is stored as 0x96 using uint8, for a total storage of 0x011896 (3 bytes). This saves 25% of storage compared to using the full 4-byte uint16 format.

[0178] Example 2: Both row and column dimensions exceed the threshold (e.g., matrix dimension = 500×400, both row and column dimensions > 255).

[0179] At this point, the maximum values ​​of both row and column indices exceed the threshold. Therefore, they are stored using a 2-byte uint16 format to adapt to the over-threshold dimension. A single location information occupies 4 bytes (2 bytes for the row + 2 bytes for the column). For example, in the location information of row 350 and column 320, the row index 350 is stored as 0x015E, the column index 320 is stored as 0x0140, and the total storage is 0x015E0140 (4 bytes). Although the number of bytes increases, adapting to the over-threshold dimension ensures no index overflow, while maintaining a storage efficiency advantage compared to floating-point format (4 bytes / index).

[0180] In both scenarios described above, by adjusting the adaptive storage format, the number of bytes stored for a single location can dynamically change within the range of 2-4 bytes, adapting to the storage needs of different low-resource platforms.

[0181] For smart speakers (such as those equipped with STM32F103 chips with 64KB of Flash storage), using a 2-3 byte location information storage format can reduce the storage size of a certain convolutional layer's weights from 32KB in floating-point format to 8KB, meeting the storage limit. For smart bracelets (such as those equipped with nRF52832 chips with 32KB of Flash storage), by setting a low byte limit of 2 bytes, the storage size can be further compressed to ensure that the model can be deployed completely.

[0182] This embodiment achieves dual adaptation of location information storage format and weight matrix dimension, as well as low-resource platform storage capacity, by dynamically selecting between low-byte-count integer format and integer format adapted to exceed threshold dimensions. This avoids storage redundancy caused by fixed formats and ensures the accuracy of data parsing.

[0183] In one embodiment, step S14, which involves splitting each of the second type of weights into at least two sub-weights according to a preset rule, includes:

[0184] S141: Obtain the specific value of a single second-class weight, and determine the upper and lower limits of the value's representation in a preset low-precision integer format.

[0185] The numerical representation of the upper / lower limit is the maximum / minimum floating-point weight that can be represented by a preset low-precision integer format through quantization mapping, i.e. the boundary of the theoretical precision range (e.g., int8 corresponds to [-0.5, 0.5]).

[0186] For example, in Project A, the second type of weight 0.7 is extracted through the model parameter reading interface. Combined with the quantization step size of 0.00390625 in int8 format, its numerical representation is determined to be an upper limit of 0.5 and a lower limit of -0.5.

[0187] This step clarifies the boundaries of sub-weights, providing a basis for splitting and ensuring that sub-weights can be quantized in a low-precision format.

[0188] S142: Based on the upper and lower limits of the numerical representation, the second type of weight is split into high-position sub-weights and low-position sub-weights, wherein the absolute value of the low-position sub-weight does not exceed the numerical representation range of the preset low-precision integer format, and the high-position sub-weight is used to characterize the numerical magnitude of the second type of weight that exceeds the low-position sub-weight.

[0189] For example, in Project A, the second type of weight is 0.9, which exceeds the range that int8 can represent. After being quantized by the q8 formula round(0.9×256), it yields an integer value of 230 (>127), which needs to be split into two int8 sub-weights: a high-order sub-weight of 1 and a low-order sub-weight of -26.

[0190] This step employs a splitting logic of "determining details in the lower digits and quantifying the order in the higher digits" to ensure that the sub-weights conform to the low-precision format requirements, while also providing a clear logic for subsequent restoration. Furthermore, splitting in this unified manner ensures consistent parsing, avoiding a different splitting method for each value, which would be inconvenient in practical operation.

[0191] S143: Verify whether the high-order sub-weights and low-order sub-weights after splitting both fall within the numerical range of the preset low-precision integer format. If both are satisfied, the splitting of the second type of weights is completed; if not, the splitting method is readjusted until all the sub-weights obtained by splitting meet the numerical range requirements.

[0192] In this invention, all splitting operations are strictly based on the quantized integer values ​​and follow a 2... 8The displacement decomposition rule uses 256 as the basic unit. For example, a second-class weight, after quantization, yields an int16 value of 256 (approximately 1.0 in floating-point). According to the rule, it is split into a high-order sub-weight of 1 and a low-order sub-weight of 0. Verification shows that 1 ∈ [-128, 127] and 0 ∈ [-128, 127], meeting the storage requirements, thus the split is complete. Another example is the quantized value 230 (approximately 0.9 in floating-point), which is split into a high-order 1 and a low-order -26, both within the int8 range, also a valid split. It should be noted that this scheme does not support "making up" valid integers by adjusting the size of floating-point sub-items. All sub-weights must be generated from the quantized value using a deterministic algorithm (such as high 8-bit / low 8-bit extraction or equivalent bitwise operations) to ensure that the model can be uniquely and efficiently parsed and restored during deployment.

[0193] This step, through rigorous range verification, ensures the storability and computational consistency of sub-weights, fundamentally avoiding quantization distortion caused by illegal splitting, and providing reliable support for high-precision command word recognition reasoning.

[0194] This embodiment refines the "second-type weight decomposition" step, with the core being "determining boundaries - high and low byte decomposition - verification and adjustment." For example, in Project A, for the second-type weights of the smart speaker command word recognition model, the upper and lower limits of the int8 format representation are first defined. Then, the weights are decomposed into "magnitude-based high-byte sub-weights" and "detail-based low-byte sub-weights." Finally, the sub-weights are verified to ensure they conform to the range and are adjusted accordingly. This decomposition method ensures that the sub-weights can be quantized using low-precision formats, while providing clear logic for subsequent combination and reconstruction, solving the problem that out-of-range weights cannot be directly quantized.

[0195] Reference Figure 2 This application also provides an inference computation method for a command word recognition learning network layer. This method is implemented based on stored data obtained from the weight quantization method for the command word recognition learning network layer described in any of the above embodiments, and includes:

[0196] S21: Obtain the activation value data corresponding to the voice of the command word to be processed, as well as the quantized first type weight, quantized sub-weight, second type weight position information and sub-weight combination rules stored according to the preset storage structure.

[0197] Activation value data refers to the feature data of the command word speech after preprocessing (such as Fourier transform and Mel filtering) and input into the network layer. It is the input parameter for inference calculation.

[0198] Quantized storage data refers to the quantized first-class weights, sub-weights, position information, and combination rules stored through the command word recognition learning network layer weight quantization method in any of the above embodiments.

[0199] For example, in Project A, when a user says the command "turn on the lights," the voice signal is preprocessed to obtain 128-dimensional activation value data (a weight matrix of 128×64, with activation value data as a 64-dimensional vector). Simultaneously, the quantized first-class weights (7936 int8 values), sub-weights (512 int8 values), position information (256 groups of uint8 values), and combination rules (left shift by 8 bits + superposition) are read from the storage medium. In a specific instance, the activation value data is [0.1, 0.3, -0.2, ..., 0.4] (64 dimensions). After the quantized storage data is read, it is parsed sequentially into combination rule 01, the first-class weight array, the number of sub-weights (256), the sub-weight array, and the position information array.

[0200] This step clarifies the input data for inference calculations, ensuring the accuracy of the data source and providing a foundation for subsequent calculations.

[0201] S22: Based on the sub-weight combination rules, determine the high-position sub-weights and low-position sub-weights corresponding to the second type of weights, and restore the calculation logic of the second type of weights.

[0202] For example, in Project A, the combination rule is "left shift by 8 bits + superposition". Therefore, the sub-weight array is divided into high-order sub-weights (the first 256) and low-order sub-weights (the last 256) in sequence. The calculation logic is: the calculated value corresponding to the second type of weight = (high-order sub-weight << 8 + low-order sub-weight) × activation value. In a specific instance, the first 256 sub-weights in the sub-weight array are [128, -128, ..., 100] (high-order), and the last 256 are [51, -77, ..., 39] (low-order). The calculation logic is determined as "left shift the high-order weight by 8 bits, superposition it with the low-order weight, and then multiply it by the activation value".

[0203] This step uses combination rules to quickly locate the sub-weight type and calculation logic, avoiding confusion in the calculation logic and ensuring accurate sparse calculation.

[0204] S23: Perform a conventional matrix multiplication operation on the quantized first type of weights and the activation value data to obtain the first inference calculation result.

[0205] Regular matrix multiplication refers to the standard multiplication operation between the weight matrix and the activation value vector. The formula is: First inference result[i] = Σ(First type weight[i][j] × activation value[j]) (j from 1 to the column dimension).

[0206] For example, in Project A, the first type of weights is a 128×64 matrix, and the activation values ​​are 64-dimensional vectors. Regular matrix multiplication calculates 128 result values, each being the sum of the products of the corresponding row's 64 weights and activation values. The calculation maintains the int8 format to avoid format conversion. In a specific instance, the data in the third row of the first type of weights is [64, -102, ..., 26] (int8), and the j-th element of the activation value is 0.3 (77 after quantization). Therefore, the sum of the products in this row is 64×77 + (-102)×51 + ... + 26×39, and the result is stored in int32 format.

[0207] This step involves conventional matrix multiplication, covering most weight calculations, maintaining low-precision format operations, reducing computational power consumption, and since this step accounts for a large proportion of the computation time, the efficiency improvement is significant.

[0208] S24: Based on the second type of weight position information, extract the target activation value corresponding to the second type of weight position from the activation value data, and perform sparse calculation in combination with the quantized sub-weights: first, process the high-position sub-weights according to the preset bit operation logic and multiply them with the target activation value, then multiply the low-position sub-weights with the target activation value, and add the two product results to obtain the second inference calculation result.

[0209] Sparse computation refers to computing only the activation values ​​corresponding to the positions of the second type of weights, without having to traverse all activation values, thus reducing the amount of computation.

[0210] For example, in Project A, the position information of a second type of weight is read, and its corresponding high-order sub-weight 1 and low-order sub-weight -77 are obtained. First, the high-order sub-weight 1 is shifted left by 8 bits to obtain 256, and then added to the low-order sub-weight -77 to restore the original quantized value 179. Next, the target activation value (assuming its quantized value is Q) corresponding to this weight position is extracted from the activation value data, and the multiplication operation 179*Q is performed to obtain the sparse computation sub-result at that position.

[0211] In this embodiment, because sparse computation only processes 1%-3% of the weights, the amount of computation is greatly reduced, significantly reducing the computing power burden on low-resource platforms, while ensuring the computational accuracy of weights outside the range.

[0212] S25: The first reasoning calculation result and the second reasoning calculation result are added together to obtain the final command word recognition reasoning result.

[0213] For example, in Project A, the first inference calculation result (in int32 format) and the second inference calculation result (in int32 format) are summed line by line to obtain 128 final result values. These are then post-processed (e.g., using softmax) to obtain the command word recognition result. In a specific instance, the third line of the first inference result is 15625, and the third line of the second inference result is 2527063. The sum is 2542688, which, after post-processing, is determined to be the command word "turn on the lights".

[0214] This step ensures that the inference accuracy is close to that of floating-point calculation by accumulating and fusing the calculation results of the two types of weights, and the accuracy of command word recognition is significantly improved compared with traditional int8 quantization.

[0215] This embodiment discloses a command word recognition inference calculation method based on quantized data, the core of which is a fusion scheme of "conventional calculation + sparse calculation". For example, in Project A, after the smart speaker obtains the activation value data of the command word speech, it first restores the calculation logic based on the quantized stored data, then performs conventional matrix multiplication on the first type of weights (covering most calculations), performs sparse calculation on the second type of weights (accurately handling out-of-range weights), and finally accumulates the results to obtain the final recognition result. This method, based on the quantized data of the command word recognition learning network layer weight quantization method described in any of the above embodiments, ensures both the computing power and storage adaptability of low-resource platforms and solves the problem of insufficient accuracy in traditional low-precision quantization.

[0216] The inference computation method of the command word recognition learning network layer in this embodiment improves the command word recognition accuracy from 70% with traditional int8 quantization to 94%, approaching the precision of floating-point calculation; conventional calculations maintain the int8 format, sparse calculations reduce workload by 97%, shorten inference time, and are compatible with the computing power of MCUs in smart speakers, etc.; inference based on quantized low-precision data eliminates the need to store floating-point weights, reducing storage usage and meeting the requirements of low-resource platforms; it has strong real-time performance, with no complex format conversions or redundant calculations throughout the inference process, and the single command word recognition response time meets user interaction needs.

[0217] In one embodiment, step S23, which involves performing a conventional matrix multiplication operation on the quantized first type of weights and the activation value data to obtain the first inference calculation result, includes:

[0218] S231: Accelerate the matrix multiplication of the quantized first-class weight and activation value data by calling the adapted hardware API interface or through assembly instructions.

[0219] The adapted hardware API interface refers to the matrix multiplication acceleration interface provided by low-resource platforms (such as smart speaker MCUs), which is optimized for formats such as int8 and has higher computational efficiency than software implementations.

[0220] Assembly instructions are machine instructions that directly manipulate hardware. They can be customized to implement matrix multiplication and reduce software overhead.

[0221] For example, in Project A, the smart speaker uses an STM32H7 MCU and calls the arm_mat_mult_q7 API interface (compatible with int8 matrix multiplication) in its built-in CMSIS-DSP library to accelerate the calculation of the first type of weights and activation values. If there is no compatible API, the matrix multiplication function is written using ARM assembly instructions to directly manipulate registers to perform the operation. In a specific instance, when calling the arm_mat_mult_q7 interface, the inputs are the first type of weight matrix (q7_t type, corresponding to int8) and the activation value vector (q7_t type), and the output is the first inference result (q31_t type, corresponding to int32). The interface is optimized internally through hardware pipeline, improving the computational efficiency by 3 times.

[0222] Compared to pure software implementation, the hardware API or assembly instructions in this step improve the matrix multiplication calculation speed by 2-3 times, significantly reduce the conventional calculation time, and improve real-time performance.

[0223] S232: During the calculation process, the computational data is kept in a preset low-precision integer format for numerical interaction to avoid additional computing power consumption due to data format conversion.

[0224] For example, in Project A, the first type of weights and activation values ​​are input into the calculation interface in int8 format. During the calculation, intermediate results are temporarily stored in int16 or int32 format (to avoid overflow), but are not converted to floating-point format, and integer operations are maintained throughout. In a specific instance, the int8 weight 64 (corresponding to 0.25) is multiplied by the int8 activation value 77 (corresponding to 0.3), and the intermediate result is 64 × 77 = 4928 (int16). When accumulating, it is converted to int32, and no floating-point format is involved throughout the process.

[0225] This step avoids the computational cost of format conversion (floating-point operations take 5-10 times longer than integer operations), reduces inference time by 30%, and avoids precision loss during the conversion process.

[0226] S231: After completing the matrix multiplication operation, the result is numerically normalized to ensure that the result value falls within the preset valid range, and the first inference calculation result is obtained.

[0227] Numerical normalization refers to limiting the accumulated result (int32) of matrix multiplication to a preset valid range (such as [-2^30, 2^30]) to avoid result distortion caused by overflow.

[0228] For example, in Project A, the preset valid range is [-1073741824, 1073741823] (2^30). The accumulated result is judged. If it exceeds the upper limit, it is truncated to 1073741823; if it is below the lower limit, it is truncated to -1073741824. In a specific instance, the accumulated result of one row is 1200000000 (exceeding the upper limit), which is rounded to 1073741823; the result of another row is -1500000000 (below the lower limit), which is rounded to -1073741824.

[0229] This step avoids recognition errors caused by result overflow. After normalization, the result distortion rate is less than 0.1%, ensuring inference accuracy.

[0230] This embodiment refines the steps of "conventional matrix multiplication," with the core being "accelerated computation + format preservation + result normalization." For example, in Project A, for the STM32H7 MCU of a smart speaker, hardware-adaptive APIs or assembly instructions are used to accelerate matrix multiplication in int8 format. During the operation, low-precision integer format is maintained to avoid format conversion overhead, and finally, the result is normalized to prevent overflow. This method further optimizes the efficiency and stability of conventional computation, ensuring rapid calculation of most weights on low-resource platforms.

[0231] The inference calculation method of the command word recognition learning network layer in this embodiment has a faster calculation speed. Hardware API / assembly instruction acceleration shortens the time of conventional matrix multiplication; it consumes less computing power, maintains integer format throughout the process, has no floating-point conversion, reduces computing power consumption, and lowers MCU power consumption; the results are more stable, numerical normalization avoids overflow distortion, improves the accuracy of inference results, and ensures the reliability of command word recognition.

[0232] In one embodiment, step S24, which involves extracting the target activation value corresponding to the second type of weight position from the activation value data based on the second type of weight position information and performing sparse computation in combination with the quantized sub-weights, includes:

[0233] S241: Read the stored second type of weight position information, and determine the row index and column index of the second type of weight in the weight matrix based on the position information.

[0234] For example, in Project A, location information is stored in uint8 format, with each group of location information occupying 2 bytes (1 byte for row index + 1 byte for column index). Reading a group of location information 0x0A19 (hexadecimal), it is parsed as row index 10 (0x0A) and column index 25 (0x19), meaning this second type of weight is located in the 10th row and 25th column of the weight matrix. In a specific instance, 256 groups of location information are traversed, parsed sequentially into row and column indices such as (3,5), (10,25), (28,42), etc., and stored as an index array for subsequent extraction of activation values.

[0235] This step can quickly locate the position of the second type of weight in the matrix, providing a basis for accurately extracting activation values ​​and determining the result attribution, and avoiding blind traversal.

[0236] S242: Locate and extract the corresponding target activation value from the activation value data to be processed based on the column index, and determine the position of the target activation value in the calculation result based on the row index.

[0237] For example, in Project A, column index 25 corresponds to the 25th element of the activation value vector, and the quantized value 64 (int8) of this element is extracted as the target activation value; row index 10 corresponds to the 10th element of the first inference result, and it is determined that the sparse computation sub-result needs to be accumulated into the first inference result

[10] . In a specific instance, column index 5 corresponds to the quantized value -51 of the 5th element of the activation value, and row index 3 corresponds to the first inference result [3], so the target activation value is -51, and the position belongs to the first inference result [3].

[0238] This step can accurately extract the target activation value, avoid the waste of computing power caused by extracting irrelevant data, and at the same time clarify the location of the result to ensure accurate accumulation.

[0239] S243: Call the high-order sub-weights and low-order sub-weights in the quantized sub-weights. First, process the high-order sub-weights according to the preset bit operation logic and multiply them with the target activation value to obtain the high-order operation result. Then, multiply the low-order sub-weights with the target activation value to obtain the low-order operation result.

[0240] For example, in Project A, the default bitwise operation logic is to shift left by 8 bits. The high-order sub-weight -128 (corresponding to floating-point -0.5) is shifted left by 8 bits to obtain -32768, which is multiplied by the target activation value 64, resulting in the high-order operation result -32768 × 64 = -2099960 (int32). The low-order sub-weight -77 (corresponding to floating-point -0.3) is multiplied by the target activation value 64, resulting in the low-order operation result -77 × 64 = -4928 (int32). In a specific instance, another set of high-order sub-weights 100 (corresponding to floating-point 0.390625) is shifted left by 8 bits to obtain 25600, which is multiplied by the target activation value 39 to obtain 25600 × 39 = 998400. The low-order sub-weight 39 (corresponding to floating-point 0.15234375) is multiplied by 39 to obtain 1521. The high-order and low-order operation results are stored separately.

[0241] This step calculates the high and low digits separately to avoid numerical overflow caused by calculating all at once, while maintaining integer operations to reduce computational power consumption.

[0242] S244: The high-order operation result is superimposed with the low-order operation result to obtain a sparse calculation sub-result corresponding to a single second-type weight, and the sub-result is accumulated into the value corresponding to its belonging position until the sparse calculation of all second-type weights is completed to obtain the second inference calculation result.

[0243] For example, in Project A, the result of the high-order operation -2099960 is superimposed with the result of the low-order operation -4928 to obtain the sparse computation sub-result -2104888. This result is added to the first inference result

[10] (assuming the original first inference result

[10] is 56789), resulting in 56789+(-2104888)=-2048099. This process is repeated for all 256 second-class weights to obtain the second inference calculation result. In a specific instance, the first inference result for a certain location is 12345, and the corresponding sparse computation sub-result is 998400+1521=999921. After accumulation, it becomes 12345+999921=1012266, which is used as the final intermediate result for that location.

[0244] This step involves superimposing and accumulating the calculation results of the second type of weights to ensure that the influence of weights outside the range is accurately incorporated into the reasoning results, thereby improving the recognition accuracy.

[0245] This embodiment refines the "sparse computation" steps, with the core being "location-extraction-distribution-accumulation". For example, in Project A, during the smart speaker's inference process, the row and column indices of the second type of weights are first parsed to extract the corresponding target activation values ​​and determine the result's location. Then, the products of the high and low sub-weights and the activation values ​​are calculated separately, and these products are then accumulated and added to the corresponding position of the first inference result. This method accurately handles the calculation of out-of-range weights, ensuring inference accuracy without increasing computational power consumption, perfectly adapting to the needs of low-resource platforms.

[0246] Reference Figure 3 The present invention also provides a computer device, the internal structure of which can be as follows: Figure 3As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating devices, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores weights, etc. The network interface is used for communication with external terminals via a network connection. Furthermore, the computer device may also include an input device and a display screen, etc. When the computer program is executed by the processor, it implements the steps of the command word recognition learning network layer weight quantization method described in any of the above embodiments, and / or the inference calculation method of the command word recognition learning network described in any of the above embodiments. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0247] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the command word recognition learning network layer weight quantization method and / or the command word recognition learning network inference computation method described in any of the above embodiments. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0248] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0249] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0250] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A command word recognition learning network layer weight value quantization method characterized by, The method comprises: obtaining all weight values of a same-layer command word recognition learning network; based on a weight precision range of the same-layer command word recognition learning network, dividing the all weight values into first type weight values within the weight precision range and second type weight values beyond the weight precision range; for the first type weight values, using a fixed-point quantization mode corresponding to a preset low-precision integer format for quantization; for each second type weight value, splitting it into at least two sub weight values according to a preset rule, the preset rule comprising: each sub weight value falls within a numerical range of the low-precision integer format, and the sub weight values can be restored to the original second type weight value through a preset combination rule; for each split sub weight value, using the low-precision integer format for quantization, and recording position information of each second type weight value in a weight matrix and a corresponding sub weight value combination rule; storing the quantized first type weight values, the quantized sub weight values, the position information and the combination rule according to a preset storage structure; the storing of the quantized first type weight values, the quantized sub weight values, the position information and the combination rule according to the preset storage structure comprises: storing the preset combination rule, the quantized first type weight values, the number information of the quantized sub weight values, the quantized sub weight values and the position information of each second type weight value in the weight matrix in a preset order; wherein the storage format of the position information is adaptively determined according to the row and column dimension sizes of the weight matrix: if the row dimension or the column dimension of the weight matrix is less than or equal to a preset dimension threshold, a first preset storage format is used to store the position information, the first preset storage format being to store the row index and the column index in a low byte number integer format, and the total storage byte number of a single position information being less than a preset low byte upper limit; if the row dimension or the column dimension of the weight matrix is greater than the preset dimension threshold, a second preset storage format is used to store the position information, the second preset storage format being to store the index beyond the threshold in an integer format adapted to the threshold-exceeding dimension and to store the index not beyond the threshold in a low byte number integer format, the total storage byte number of a single position information being adapted to the dimension of the weight matrix and meeting the storage requirement of a low-resource platform, so as to adapt to the storage requirement of the low-resource platform.

2. The command word recognition learning network layer weight quantization method according to claim 1, characterized by, the dividing of the all weight values into the first type weight values within the weight precision range and the second type weight values beyond the weight precision range based on the weight precision range of the same-layer command word recognition learning network comprises: obtaining actual distribution characteristics of all weight values of a same-layer command word recognition learning network, the actual distribution characteristics comprising a numerical interval of the weight values, an extreme value range and a proportion of the weight values beyond a preset initial weight precision range; according to the actual distribution characteristics and a numerical representation range of a preset low-precision integer format, dynamically adjusting the initial weight precision range to obtain the weight precision range; using the weight precision range as a classification basis, dividing the weight values falling within the weight precision range into first type weight values and the weight values beyond the weight precision range into second type weight values.

3. The command word recognition learning network layer weight quantization method according to claim 2, characterized by, The initial weight precision range is dynamically adjusted according to the actual distribution characteristics and the numerical value representation range of the preset low-precision integer format to obtain the weight precision range, including: Determine the theoretical precision interval of the weight quantization under the numerical value representation range of the preset low-precision integer format; Extract the proportion of weights exceeding the initial weight precision range from the actual distribution characteristics of all weights, and compare the proportion with the preset reasonable proportion interval; If the proportion is higher than the upper limit of the reasonable proportion interval, expand the upper and lower limit thresholds of the initial weight precision range in the direction of the boundary of the theoretical precision interval, so that more weights fall into the adjusted weight precision range, until the proportion of weights exceeding the initial weight precision range falls within the reasonable proportion interval; or, If the proportion is lower than the lower limit of the reasonable proportion interval, reduce the upper and lower limit thresholds of the initial weight precision range in the direction of the numerical central interval of the weight, optimize the quantization precision while ensuring that most weights are still within the interval, until the proportion of weights exceeding the initial weight precision range rises within the reasonable proportion interval; Confirm that the adjusted weight precision range does not exceed the theoretical precision interval, and determine the adjusted range that meets this condition as the weight precision range.

4. The command word recognition learning network layer weight quantization method according to claim 1, characterized by, The preset combination rule includes: Divide at least two sub-weights obtained after splitting into high-bit sub-weights and low-bit sub-weights; Process the high-bit sub-weights through a preset bit operation logic to obtain high-bit processed values, wherein the preset bit operation logic is a left shift operation on the high-bit sub-weights according to a preset number of bits; Perform numerical superposition operation on the high-bit processed values and the low-bit sub-weights to obtain the operation result, which is the restored original second-type weight value. 5.The command word recognition learning network layer weight quantization method according to claim 1, wherein, The method for splitting each second-type weight value into at least two sub-weights according to a preset rule includes: Obtain the specific numerical value of a single second-type weight value, and determine the upper and lower limits of the numerical value representation under the preset low-precision integer format; Based on the upper and lower limits of the numerical value representation, split the second-type weight value into high-bit sub-weights and low-bit sub-weights, wherein the absolute value of the low-bit sub-weights does not exceed the numerical value representation range of the preset low-precision integer format, and the high-bit sub-weights are used to represent the numerical magnitude of the part of the second-type weight value exceeding the low-bit sub-weights; Verify whether the split high-bit sub-weights and low-bit sub-weights fall within the numerical range of the preset low-precision integer format, if both meet the requirements, the splitting of the second-type weight value is completed; if not, adjust the splitting method until all sub-weights meet the numerical range requirements.

6. An inference calculation method of a command word recognition learning network layer, characterized by, The method is based on the storage data obtained by the command word recognition learning network layer weight quantization method of any one of claims 1-5, including: Obtain the activation value data corresponding to the command word voice to be processed, and the quantized first-type weight value, the quantized sub-weight value, the second-type weight position information, and the sub-weight combination rule stored according to the preset storage structure; Determine the high-bit sub-weights and low-bit sub-weights corresponding to the second-type weight value based on the sub-weight combination rule, and restore the calculation logic of the second-type weight value; perform a regular matrix multiplication operation on the first type of quantized weight value and the activation value data to obtain a first inference calculation result; According to the second type of weight position information, the target activation value corresponding to the second type of weight position is extracted from the activation value data, and sparse calculation is performed combined with the quantized sub-weight value: first, the high-bit sub-weight value is processed according to the preset bit operation logic and multiplied by the target activation value, then the low-bit sub-weight value is multiplied by the target activation value, and the two product results are superimposed to obtain a second inference calculation result. The first inference calculation result and the second inference calculation result are accumulated to obtain the final command word recognition inference result. 7.The method of claim 6, wherein, The first type of quantized weight value and the activation value data are multiplied by calling an adaptive hardware API interface or through assembly instructions to realize matrix multiplication acceleration calculation; During the calculation process, the operation data is kept in a preset low-precision integer format for numerical interaction, avoiding additional power consumption caused by data format conversion; After completing the matrix multiplication operation, the numerical value of the operation result is normalized to ensure that the result value falls within a preset effective range, and the first inference calculation result is obtained. The second type of weight position information is read, and the corresponding row index and column index of the second type of weight value in the weight matrix are determined according to the position information. 8.The inference calculation method of the command word recognition learning network layer according to claim 6, characterized in that, According to the column index, the corresponding target activation value is located and extracted from the activation value data to be processed, and the row index is used to determine the ownership position of the result after the target activation value participates in the calculation. The high-bit sub-weight value and the low-bit sub-weight value in the quantized sub-weight value are called, the high-bit sub-weight value is processed according to the preset bit operation logic and multiplied by the target activation value to obtain a high-bit operation result; then the low-bit sub-weight value is multiplied by the target activation value to obtain a low-bit operation result; The high-bit operation result and the low-bit operation result are superimposed to obtain a sparse calculation sub-result corresponding to a single second type of weight value, and the sub-result is accumulated into the value corresponding to the ownership position, until the sparse calculation of all second type of weight values is completed, and the second inference calculation result is obtained. The processor executes the computer program to realize the steps of the command word recognition learning network layer weight quantization method of any one of claims 1-5, and / or the inference calculation method of the command word recognition learning network of any one of claims 6-8. ​ 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8. ​

Images