Neural network model training method and device, neural network model calculation method and device, electronic equipment and computer program product

Abstract

The invention relates to a neural network model training method and device, a neural network model calculation method and device, electronic equipment and a computer program product. According to the training method of the neural network model, the neural network model comprises a neural network layer, and the method comprises the following steps: dividing a target tensor in the neural network layer into at least two data blocks; for each data block, determining a corresponding public index according to the amplitude information of the elements in the data block; normalizing all elements in the data block relative to the common index to obtain a corresponding mantissa; for each element, only corresponding mantissa and sign bit are stored; the public indexes are stored according to data blocks, layers or channels; when the neural network model is trained, the public index, the bit width of the mantissa and the weight parameter of the neural network layer are updated through back propagation. According to the technical scheme, the requirements for the storage space and the memory bandwidth of the neural network model can be remarkably reduced on the premise that the precision is controllable.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a method, apparatus, electronic device, and computer program product for training and calculating a neural network model. Background Technology

[0002] In related technologies, as the scale of deep neural network (DNN) models continues to expand, the number of model parameters and intermediate activation data increases dramatically, placing higher demands on storage capacity, memory bandwidth, and energy consumption. To reduce inference and training costs, low-precision numerical representation and quantization methods are widely used in related technologies, such as the following three numerical representations: 1) Fixed-point quantization method By mapping floating-point numbers to integer representations with a uniform scaling factor, storage and computational complexity can be reduced, but significant precision loss can easily occur in network layers with a large numerical dynamic range.

[0003] 2) Value-by-value low-precision floating-point representation Formats such as FP16 and FP8 store the exponent and mantissa independently for each value. While this improves the dynamic range of the value, the repeated storage of the exponent causes additional storage and bandwidth overhead.

[0004] 3) Block Floating Point (BFP) By dividing a set of values ​​into blocks and sharing a common exponent within each block, exponent bit redundancy is reduced, thus balancing dynamic range and storage efficiency to some extent.

[0005] The above methods have achieved certain results in reducing resource consumption. However, the inventors of this application have found that the following shortcomings still exist when deploying large-scale neural networks for actual training and inference: 1) The index selection mechanism lacks dynamism and training friendliness. In existing schemes, the sharing index is usually statically determined based on the maximum value within the block or simple statistical rules, without considering the changes in the numerical distribution during the training of the neural network. This makes the index selection sensitive to the model accuracy and difficult to perform joint optimization during the training phase.

[0006] 2) The quantification scheme lacks an end-to-end unified design. Most solutions only optimize the representation format itself, without systematically providing a complete technical path from training, quantization, inference to hardware implementation. This results in the need for a large number of engineering patches during actual deployment, which limits the scope of application.

[0007] 3) Lack of flexibility in quantization granularity and bit width configuration Numerical distributions vary significantly across different network layers and tensor structures. Existing technologies often employ fixed block sizes and fixed mantissa bit widths, making it difficult to balance accuracy and resource efficiency. Summary of the Invention

[0008] The technical problem solved by this application is to provide a method, apparatus, electronic device and computer program product for training and calculating neural network models, which can significantly reduce the storage space and memory bandwidth requirements of neural network models while keeping the accuracy controllable.

[0009] According to a first aspect of the embodiments of this application, a method for training a neural network model is provided. The neural network model includes a neural network layer. The method includes: dividing a target tensor in the neural network layer into at least two data blocks; for each data block, determining a corresponding common exponent based on the magnitude information of the elements within the data block; normalizing all elements within the data block relative to the common exponent to obtain a corresponding mantissa; for each element, storing only the corresponding mantissa and sign bit; storing the common exponent by data block, by layer, or by channel; and updating the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layer through backpropagation during training of the neural network model.

[0010] In one implementation, the target tensor is a weight tensor or an activation tensor.

[0011] In one implementation, dividing the target tensor in the neural network layer into at least two data blocks includes: dividing the target tensor into at least two data blocks by rows, columns, channels, or a specified number of consecutive elements.

[0012] In one implementation, determining the corresponding common index based on the amplitude information of the elements within the data block includes: obtaining the amplitude information of the elements within the data block; the amplitude information being the maximum absolute value, root mean square value, p-norm, or percentile of the elements within the data block; and determining the common index corresponding to the data block based on the amplitude information.

[0013] In one implementation, determining the common index corresponding to the data block based on the amplitude information includes: determining the common index corresponding to the data block using a truncation method, a rounding method, or a learning method based on the amplitude information.

[0014] In one implementation, the public index is compressed and stored by data block, by layer, or by channel.

[0015] In one implementation, updating the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layer through backpropagation during training of the neural network model includes: using a pass-through estimator (STE) or a smooth approximation function when updating the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layer.

[0016] According to a second aspect of the embodiments of this application, a calculation method for a neural network model is provided. The neural network model includes neural network layers, and the neural network model has been trained using the training method described above. The calculation method includes: during inference, the neural network model acquires the stored common exponent and mantissa; based on the common exponent and mantissa, the elements are dequantized to restore them to floating-point numbers and then calculated, or the calculation is performed within the quantization domain.

[0017] In one embodiment, the calculation method of the neural network model further includes: periodically updating the common index at specified time intervals in online inference or adaptive scenarios, or updating the common index when the inference deviation exceeds a specified deviation range.

[0018] According to a third aspect of the embodiments of this application, a training apparatus for a neural network model is provided, the neural network model including neural network layers, the apparatus comprising: The tensor partitioning module is configured to partition the target tensor in the neural network layer into at least two data blocks; The first determining module is configured to determine the corresponding common index for each data block based on the amplitude information of the elements within the data block. The tail quantization module is configured to normalize all elements in the data block relative to the common index to obtain the corresponding tail number; The storage module is configured to store only the corresponding mantissa and sign bit for each element; the common index is stored by data block, by layer, or by channel; The training module is configured to update the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layers through backpropagation when training the neural network model.

[0019] According to a fourth aspect of the embodiments of this application, a computing device for a neural network model is provided, the neural network model including neural network layers, the neural network model having been trained using the training method described above, the computing device comprising: The acquisition module is configured to acquire the stored common exponent and the mantissa during inference of the neural network model; The calculation module is configured to, based on the common exponent and the mantissa, restore the element to a floating-point number through dequantization and perform calculations, or to complete the operation within the quantization domain.

[0020] According to a fifth aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0021] According to a sixth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0022] Compared with existing technologies, the beneficial effects of this application are as follows: By dividing the target tensor in the neural network layer into at least two data blocks, for each data block, the common exponent corresponding to the data block is determined based on the amplitude information of the elements within the data block. Then, all elements within the data block are normalized relative to the common exponent to obtain the corresponding mantissa. In this way, the common exponent can be determined specifically based on the amplitude information of each data block. Then, for each element within the data block, only the corresponding mantissa and sign bit are stored. The common exponent is stored by data block, by layer, or by channel. In this way, the introduction of floating-point representation of the common exponent at the data block granularity can significantly reduce exponent bit redundancy, thereby reducing data storage space and bandwidth requirements. When training the neural network model, the bit width of the common exponent and mantissa and the weight parameters of the neural network layer are updated through backpropagation. This enables joint optimization of the bit width of the common exponent and mantissa and the weight parameters of the neural network layer, making the quantization parameters trainable and the accuracy controllable. Thus, the storage space and memory bandwidth requirements of the neural network model can be significantly reduced under the premise of controllable accuracy.

[0023] During inference, the neural network model obtains the stored common exponent and mantissa, and performs calculations within the quantization domain based on the common exponent and mantissa. This avoids the complete dequantization process and improves energy efficiency. Therefore, it can improve the energy efficiency ratio of neural networks in edge devices or high-performance inference scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a training method for a neural network model according to an exemplary embodiment.

[0025] Figure 2 This is a flowchart illustrating a method for training a neural network model according to another exemplary embodiment.

[0026] Figure 3 This is a flowchart illustrating a computation method for a neural network model according to an exemplary embodiment.

[0027] Figure 4 This is a block diagram illustrating a training apparatus for a neural network model according to an exemplary embodiment.

[0028] Figure 5 This is a block diagram of a computing device for a neural network model according to another exemplary embodiment.

[0029] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0030] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. Specific embodiments of this application will be described below in conjunction with the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. Without departing from the spirit and scope of this application, those skilled in the art can modify and substitute the embodiments of this application, and the resulting embodiments are also within the protection scope of this application.

[0031] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] Before introducing the training method of the neural network model provided in this application, let's first introduce standard floating-point numbers and block floating-point numbers (BFP).

[0033] Standard floating-point numbers: The data structure of standard floating-point numbers includes one sign bit and the exponent is located in the mantissa. Before storage, the exponent and mantissa need to be quantized. The storage method is to store the exponent and mantissa for each number. The exponent and mantissa together determine the precision.

[0034] Block Floating-Point Numbers (BFP): The data structure of a block floating-point number includes a common exponent and multiple "mantissa-only" data. Before storage, only the "mantissa" is quantized. The exponent is shared or stored separately. The storage method is to store one copy of the exponent and store only the mantissa for each number. The common exponent determines the dynamic range, and the mantissa precision determines the precision.

[0035] One embodiment of this application provides a method for training a neural network model. The neural network model may include multiple neural network layers, and any one of these layers can be trained using the method provided in this embodiment. Please refer to... Figure 1 The training method for this neural network model may include the following steps S101 to S105: Step S101: Divide the target tensor in the neural network layer into at least two data blocks.

[0036] In this step, for neural network layers with moderate sensitivity, the target tensor in the neural network layer is divided into at least two data blocks. Sensitivity refers to the degree to which the model output responds to changes in the input parameters.

[0037] In one embodiment, the target tensor can be a weight tensor. In another embodiment, the target tensor can be an activation tensor.

[0038] In one embodiment, the target tensor is divided into at least two data blocks according to channels. The following example illustrates this, using the target tensor as the weight tensor and dividing the weight tensor by channels.

[0039] Suppose the weight tensor of a certain neural network layer is: (1) Where W is the weight tensor, Number of output channels R is the number of input channels, and R is a real number.

[0040] The weight tensor is divided into multiple data blocks according to the channel partitioning method: (2) Each data block Each data block corresponds to one output channel.

[0041] Dividing the target tensor according to channels ensures that the numerical distribution within the data block is relatively concentrated, which is suitable for shared exponential representation.

[0042] In another embodiment, the target tensor can be divided into at least two data blocks by rows. This tensor partitioning method is suitable for data parallelism or large sample sizes. Assuming the target tensor is [Out_Features, In_Features,], where Out_Features is the number of output features and In_Features is the number of input features, partitioning by rows means splitting the Out_Features dimension. Each data block has complete input features but only accounts for a portion of the output features.

[0043] In another embodiment, the target tensor is divided into at least two data blocks by columns. This tensor partitioning method is suitable for layer splitting in model parallelism. Column-wise partitioning means splitting along the In_Features dimension. Each data block processes only a portion of the input data features, and each data block has the complete output features.

[0044] In another embodiment, the target tensor is divided into at least two data blocks by a specified number of consecutive elements. This tensor partitioning method is suitable for sparse matrix storage, matrix block partitioning, or vectorized memory access optimization. This tensor partitioning method does not consider the logical dimension (rows or columns) of the data, but instead flattens the data of the target tensor into one dimension in memory and then divides it according to a specified physical size (e.g., each data block consists of 128 elements).

[0045] Step S102: For each data block, determine the corresponding common index based on the amplitude information of the elements within the data block.

[0046] In one embodiment, such as Figure 2 As shown, step S102 may include the following steps S1021-S1022: Step S1021: Obtain the amplitude information of the elements within the data block.

[0047] In one embodiment, the magnitude information of an element within a data block can be the maximum absolute value of the element within the data block. For any data block... Get the maximum absolute value of elements within a data block. As amplitude information: (3) In another embodiment, the magnitude information of the elements within the data block can be the root mean square value of the elements within the data block.

[0048] In another embodiment, the magnitude information of elements within a data block can be the p-norm.

[0049] In another embodiment, the magnitude information of elements within a data block can be percentiles, for example, the magnitude information of elements within a data block can be the 95th percentile or the 90th percentile, but is not limited thereto.

[0050] Step S1022: Determine the common index corresponding to the data block based on the amplitude information of the elements within the data block.

[0051] In one embodiment, the common index corresponding to the data block can be determined by using a truncation method, a rounding method, or a learning method based on the magnitude information of the elements within the data block. : (4) Operators that indicate truncation, rounding, or learning modes.

[0052] During the training phase, the process can employ differentiable approximations or straight-through estimators (STEs) for backpropagation.

[0053] The learning approach refers to treating the common index as a learnable parameter (usually discrete, requiring backpropagation of gradients using a STE (Straight-Through Estimator)) or dynamically updating the common index during training through an optimization algorithm.

[0054] Step S103: Normalize all elements in the data block relative to the common exponent to obtain the corresponding mantissa.

[0055] In this step, all elements within the data block are normalized relative to a common exponent to obtain the corresponding mantissa. For any element within the data block... Its common exponent floating-point representation is: (5) The mantissa is a quantization function, with the mantissa represented in m-bit fixed-point or floating-point format. All elements within the data block share the same common exponent. .

[0056] Step S104: For each element, only the corresponding mantissa and sign bit are stored; the common index is stored by data block, by layer, or by channel.

[0057] In this step, for each element It only stores the corresponding mantissa and sign bit. That is, the stored content includes: a common exponent for each data block. The mantissa and sign bit of each element.

[0058] In one embodiment, the public index is stored in blocks. Each block has its own independent public index.

[0059] In one embodiment, the common index is stored layer by layer. All data blocks with the same common index in each neural network layer share a single common index.

[0060] In one embodiment, the common index is stored per channel. For each output channel, a separate common index is stored for all data blocks that share the same common index.

[0061] In one embodiment, the public index is compressed and stored by data block, by layer, or by channel.

[0062] Step S105: When training the neural network model, update the bit width of the common exponent and mantissa and the weight parameters of the neural network layers through backpropagation.

[0063] In this step, during the training of the neural network model, the bit width of the common exponent and mantissa, as well as the weight parameters of the neural network layer, are updated through backpropagation, thereby achieving joint optimization of the bit width of the common exponent and mantissa, and the weight parameters of the neural network layer.

[0064] During the training phase of the neural network model, quantization operators are inserted into the forward computation graph, and backpropagation is performed using the following strategy: a straight-through estimator (STE) or a smooth approximation function is used for the selection of the common exponent and tail quantization. That is, the quantized value is used during forward propagation. When updating the bit width of the common exponent and mantissa, as well as the weight parameters of the neural network layer, a pass-through estimator (STE) or a smooth approximation function is used to achieve joint optimization of the bit width configuration parameters of the common exponent and mantissa at the data block level and the weight parameters of the neural network layer.

[0065] In one embodiment, different neural network layers of a neural network model can employ different data block sizes and different exponent sharing strategies to further balance accuracy and efficiency. In this embodiment, the bit width of the mantissa and the granularity of the data block can be flexibly configured by layer and by data block, which can enhance the adaptability of the mantissa quantification scheme to different network structures and layers.

[0066] In one embodiment, the common exponential floating-point representation, fixed-point quantization, and value-wise floating-point representation from this application can be used interchangeably in the same neural network model for neural network layers with different sensitivities. Specifically, the sensitivity of the neural network layer can be divided into three levels: high, medium, and low. For neural network layers with high sensitivity, value-wise floating-point representation is used to improve the dynamic range of the values. For neural network layers with medium sensitivity, the common exponential floating-point representation from this application is used (the technical solutions described in steps S101-S103 above), which can significantly reduce the storage space and memory bandwidth requirements of the neural network model while maintaining controllable accuracy. For neural network layers with low sensitivity, the fixed-point quantization method from this application is used to represent the floating-point values, which can reduce storage and computational complexity without easily causing significant loss of accuracy.

[0067] In this embodiment, the target tensor in the neural network layer is divided into at least two data blocks. For each data block, the common exponent corresponding to the data block is determined based on the amplitude information of the elements within the data block. Then, all elements within the data block are normalized relative to the common exponent to obtain the corresponding mantissa. In this way, the common exponent can be determined specifically based on the amplitude information of each data block. Then, for each element within the data block, only the corresponding mantissa and sign bit are stored. The common exponent is stored by data block, by layer, or by channel. When training the neural network model, the bit width of the common exponent and mantissa and the weight parameters of the neural network layer are updated through backpropagation. This enables joint optimization of the bit width of the common exponent and mantissa and the weight parameters of the neural network layer, thereby significantly reducing the storage space and memory bandwidth requirements of the neural network model while maintaining controllable accuracy.

[0068] Another embodiment of this application provides a method for calculating a neural network model. The neural network model has been trained using the training method described above. Please refer to... Figure 3 The calculation method of this neural network model may include the following steps S301 to S302: Step S301: During inference, the neural network model obtains the stored common exponent and mantissa.

[0069] Step S302: Based on the common exponent and mantissa, the elements are dequantized to restore them to floating-point numbers and then calculated, or the operation is completed within the quantization domain.

[0070] During the inference phase, the neural network model can be computed in the following two ways: I. Inverse Quantization Method During inference, the neural network model retrieves the stored common exponent and mantissa, restores the elements to floating-point numbers through dequantization, and then performs calculations based on the floating-point numbers, such as performing standard multiply-accumulate operations.

[0071] II. Calculation Method of Equivalent Quantization Domain During inference, the neural network model obtains the stored common exponent and mantissa, and performs the calculations directly in the quantization domain in the hardware. Specifically, this includes performing multiplication and addition operations on the mantissa and uniformly shifting or aligning the exponent.

[0072] Let me first introduce the mathematical principles behind the calculation of the equivalent quantization field. Using the associative law of multiplication, the exponent part is extracted: = (6) This means that the multiplication unit only needs to process the mantissa. Sum of last digits Fixed-point multiplication, exponent part , It was temporarily "set aside".

[0073] The calculation process is described below (using matrix multiplication as an example). Assume we are calculating C = A × B, where A, B, and C are matrices, and A is divided into rows (mantissa). ,index B is divided into blocks by column (last digit) ,index ).

[0074] (1) Pure integer multiplication and addition: The computing core is only responsible for executing: Acc int =∑(M 1i ×M 2i ).

[0075] This involves fixed-point integer arithmetic without any exponent alignment logic, making it extremely fast and efficient to implement using DSP slicing.

[0076] (2) Index processing (delay or merging): Calculate the exponential factor corresponding to each partial product: .

[0077] There are two ways to handle this: Method A (accumulation followed by adjustment): After calculating the total sum, multiply by 2 all at once. Etotal (i.e., shifting).

[0078] Method B (Aligned Accumulation): If matrix multiplication involves multiple blocks, and different data blocks have different... At this point, it is necessary to multiply the integer products Acc of different data blocks before accumulation. int Shift and align, then add.

[0079] The technical solution provided in this application can restore elements to floating-point numbers and perform calculations by dequantization based on the common exponent and mantissa, or it can complete the calculation within the quantization domain, which facilitates efficient deployment on dedicated accelerators or general-purpose processors.

[0080] The technical solution provided in this application allows for customized shared exponent alignment and computation paths for operators such as matrix multiplication and convolution, thereby further improving hardware execution efficiency.

[0081] During inference, the neural network model obtains the stored common exponent and mantissa, and performs calculations within the quantization domain based on the common exponent and mantissa. This avoids the complete dequantization process, improves energy efficiency, and can further enhance hardware execution efficiency. Consequently, it can improve the energy efficiency ratio of neural networks in edge devices or high-performance inference scenarios.

[0082] In one embodiment, in online inference or adaptive scenarios, the public index can be updated periodically at specified time intervals.

[0083] In another embodiment, in online inference or adaptive scenarios, the common index is updated when the inference bias of the neural network model exceeds a specified bias range to adapt to changes in the distribution of input data.

[0084] Another exemplary embodiment of this application provides a training apparatus for a neural network model. The neural network model includes neural network layers, such as... Figure 4 As shown, in this embodiment, the training device for the neural network model includes: Tensor partitioning module 41 is configured to partition the target tensor in the neural network layer into at least two data blocks.

[0085] The first determining module 42 is configured to determine the corresponding common index for each data block based on the amplitude information of the elements within the data block. The first determining module 42 can also be configured to broadcast the common index to the elements within the data block.

[0086] The mantissa quantization module 43 is configured to normalize all elements within a data block relative to a common exponent to obtain the corresponding mantissa. The mantissa quantization module 43 can also be configured to align the mantissa data.

[0087] Storage module 44 is configured to store only the corresponding mantissa and sign bit for each element; the common index is stored by data block, by layer, or by channel.

[0088] Training module 45 is configured to update the bit width of the common exponent, the mantissa, and the weight parameters of the neural network layers via backpropagation when training the neural network model.

[0089] In one embodiment, the training apparatus for the neural network model may further include: The configuration control unit is configured to set the data block size, the mantissa bit width, and the common exponent update strategy. The common exponent update strategy can be to update the common exponent, the mantissa bit width, and the weight parameters of the neural network layer through backpropagation, or to update the common exponent periodically at specified time intervals, or to update the common exponent when the inference bias exceeds a specified bias range.

[0090] Another exemplary embodiment of this application provides a computing device for a neural network model. The neural network model includes neural network layers, and the neural network model has been trained using the training method described above, such as... Figure 5 As shown, in this embodiment, the computing device for the neural network model includes: The acquisition module 51 is configured to acquire the stored common exponent and mantissa during inference of the neural network model.

[0091] The calculation module 52 is configured to either restore the elements to floating-point numbers by dequantization based on the common exponent and mantissa and perform calculations, or to complete the calculations within the quantization domain.

[0092] The method and apparatus embodiments of this application can complement each other.

[0093] Embodiments of this application also propose an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the training method and calculation method of the neural network model of any of the above embodiments.

[0094] Embodiments of this application also propose a computer-readable storage medium that, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the training method and calculation method of the neural network model of any of the above embodiments.

[0095] Embodiments of this application also propose a computer program product, including a computer program that, when executed by a processor, implements the training method and calculation method of the neural network model of any of the above embodiments.

[0096] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0097] Embodiments of this application also provide an electronic device 600, such as... Figure 6 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs executable by the processor 602; the processor 602 is used to execute the computer programs in the memory 601 to implement the training method and calculation method of the neural network model provided in any of the above embodiments.

[0098] The electronic device 600 also includes a communication interface 603. The processor 602, memory 601, and communication interface 603 are connected via a communication bus and communicate with each other.

[0099] Processor 602 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0100] Communication interface 603 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0101] The memory 601 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0102] In this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0103] The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. A method for training a neural network model, characterized in that, The neural network model includes neural network layers, and the method includes: The target tensor in the neural network layer is divided into at least two data blocks; For each data block, a corresponding common index is determined based on the magnitude information of the elements within the data block; Normalize all elements within the data block relative to the common index to obtain the corresponding mantissa; For each element, only the corresponding mantissa and sign bit are stored; the common index is stored by data block, by layer, or by channel; During the training of the neural network model, the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layers are updated through backpropagation.

2. The training method for the neural network model as described in claim 1, characterized in that, The target tensor is either a weight tensor or an activation tensor.

3. The training method for the neural network model as described in claim 1, characterized in that, The step of dividing the target tensor in the neural network layer into at least two data blocks includes: The target tensor is divided into at least two data blocks by rows, columns, channels, or a specified number of consecutive elements.

4. The training method for the neural network model as described in claim 1, characterized in that, The step of determining the corresponding common index based on the amplitude information of the elements within the data block includes: Obtain the amplitude information of the elements within the data block; the amplitude information is the maximum absolute value, root mean square value, p-norm, or percentile of the elements within the data block; The common index corresponding to the data block is determined based on the amplitude information.

5. The training method for the neural network model as described in claim 4, characterized in that, Determining the common index corresponding to the data block based on the amplitude information includes: Based on the magnitude information, the common index corresponding to the data block is determined using a truncation method, a rounding method, or a learning method.

6. The training method for the neural network model as described in claim 1, characterized in that, The public index is compressed and stored according to the data blocks, layers, or channels.

7. The training method for the neural network model as described in claim 1, characterized in that, The step of updating the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layers through backpropagation during training of the neural network model includes: When updating the common index, the bit width of the mantissa, and the weight parameters of the neural network layer, a pass-through estimator (STE) or a smooth approximation function is used.

8. A method for calculating a neural network model, characterized in that, The neural network model includes neural network layers, and the neural network model has been trained using the training method described in any one of claims 1 to 7. The calculation method includes: During inference, the neural network model acquires the stored common exponent and the tail number; Based on the common exponent and the mantissa, the element is restored to a floating-point number through dequantization and then calculated, or the operation is performed within the quantization domain.

9. The calculation method for the neural network model as described in claim 8, characterized in that, Also includes: In online inference or adaptive scenarios, the common index is updated periodically at specified time intervals, or when the inference deviation exceeds a specified deviation range.

10. A training device for a neural network model, characterized in that, The neural network model includes neural network layers, and the device includes: The tensor partitioning module is configured to partition the target tensor in the neural network layer into at least two data blocks; The first determining module is configured to determine the corresponding common index for each data block based on the amplitude information of the elements within the data block. The tail quantization module is configured to normalize all elements in the data block relative to the common index to obtain the corresponding tail number; The storage module is configured to store only the corresponding mantissa and sign bit for each element; the common index is stored by data block, by layer, or by channel; The training module is configured to update the common exponent, the bit width of the mantissa, and the weight parameters of the neural network layers through backpropagation when training the neural network model.

11. A computing device for a neural network model, characterized in that, The neural network model includes neural network layers, and the neural network model has been trained using the training method described in any one of claims 1 to 7. The computing device includes: The acquisition module is configured to acquire the stored common exponent and the mantissa during inference of the neural network model; The calculation module is configured to, based on the common exponent and the mantissa, restore the element to a floating-point number through dequantization and perform calculations, or to complete the operation within the quantization domain.

12. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method as described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images