Data processing method and device, electronic equipment and storage medium
By dividing the deep learning model into block structures and performing quantized perceptual training, the problem of accuracy loss in model compression algorithms on edge devices is solved, achieving a reduction in model size and power consumption while ensuring accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
- Filing Date
- 2023-01-16
- Publication Date
- 2026-06-09
AI Technical Summary
Existing model compression algorithms can lead to a loss of accuracy in deep learning models when deployed on edge devices, especially in low-bit quantization and deeper neural networks. Traditional quantization algorithms such as PTQ and QAT suffer from accuracy loss.
By obtaining the quantization bits corresponding to each network layer of the model to be compressed, the model is divided into block structures based on the simulated power, and partial quantization-aware training and loss function adjustment are performed to ensure the inference accuracy of the model.
Within a reasonable range that ensures model accuracy, reduce model size, decrease memory usage and power consumption, and increase computation speed.
Smart Images
Figure CN116306879B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a data processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] With the widespread application of deep learning algorithms in fields such as computer vision, speech processing, and autonomous driving, deep learning models (networks) are being designed to be increasingly large to meet the demands of complex applications, high precision, and multi-tasking. When deploying deep learning algorithms on edge devices or mobile platforms—devices with high limitations on power, latency, and memory resources—model compression algorithms are often required for optimization. While data processing methods can effectively reduce hardware memory usage and improve computational speed, existing model compression algorithms inevitably result in some loss of model accuracy. Summary of the Invention
[0003] In view of the above problems, this application proposes a data processing method, apparatus, electronic device, and storage medium to improve the above problems.
[0004] In a first aspect, embodiments of this application provide a data processing method, the method comprising: acquiring a model to be compressed, the model to be compressed comprising multiple network layers; acquiring quantization bits corresponding to each network layer of the model to be compressed, the quantization bits being determined based on the analog power of the network layer during forward inference; dividing the model to be compressed into block structures, the block structures comprising at least one of the network layers; performing quantization-aware training on some parameters in the block structures based on the corresponding quantization bits, and adjusting and updating the remaining parameters in the block structures according to a loss function to ensure the inference accuracy of the model to be compressed.
[0005] Secondly, embodiments of this application provide a data processing apparatus, comprising: a model acquisition module for acquiring a model to be compressed, the model to be compressed including multiple network layers; a quantization bit acquisition module for acquiring quantization bits corresponding to each network layer of the model to be compressed, the quantization bits being determined based on the analog power of the network layer during forward inference; a model partitioning module for partitioning the model to be compressed into block structures, the block structures including at least one of the network layers; and a model compression module for performing quantization-aware training on some parameters in the block structures based on the corresponding quantization bits, and adjusting and updating the remaining parameters in the block structures according to a loss function to ensure the inference accuracy of the model to be compressed.
[0006] Thirdly, this application provides an electronic device including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method described in the first aspect above.
[0007] Fourthly, this application provides a computer-readable storage medium storing program code, wherein the method described in the first aspect is executed when the program code is run.
[0008] This application provides a data processing method, apparatus, electronic device, and storage medium. The method involves acquiring a model to be compressed, which includes multiple network layers; acquiring the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the simulated power of the network layer during forward inference; dividing the model to be compressed into block structures, each block structure including at least one of the network layers; performing quantization-aware training on some parameters in the block structure based on the corresponding quantization bits; and adjusting and updating the remaining parameters in the block structure according to a loss function to ensure the inference accuracy of the model to be compressed. This approach achieves a reduction in model size, thereby reducing memory usage and power consumption, compared to traditional quantization-aware training methods, by incorporating some quantization noise for optimization iteration, while the gradients of the remaining tensors can continue to propagate unbiasedly like a floating-point model. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 A flowchart of a data processing method according to an embodiment of this application is shown.
[0011] Figure 2 A flowchart of a data processing method according to another embodiment of this application is shown.
[0012] Figure 3 A flowchart of a data processing method according to another embodiment of this application is shown.
[0013] Figure 4 A structural block diagram of a data processing apparatus according to an embodiment of this application is shown.
[0014] Figure 5 A structural block diagram of an electronic device for performing a data processing method according to an embodiment of the present application is shown.
[0015] Figure 6 It is a storage unit in this application embodiment for storing or carrying program code that implements a data processing method according to this application embodiment. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0017] Model compression methods can effectively reduce hardware memory usage and improve computation speed. Currently, commonly used model compression algorithms include quantization, pruning, and distillation. Pruning and distillation achieve model compression by reducing model parameters; the compressed model structure is usually significantly different from the original. Quantization reduces the number of bits used to represent model parameters, such as quantizing a 32-bit floating-point model to 4, 8, or 16-bit fixed-point numbers. This allows for further model compression even when the model structure has already been optimized.
[0018] Quantization strategies can be divided into post-training quantization (PTQ) and training-aware quantization (QAT). PTQ directly quantizes the parameters after training the deep learning network, converting 32-bit floating-point numbers to fixed-point numbers. QAT, on the other hand, builds upon PTQ by continuing training for several iterations. During training, the quantized parameters are used for forward inference, and the floating-point parameters are updated backward based on the loss function. QAT offers higher accuracy than PTQ. However, for PTQ, since no quantization perturbation is applied during training, direct post-training quantization leads to a significant loss of accuracy, especially in deeper neural networks and when using low-bit quantization. For QAT, quantization is performed during the forward inference phase of training. This conversion is not differentiable, and backpropagation gradients are typically approximated using a pass-through estimator (STE), which also results in some accuracy loss. In short, existing quantization algorithms inevitably sacrifice model accuracy to some extent.
[0019] To facilitate understanding of the solutions described in the embodiments of this application, the terms involved in the embodiments of this application are briefly explained below:
[0020] AI (Artificial Intelligence): Represents artificial intelligence.
[0021] PTQ (Post Training Quantization): Quantization of representations after training.
[0022] QAT (Quantization Aware Training): Quantization Aware Training.
[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0024] Please see Figure 1 One embodiment of this application provides a data processing method, the method comprising:
[0025] Step S110: Obtain the model to be compressed, which includes multiple network layers.
[0026] In this embodiment, the model to be compressed is a trained floating-point precision model, which may include multiple network layers. For example, model A to be compressed may include convolutional layers, normalization layers, activation layers, and downsampling layers. In at least one embodiment, the multiple network layers included in the model to be compressed may be different, or some of the network layers may be the same, or the model to be compressed may include only one network layer; the specific implementation is not limited. This embodiment uses an example of a model to be compressed including multiple different network layers for illustration.
[0027] The specific type of the model to be compressed is not limited. For example, the model to be compressed can be an AI neural network model used for visual processing, speech processing, or natural language processing.
[0028] As one approach, when it is necessary to deploy a model to edge devices with limited memory resources, such as mobile phones, the pre-trained floating-point precision model corresponding to the model to be deployed can be obtained as the model to be compressed.
[0029] Step S120: Obtain the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the analog power of the network layer during forward inference.
[0030] In this context, quantization bits can be understood as the number of bits used to represent the parameters after quantization. For example, a quantization bit of 4 bits means that the model's parameters (from 32-bit floating-point numbers) are quantized to 4-bit fixed-point numbers. One approach is to reduce the model size by decreasing the number of bits used to represent the parameters in the network layers of the model to be compressed. In other words, by reducing the number of bits used to represent the parameters, the numerical representation space (bit space) can be reduced, thereby reducing the model size.
[0031] In this embodiment, the quantization bits corresponding to each network layer of the model to be compressed can be different. For example, assuming the model to be compressed includes network layer A, network layer B, and network layer C, then the quantization bits corresponding to network layer A can be 4 bits, the quantization bits corresponding to network layer B can be 8 bits, and the quantization bits corresponding to network layer C can be 16 bits. Alternatively, the quantization bits corresponding to each network layer of the model to be compressed can be partially the same. For example, in the above example, the quantization bits corresponding to network layer A can be 4 bits, the quantization bits corresponding to network layer B can also be 4 bits, and the quantization bits corresponding to network layer C can be 8 bits. The values of the quantization bits mentioned above are only illustrative examples and do not constitute a limitation of this solution.
[0032] The higher the value of the quantization bits, the higher the accuracy of the quantized model; the lower the value of the quantization bits, the smaller the size of the quantized model. It is understandable that, due to the different structures of different network layers, the parameters of the network layers will also differ. In order to preserve the original structure of the network layers as much as possible, the quantization bits corresponding to each network layer can be different, or only some network layers can have the same quantization bits, rather than all network layers having the same quantization bits. That is, this application uses a mixed precision quantization of the network layers of the model to be compressed, composed of different quantization bits.
[0033] In this embodiment, the quantization bits are determined based on the simulated power of the network layer during forward inference. One approach is to calculate the second derivative of the model's loss function with respect to the weight parameters. Then, a hardware simulator uses the simulated power of each network layer during forward inference as sensitivity information (latency, power, and model size can all be used as sensitivity information; this embodiment uses simulated power as an example). A linear programming problem is constructed to solve for the quantization bits of each network layer by minimizing the sum of perturbation information under the constraint of a network and total power (i.e., the sum of the simulated power corresponding to each network layer). The perturbation information can be understood as the degree of information change during the quantization of parameters from floating-point to fixed-point numbers.
[0034] In determining the quantization bits for each network layer, the appropriate quantization bits can be calculated based on the magnitude of the second derivative of the model's loss function with respect to the weight parameters. Specifically, regarding the sensitivity information of simulated power, if the second derivative of a network layer is higher and steeper, it indicates that this network layer is more sensitive to power. In this case, to ensure the model's inference accuracy (assuming that a larger model size leads to higher inference accuracy), a larger number of quantization bits can be allocated, such as 8 bits or 16 bits. Conversely, if the second derivative of a network layer is relatively small, it means that using high or low bits during quantization will have a smaller impact on power perturbation. In this case, to minimize the model size, 4 bits can be used. Through mathematical analysis, it is possible to dynamically infer which layer should be quantized with which corresponding bit count, thereby reducing the model size while ensuring the model's inference accuracy (i.e., the simulated power meets the preset highest bit requirement, which can be understood as the simulated power not exceeding a certain preset threshold, the specific value of which is not limited).
[0035] Step S130: Divide the model to be compressed into a block structure, wherein the block structure includes at least one of the network layers.
[0036] A block structure is a network block. As one implementation method, the model to be compressed can be divided into a block-based structure based on the characteristics of the model structure. For example, assuming that the model to be compressed M includes convolutional layers, normalization layers, activation layers, and downsampling layers, and that the convolutional layers, normalization layers, activation layers, and downsampling layers are usually a complete process, then the convolutional layers, normalization layers, activation layers, and downsampling layers can be grouped into a block structure.
[0037] Optionally, the partitioning of the block structure can depend on the network architecture, the user's computing power, and the size of the training data in QAT. For example, blocks with residual connections are usually not split; if the computing power is sufficient, the partitioned blocks can contain more network layers.
[0038] A block structure may include at least one of the network layers. In the case where the model to be compressed includes multiple network layers, if the block structure includes at least one network layer, then the model to be compressed may include at least one block structure; if the block structure includes one network layer, then the model to be compressed may include multiple block structures.
[0039] Step S140: Perform quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and adjust and update the remaining parameters in the block structure according to the loss function to ensure the inference accuracy of the model to be compressed.
[0040] For each block structure of the model to be compressed, quantization-aware training can be performed on some parameters of the block structure based on the quantization bits corresponding to the network layers in the block structure. If the block structure includes multiple network layers, mixed-precision quantization-aware training can be performed on some parameters of the block structure based on the quantization bits corresponding to each of the multiple network layers in the block structure. The quantization noise is caused by quantizing the original 32-bit floating-point precision values in the network layers to 4, 8, or 16-bit fixed-point integers, which greatly reduces the representation space (bit space) and results in some loss of inference precision. The remaining parameters in the block structure are then adjusted and updated according to the loss function until the inference precision of the model to be compressed reaches the preset precision or the training epochs of the model to be compressed reach the preset number of epochs, to ensure the inference precision of the model to be compressed. The remaining parameters can be understood as parameters that maintain floating-point precision. The loss function is the loss function of the model to be compressed, and the specific type of loss function can be determined according to the type of the model to be compressed. For parameters that maintain floating-point precision, the model parameters can be updated by backpropagation using gradient descent on the loss function. By performing quantized perceptual training on only a subset of parameters within the block structure, the model's structure can be preserved to the greatest extent possible, thereby reducing the model size while maintaining its inference accuracy as much as possible. The specific values for the preset accuracy and the preset number of iterations are not limited.
[0041] In the process of quantization-aware training of some parameters in the block structure based on the quantization bits corresponding to the network layers, one implementation method is to introduce quantization error to represent the differences in numerical representations. This method quantizes the parameters of the network layers and activation outputs in the block structure from floating-point numbers to fixed-point integers, then dequantizes the fixed-point integers back to floating-point numbers, and replaces the original parameters and activation outputs with the dequantized floating-point numbers. The formula for quantization-aware training is as follows:
[0042]
[0043] Among them, W ′ The parameters represent the two types mentioned above after quantization and dequantization, with s representing the quantization scale and z representing the quantization zeros. It should be noted that if a zero matrix is chosen to replace the original parameters and activation outputs, then the quantization-aware training process is similar to the pruning process.
[0044] This application provides a data processing method that involves obtaining a model to be compressed, the model comprising multiple network layers; obtaining the quantization bits corresponding to each network layer of the model to be compressed, the quantization bits being determined based on the simulated power of the network layer during forward inference; dividing the model to be compressed into block structures, each block structure comprising at least one of the network layers; performing quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and adjusting and updating the remaining parameters in the block structure according to a loss function to ensure the inference accuracy of the model to be compressed. Thus, compared to traditional quantization-aware training methods, this method incorporates some quantization noise for optimization iteration, while the gradients of the remaining tensors can continue to propagate unbiasedly like a floating-point model. This allows for a reduction in model size while maintaining reasonable inference accuracy, thereby reducing memory usage and power consumption.
[0045] Please see Figure 2 Another embodiment of this application provides a data processing method, the method comprising:
[0046] Step S210: Obtain the model to be compressed, which includes multiple network layers.
[0047] The specific implementation of step S210 can be referred to the relevant description of step S110 in the foregoing embodiments, and will not be repeated here.
[0048] Step S220: Obtain the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the analog power of the network layer during forward inference.
[0049] The specific implementation of step S220 can be referred to the relevant description of step S120 in the foregoing embodiments, and will not be repeated here.
[0050] Step S230: Divide the model to be compressed into a block structure, wherein the block structure includes at least one of the network layers.
[0051] The specific implementation of step S230 can be referred to the relevant description of step S130 in the foregoing embodiments, and will not be repeated here.
[0052] Step S240: Quantize some parameters in the block structure into integers of the corresponding quantization bits.
[0053] In the process of quantization-aware training of some parameters in the block structure based on the quantization bits corresponding to the network layers, a subset of specific blocks can be selected from the block structure after the model to be compressed is divided to add quantization perturbation. There are several ways to select specific blocks. For example, some specific blocks can be selected randomly; blocks can be selected block by block, that is, each block or several blocks can be selected sequentially starting from the first block of the network; or the block division can be used to assign an index, and one or several blocks can be randomly sampled as candidates each time. This results in greater randomness in the sampling, and if the number of iterations is sufficient, it can be ensured that each block is sampled; or the number of blocks sampled each time can be increased progressively, such as sampling 1 block the first time, 2 blocks the second time, 4 blocks the third time, etc.
[0054] One approach is to first quantize some parameters in the block structure into quantization bits corresponding to the network layers included in the block structure. Specifically, as one implementation method, some parameters in the block structure can be progressively quantized into integers of the quantization bits corresponding to the network layers included in the block structure. That is, for some parameters in the block structure, the proportion of quantization noise is progressively added. For example, the proportion of quantization operation can be gradually increased for the sampled block tensor at ratios of 10%, 30%, 50%, etc. The numerical proportions here are only examples and do not constitute a limitation of this scheme. By progressively increasing the proportion of quantization noise, it will not directly cause significant disturbance to the initially trained floating-point precision model and can accelerate the training of the model.
[0055] In at least one embodiment, the proportion of added quantization noise can also be increased to 100%.
[0056] Step S250: Dequantize the integer into a floating-point number and replace some parameters in the block structure with the floating-point number; and adjust and update the remaining parameters in the block structure according to the loss function to ensure the inference accuracy of the model to be compressed.
[0057] As described in the foregoing embodiments, after quantizing some parameters in the block structure into integers corresponding to the quantization bits of the network layers included in the block structure, the integers can be dequantized into floating-point numbers, and the floating-point numbers can be used to replace some parameters in the block structure. To further simplify the model, after dequantizing the integers into floating-point numbers, the floating-point numbers can be pruned, and the pruned floating-point numbers can be used to replace some parameters in the block structure.
[0058] One approach to pruning floating-point numbers is to remove the smallest precision value by taking zeros from it. For example, if the smallest precision value is 0.0002, removing zeros from 0.0002 can reduce the number of model parameters to some extent. Adding quantization noise during training is also relatively easy to combine with pruning algorithms.
[0059] As for the remaining parameters in the block structure, under the perturbation brought to the model by the aforementioned partial parameter quantization, they can be adjusted and updated according to the loss function. Compared with the existing quantization-aware training methods, this application only performs partial QAT and updates the tensor that maintains floating-point precision. This ensures that the tensor that maintains floating-point precision is not affected by quantization error and can obtain accurate gradient calculation and its own parameter update, so that the adjusted model has better robustness and thus improves the inference accuracy of the model.
[0060] This application provides a data processing method that involves: acquiring a model to be compressed, the model comprising multiple network layers; acquiring the quantization bits corresponding to each network layer of the model to be compressed, the quantization bits being determined based on the simulated power of the network layer during forward inference; dividing the model to be compressed into block structures, each block structure comprising at least one of the network layers; quantizing some parameters in the block structure into integers corresponding to the quantization bits; dequantizing the integers into floating-point numbers and replacing some parameters in the block structure with the floating-point numbers; and adjusting and updating the remaining parameters in the block structure according to a loss function to ensure the inference accuracy of the model to be compressed. This method achieves a reduction in model size, thereby reducing memory usage and power consumption, compared to traditional quantization-aware training methods, by incorporating some quantization noise for optimization iterations, while the gradients of the remaining tensors can continue to propagate unbiasedly like a floating-point model.
[0061] By combining the quantization of reducing the number of parameter representation bits with the pruning that directly reduces the number of model parameters, we can actively explore extreme model compression while ensuring that the inference accuracy is within a reasonable range, thereby reducing the time spent on forward inference.
[0062] Please see Figure 3 Another embodiment of this application provides a data processing method, the method comprising:
[0063] Step S310: Obtain the target task model.
[0064] The target task can be a task in fields such as vision, speech, and / or natural language processing. For example, the target task can be a vision task, object detection, etc. The target task model is a model corresponding to the target task. For example, if the target task is object detection, then the target task model is an object detection model. The specific type of the target task and the specific type of the target task model are not limited.
[0065] One approach is to load an existing model corresponding to the target task as the target task model.
[0066] Step S320: Perform floating-point precision training on the target task model to obtain a floating-point precision model.
[0067] To better compress the model, the target task model can be trained with floating-point precision first to obtain a floating-point precision model. The specific process of floating-point precision training will not be described here.
[0068] Step S330: Use the floating-point precision model as the model to be compressed, the model to be compressed includes multiple network layers.
[0069] To ensure the inference accuracy of the model, the size of the floating-point precision model is usually large. In order to deploy the floating-point precision model on edge devices with limited memory resources, such as mobile phones, the floating-point precision model can be compressed first. That is, the floating-point precision model can be used as the model to be compressed, which can include multiple network layers.
[0070] Step S340: Obtain the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the analog power of the network layer during forward inference.
[0071] The specific implementation of step S340 can be referred to the relevant descriptions of steps S120 and S220 in the foregoing embodiments, and will not be repeated here.
[0072] Step S350: Divide the model to be compressed into a block structure, wherein the block structure includes at least one of the network layers.
[0073] The specific implementation of step S350 can be referred to the relevant description of step S130 in the foregoing embodiments, and will not be repeated here.
[0074] Step S360: Perform quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and adjust and update the remaining parameters in the block structure according to the loss function to ensure the inference accuracy of the model to be compressed.
[0075] The specific implementation of step S360 can be referred to the relevant descriptions of steps S140, S240 and S250 in the foregoing embodiments, and will not be repeated here.
[0076] This application provides a data processing method that involves: acquiring a target task model; performing floating-point precision training on the target task model to obtain a floating-point precision model; using the floating-point precision model as a model to be compressed, the model to be compressed comprising multiple network layers; acquiring the quantization bits corresponding to each network layer of the model to be compressed, the quantization bits being determined based on the simulated power of the network layer during forward inference; dividing the model to be compressed into block structures, each block structure comprising at least one of the network layers; performing quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and adjusting and updating the remaining parameters in the block structure according to a loss function to ensure the inference accuracy of the model to be compressed. This method achieves a reduction in model size, thereby reducing memory usage and power consumption, compared to traditional quantization-aware training methods, by incorporating some quantization noise for optimization iteration, while the gradients of the remaining tensors can continue to propagate unbiasedly like in a floating-point model.
[0077] Please see Figure 4 This application provides a data processing apparatus 400, which includes:
[0078] The model acquisition module 410 is used to acquire the model to be compressed, which includes multiple network layers.
[0079] In one approach, the model acquisition module 410 can be used to acquire a target task model; perform floating-point precision training on the target task model to obtain a floating-point precision model; and use the floating-point precision model as a model to be compressed.
[0080] The quantization bit acquisition module 420 is used to acquire the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the analog power of the network layer during forward inference.
[0081] The model partitioning module 430 is used to partition the model to be compressed into a block structure, wherein the block structure includes at least one of the network layers.
[0082] In one approach, the model partitioning module 430 can be used to partition the model to be compressed into a block-based structure based on the model structure characteristics of the model to be compressed, wherein the block structure includes at least one of the network layers.
[0083] The model compression module 440 is used to perform quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and to adjust and update the remaining parameters in the block structure according to the loss function to ensure the inference accuracy of the model to be compressed.
[0084] In one approach, the model compression module 440 can be used to quantize some parameters in the block structure into integers of the corresponding quantization bits; dequantize the integers into floating-point numbers, and replace some parameters in the block structure with the floating-point numbers. Specifically, quantizing some parameters in the block structure into integers of the corresponding quantization bits can include: progressively quantizing some parameters in the block structure into integers of the corresponding quantization bits. Dequantizing the integers into floating-point numbers and replacing some parameters in the block structure with the floating-point numbers can include: dequantizing the integers into floating-point numbers; pruning the floating-point numbers; and replacing some parameters in the block structure with the pruned floating-point numbers.
[0085] As one implementation method, the floating-point number can be pruned by taking zeros from the floating-point number with the smallest floating-point precision.
[0086] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0087] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.
[0088] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.
[0089] Please see Figure 5Based on the aforementioned data processing methods and apparatus, this application also provides an electronic device 100 capable of executing the aforementioned data processing methods and apparatus. In this application embodiment, the electronic device 100 can be a client or a server. The electronic device 100 includes a memory 102 and one or more (only one is shown in the figure) processors 104 coupled to each other, with a communication line connecting the memory 102 and the processors 104. The memory 102 stores programs capable of executing the contents of the aforementioned embodiments, and the processors 104 can execute the programs stored in the memory 102.
[0090] The processor 104 may include one or more processing cores. The processor 104 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 102, and by calling data stored in the memory 102. Optionally, the processor 104 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 104 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 104 and may be implemented separately using a communication chip.
[0091] The memory 102 may include random access memory (RAM) or read-only memory (ROM). The memory 102 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the aforementioned embodiments. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).
[0092] Please refer to Figure 6 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 500 stores program code that can be called by a processor to execute the methods described in the above method embodiments.
[0093] The computer-readable storage medium 500 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 510 may be compressed, for example, in a suitable form.
[0094] In summary, the data processing method, apparatus, electronic device, and storage medium provided in this application involve: acquiring a model to be compressed, which includes multiple network layers; acquiring the quantization bits corresponding to each network layer of the model to be compressed, wherein the quantization bits are determined based on the simulated power of the network layer during forward inference; dividing the model to be compressed into block structures, each block structure including at least one of the network layers; performing quantization-aware training on some parameters in the block structure based on the corresponding quantization bits; and adjusting and updating the remaining parameters in the block structure according to a loss function to ensure the inference accuracy of the model to be compressed. Thus, compared to traditional quantization-aware training methods, this approach incorporates some quantization noise for optimization iteration, while the gradients of the remaining tensors can continue to propagate unbiasedly like a floating-point model. This allows for a reduction in model size while maintaining reasonable inference accuracy, thereby reducing memory usage and power consumption.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A data processing method, characterized in that, Applied to electronic devices, the method includes: Obtain the target task model, wherein the target task is a visual processing task, a speech processing task, a natural language processing task, or an object detection task. The target task model is trained with floating-point precision to obtain a floating-point precision model. The floating-point precision model is used as the model to be compressed, and the model to be compressed includes multiple network layers; The quantization bits corresponding to each network layer of the model to be compressed are obtained. The quantization bits are determined based on the analog power of the network layer during forward inference. The quantization bits corresponding to each network layer are different. The process of obtaining the quantization bits includes: constructing a linear programming problem using a hardware simulator with the sensitivity information of each network layer in the model to be compressed during forward inference, in order to solve the problem of minimizing the sum of perturbation information under the constraint of the total power of the network to be compressed, so as to obtain the quantization bits of each network layer. The total power is the sum of the analog power corresponding to each network layer. The perturbation information includes the degree of information change in the process of quantizing the parameters from floating point to fixed point. The sensitivity information is the analog power. The model to be compressed is divided into a block structure, and the block structure includes at least one of the network layers; Based on the corresponding quantization bits, quantization-aware training is performed on some parameters in the block structure, and the remaining parameters in the block structure are adjusted and updated according to the loss function to ensure the inference accuracy of the model to be compressed.
2. The method according to claim 1, characterized in that, The step of dividing the model to be compressed into a block structure includes: Based on the structural characteristics of the model to be compressed, the model to be compressed is divided into a block-based structure.
3. The method according to claim 1, characterized in that, The step of performing quantization-aware training on some parameters in the block structure based on the corresponding quantization bits includes: Quantize some parameters in the block structure into integers of the corresponding quantized bits; The integer is dequantized into a floating-point number, and the floating-point number is used to replace some parameters in the block structure.
4. The method according to claim 3, characterized in that, The step of quantizing a portion of the parameters in the block structure into integers of the corresponding quantized bits includes: Some parameters in the block structure are progressively quantized into integers of the corresponding quantization bits.
5. The method according to claim 3, characterized in that, The step of dequantizing the integer into a floating-point number and replacing some parameters in the block structure with the floating-point number includes: The integer is dequantized into a floating-point number; The floating-point number is pruned, and some parameters in the block structure are replaced with the pruned floating-point number.
6. The method according to claim 5, characterized in that, The pruning process for the floating-point numbers includes: Remove zeros from the floating-point number with the smallest precision value.
7. A data processing apparatus, characterized in that, Operating in an electronic device, the device includes: The model acquisition module is used to acquire a target task model, wherein the target task is a visual processing task, a speech processing task, a natural language processing task, or an object detection task; to train the target task model with floating-point precision to obtain a floating-point precision model; and to use the floating-point precision model as a model to be compressed, wherein the model to be compressed includes multiple network layers. A quantization bit acquisition module is used to acquire the quantization bits corresponding to each network layer of the model to be compressed. The quantization bits are determined based on the analog power of the network layer during forward inference. Each network layer has a different quantization bit. The process of acquiring the quantization bits includes: using a hardware simulator to construct a linear programming problem based on the sensitivity information of each network layer in the model to be compressed during forward inference, to solve for minimizing the sum of perturbation information under the constraint of the total power of the network to be compressed, so as to obtain the quantization bits of each network layer. The total power is the sum of the analog power corresponding to each network layer. The perturbation information includes the degree of information change during the process of quantizing the parameters from floating point to fixed point. The sensitivity information is the analog power. A model partitioning module is used to partition the model to be compressed into a block structure, wherein the block structure includes at least one of the network layers; The model compression module is used to perform quantization-aware training on some parameters in the block structure based on the corresponding quantization bits, and to adjust and update the remaining parameters in the block structure according to the loss function to ensure the inference accuracy of the model to be compressed.
8. An electronic device, characterized in that, Includes one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-6.