Model training method and system based on gradient sensitivity and hardware perception, storage medium and computer device

Abstract

The application discloses a model training method and system based on gradient sensitivity and hardware perception, a storage medium and a computer device. The method comprises the following steps: training a target model based on a preset learning rate decay rule; during the training process of each training step, the following operations are performed: determining a bit width reduction speed corresponding to the current training step according to a learning rate decay rate corresponding to the current training step and a preset bit width reduction coefficient, and determining a bit width reduction quantity corresponding to the current training step according to the bit width reduction speed; determining a target layer in the target model and reducing the bit width of the target layer according to the gradient sensitivity of each layer in the target model, the current hardware device parameter and the bit width reduction quantity, and taking a preset minimum bit width as a constraint, so as to compress the target model; the preset minimum bit width represents a minimum value of the settable bit width of each layer in the target model; and training the compressed target model in the current training step. The effective compression is realized under the premise of maintaining high performance. The model training can be applied to the medical health and financial technology scenes.

Description

Technical Field

[0001] This application relates to the field of model training technology, and in particular to a model training method, system, storage medium, and computer device based on gradient sensitivity and hardware awareness. Background Technology

[0002] With the widespread application of deep learning models in key areas such as healthcare and fintech, the scale of these models continues to grow, placing higher demands on computing resources and storage space. For example, in medical image diagnosis, cancer screening models based on deep neural networks often contain hundreds of millions of parameters. Directly deploying them on edge devices or cloud inference platforms results in high latency and low throughput, making it difficult to meet real-time requirements. In financial risk control scenarios, high-frequency trading models need to complete risk assessment within milliseconds, but the inference overhead of large models often becomes a system bottleneck. Therefore, model compression technology, especially mixed precision quantization, has become a core means of efficiently deploying models on resource-constrained hardware environments.

[0003] In existing technologies, the most common model quantization method is post-training quantization. Post-training quantization typically assigns a fixed bit width (e.g., using a uniform 8-bit integer) to each layer after model training, based on the statistical distribution of weights and activation values. Some improved schemes introduce dynamic tree quantization or static bit width allocation based on gradient sensitivity, that is, determining the quantization accuracy of each layer before or in the early stages of training and keeping it unchanged in subsequent processes. However, fixed or static bit width allocation ignores the differences in gradient sensitivity between layers. Using the same bit width for high gradient-sensitive network layers as for low gradient-sensitive network layers can lead to a significant decrease in model accuracy.

[0004] In summary, how to dynamically and hardware-awarely determine the quantization bit width of each layer during model training, while effectively compressing the model data volume and maintaining model accuracy, so that the model can be adapted to resource-constrained hardware devices, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of this, embodiments of this application provide a model training method, system, storage medium, and computer device based on gradient sensitivity and hardware awareness.

[0006] According to one aspect of this application, a model training method based on gradient sensitivity and hardware awareness is provided, the method comprising: The target model is trained in each training step based on a preset learning rate decay rule. The following operations are performed during the training process of each training step: Based on the learning rate decay rate and the preset bit width decay coefficient corresponding to the current training step, determine the bit width decay rate corresponding to the current training step, and determine the bit width decay amount corresponding to the current training step based on the bit width decay rate. Based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width reduction amount, the target layer in the target model is determined and its bit width is reduced to compress the target model, with a preset minimum bit width as a constraint; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model. Train the compressed target model for the current training step.

[0007] According to another aspect of this application, a model training system based on gradient sensitivity and hardware awareness is provided, the system comprising: a server and a client; The server is configured to train the target model for each training step based on a preset learning rate decay rule. During the training process of each training step, the server performs the following operations: determining the bit width decay rate corresponding to the current training step based on the learning rate decay rate and the preset bit width decay coefficient, and determining the bit width decay amount corresponding to the current training step based on the bit width decay rate; determining the target layer in the target model and reducing its bit width based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width decay amount, with a preset minimum bit width as a constraint, to compress the target model; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model; and training the compressed target model for the current training step. The client is used to receive the target model trained by the server and deploy it locally.

[0008] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described model training method based on gradient sensitivity and hardware awareness.

[0009] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described model training method based on gradient sensitivity and hardware awareness.

[0010] By employing the above technical solutions, this application provides a model training method, system, storage medium, and computer device based on gradient sensitivity and hardware awareness. By coupling and controlling the learning rate decay rate with the bit width decrease rate, the model compression process is synchronized with the training convergence process, avoiding model accuracy collapse caused by sudden bit width changes. Through dynamic bit width allocation combining gradient sensitivity and hardware device parameters, the compression strategy prioritizes the protection of critical network layers while responding in real-time to hardware resource constraints, improving training stability and resource utilization. By continuing quantization-aware training on the compressed model, the model can gradually adapt to mixed-precision representation, ultimately achieving effective compression while maintaining high performance. This method is particularly suitable for applications such as healthcare and fintech, which have stringent requirements for both model accuracy and real-time inference.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This illustration shows a flowchart of a model training method based on gradient sensitivity and hardware awareness provided in an embodiment of this application. Figure 2 This illustration shows a flowchart of another model training method based on gradient sensitivity and hardware awareness provided in an embodiment of this application. Figure 3 This illustration shows a schematic diagram of the structure of a model training system based on gradient sensitivity and hardware awareness provided in an embodiment of this application; Figure 4 A schematic diagram of the system structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0013] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0014] This embodiment provides a model training method based on gradient sensitivity and hardware awareness, such as... Figure 1As shown, the method includes: training the target model in each training step based on a preset learning rate decay rule, and performing the following operations during the training process of each training step: Step 101: Determine the bit width decay rate for the current training step based on the learning rate decay rate and the preset bit width decay coefficient, and determine the bit width decay amount for the current training step based on the bit width decay rate.

[0015] Here, the learning rate decay rate refers to how quickly the learning rate changes relative to the previous training step during model training; the bit width decrease rate refers to how quickly the overall bit width of the model decreases within each training step; the preset bit width decrease coefficient is a fixed proportional constant used to map the learning rate decay rate to the corresponding bit width decrease rate; and the bit width decrease amount refers to the total number of bits that the network layer bit width in the model should be reduced within this step, calculated based on the bit width decrease rate of the current step.

[0016] Specifically, a learning rate decay rule is first set before training begins, typically using exponential decay, so that the learning rate gradually decreases as the number of training steps increases. At the beginning of each training step, the learning rate decay rate for the current step is calculated based on the learning rate values ​​of the current step and the previous step. Then, a pre-defined bit-width reduction coefficient is obtained, and the learning rate decay rate is multiplied by this coefficient to obtain the bit-width reduction speed for the current step. This speed represents how quickly the overall bit width of the model should decrease under the current training environment. Further, based on the bit-width reduction speed and the time step size (each training step corresponds to a unit of time), the amount of bit-width reduction that should be achieved in the current step is calculated. This bit-width reduction amount may be a decimal; in this case, an error accumulation method can be used, accumulating the decimal part to the next step, and performing the actual bit-width reduction operation only when the accumulated value reaches an integer value.

[0017] Step 102: Based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width reduction amount, determine the target layer in the target model and reduce its bit width to compress the target model, with a preset minimum bit width as a constraint; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model.

[0018] Among them, gradient sensitivity refers to the sensitivity of each layer of the model to quantization error, which is obtained by statistical analysis of the absolute mean of the gradient of the weights within the layer; hardware device parameters include the remaining capacity of the current graphics processor's memory and the data bus bandwidth utilization rate; minimum bit width refers to the minimum number of quantization bits allowed for each layer of the model, usually set to 8 bits, which will lead to unacceptable accuracy loss; target layer refers to one or more network layers selected for bit width reduction in the current training step.

[0019] Specifically, firstly, the number of bit width reductions determined in step 101 is obtained and recorded as the total number of bits to be reduced. Then, considering the gradient sensitivity of each layer in the target model, the current hardware parameters, and the number of bit width reductions, the target layer is searched among the network layers of the target model with the minimum bit width as a constraint, and the bit width of the target layer is reduced. For example, the current bit width value of each layer of the model is read, and the layers are sorted from low to high according to their gradient sensitivity. Next, the real-time parameters of the current hardware device are obtained: whether the remaining memory capacity is lower than the safety threshold and whether the bandwidth utilization rate is higher than the congestion threshold. When the remaining memory capacity is lower than the safety threshold, the system forcibly reduces the bit width of all layers in the model by one bit, not counting it in the total number of bits to be reduced, as an emergency safety measure. When the bandwidth utilization rate is higher than the congestion threshold, the system prioritizes the layer with the lowest gradient sensitivity, and reduces the bit width of that layer by 1 bit each time, repeating the process until the total number of bits to be reduced is exhausted, or all layers are reduced to the minimum bit width. For example, in a medical image diagnosis model, shallow convolutional feature extraction layers usually have high gradient sensitivity, while deep classification layers have low sensitivity. When bandwidth is congested, the bit width of deep classification layers is compressed first, while the high precision of shallow layers is preserved, ensuring that the edge features of lesions are not destroyed. In financial risk control models, recurrent layers processing temporal features are highly sensitive, while the final fully connected output layer is less sensitive. Similarly, the bit width of the output layer can be compressed first to alleviate bus pressure. By co-determining gradient sensitivity ranking with hardware status, bit width compression can both prioritize sacrificing less sensitive layers to protect the model's key expressive capabilities and respond in real time to hardware resource constraints, avoiding training interruptions caused by memory overflow or bandwidth overload.

[0020] In one optional embodiment, determining the bit width reduction amount corresponding to the current training step based on the bit width reduction rate includes: accumulating the bit width reduction value of the bit width reduction variable for one training step according to the bit width reduction rate; if the bit width reduction variable is greater than or equal to 1, determining the integer part of the bit width reduction variable as the bit width reduction amount corresponding to the current training step, and updating the difference between the bit width reduction variable and the bit width reduction amount as a new bit width reduction variable; if the bit width reduction variable is less than 1, determining the target layer in the target model according to the gradient sensitivity of each layer in the target model and the current hardware device parameters, and reducing the bit width of the target layer with the minimum bit width as a constraint.

[0021] The bit width decrease variable is an accumulated counter used to record the bit width decrease amount that has not yet been triggered in multiple training steps, and its initial value is zero; the bit width decrease value is the theoretical bit width reduction amount of the step calculated based on the bit width decrease rate and the unit step time, which is usually a decimal less than 1; the integer part is the integer value greater than or equal to 1 in the bit width decrease variable, representing the actual number of bit width reduction bits that should be executed in the step.

[0022] In this embodiment, at the beginning of each training step, the learning rate decay rate of the current step is first multiplied by a preset bit-width reduction coefficient to obtain a bit-width reduction rate. Then, this bit-width reduction rate is multiplied by the unit step time (usually considered as one time unit) to calculate the bit-width reduction value corresponding to that step, which is typically a decimal less than 1. Next, this bit-width reduction value is accumulated with the currently stored bit-width reduction variable to update the bit-width reduction variable. Then, the accumulated bit-width reduction variable is checked. If the bit-width reduction variable is greater than or equal to 1, the integer part of the variable is taken as the bit-width reduction amount corresponding to the current training step. Simultaneously, the original variable is subtracted from the integer part, and the remaining decimal part is stored as a new bit-width reduction variable, to be accumulated in subsequent training steps. For example, if the accumulated bit-width reduction variable becomes 1.3, the bit-width reduction amount is 1, and the new bit-width reduction variable is 0.3. If the bit-width reduction variable is still less than 1, it is determined that the current step has not accumulated enough total reduction amount, and the bit-width reduction operation is not performed in this step. At this point, still sorting by gradient sensitivity of each layer of the model and considering current hardware parameters (such as remaining memory capacity and bandwidth utilization), and under the premise of satisfying the minimum bit width constraint, one or more layers with the lowest sensitivity are selected, and their bit width is reduced by one bit. This operation is independent of the cumulative counter, enabling real-time response when hardware resources are scarce, avoiding missing the emergency degradation opportunity by waiting for the accumulated bit width reduction. Thus, through the dual mechanism of accumulated error and conditional triggering in this embodiment, the bit width reduction operation can maintain long-term synchronization with the learning rate decay rate (achieving smooth bit-by-bit reduction through the accumulation of the fractional part), and can bypass the cumulative counter to directly perform the degradation operation when hardware resources are scarce, ensuring the stability of training and resource security.

[0023] In one optional embodiment, the current hardware device parameters include the current video memory utilization rate and the current bandwidth congestion level; the step of determining the target layer in the target model and reducing its bit width based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width reduction amount, with the minimum bit width as a constraint, includes: Identify whether the current training step is a hardware-aware training step, wherein the hardware-aware training step is determined based on a preset training step interval or a preset time interval; If the current training step is not the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the bit width reduction amount, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced; wherein, the minimum bit width constraint includes a first constraint and a second constraint, the first constraint means that the bit width of the target layer after reduction is not less than the minimum bit width corresponding to the target layer, and the second constraint means that the target layer is the layer with the widest bit width among all layers in the target model that satisfy the first constraint; the minimum bit width of the gradient-sensitive layer of the target model is the first minimum bit width, the minimum bit width of the non-gradient-sensitive layer of the target model is the second minimum bit width, the first minimum bit width is greater than the second minimum bit width, and the gradient sensitivity of the gradient-sensitive layer is greater than a preset sensitivity; If the current training step is the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the number of bit width reductions, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced. When the current memory occupancy rate is greater than the preset occupancy rate, the bit width of all layers is reduced by one bit with the first constraint as a constraint. When the current bandwidth congestion level is greater than the preset congestion level, the non-gradient-sensitive layer is reduced by one bit with the first constraint as a constraint.

[0024] Among them, the hardware-aware training step refers to a training step specifically used to collect and respond to the hardware device status (such as memory usage and bandwidth congestion) according to a preset training step interval or preset time interval; the first constraint means that the bit width of the target layer after reduction must not be lower than the minimum bit width corresponding to that layer, ensuring that each layer retains the most basic accuracy lower limit during compression; the second constraint means that among all layers that satisfy the first constraint, the layer with the widest current bit width is selected first for reduction, so that the compression operation is evenly distributed and high bit width layers are compressed first; the first minimum bit width refers to the minimum number of bits allowed for gradient-sensitive layers (gradient sensitivity is higher than the preset sensitivity), which is relatively high to protect the feature expression capability of key layers; the second minimum bit width refers to the minimum number of bits allowed for non-gradient-sensitive layers, which is relatively low to allow for greater compression; the preset memory usage refers to the warning threshold for memory usage; the preset congestion level refers to the warning threshold for bandwidth congestion.

[0025] In this embodiment, the current training step is first identified as a hardware-aware training step. This identification is based on a pre-configured training step interval (e.g., executed once every fixed number of steps) or time interval (e.g., executed once every fixed duration). If the current training step is not a hardware-aware training step, compression is performed solely based on gradient sensitivity and bit width reduction. Specifically, the bit width reduction amount determined in step 101 is obtained. Then, all layers of the model are traversed, and all layers that satisfy the first constraint (the reduced bit width is not lower than the minimum bit width corresponding to that layer) are selected. Among these candidate layers, the layer with the widest current bit width is further selected as the target layer according to the second constraint. If multiple layers have the same bit width and are all the widest, the layer with the lowest gradient sensitivity is selected for priority reduction. Each time a target layer is selected, its bit width is reduced by 1 bit, and the bit width reduction amount is reduced by 1. The above process is repeated until the bit width reduction amount is exhausted or there are no selectable layers. During this process, the first minimum bit width corresponding to the gradient-sensitive layer is significantly higher than the second minimum bit width of the non-gradient-sensitive layer. For example, in a medical image diagnostic model, the convolutional layer responsible for extracting lesion edges is labeled as a gradient-sensitive layer, with a minimum bit width of 12 bits; while the non-sensitive layer responsible for background texture has a minimum bit width of only 8 bits. When it is necessary to reduce the bit width, the layer with the widest current bit width that meets the minimum bit width constraint will be selected first, that is, the non-sensitive layer that has not yet reached the lower limit or the higher bit width among the sensitive layers.

[0026] If the current training step is a hardware-aware training step, then after completing the selective compression based on gradient sensitivity and bit width reduction, an additional hardware-state-driven forced compression is applied. This includes two additional operations: First, the current memory occupancy rate is read. When the memory occupancy rate is detected to be greater than the preset occupancy rate (i.e., memory is close to saturation), a forced bit width reduction is performed on all layers of the model, reducing the bit width by 1 bit for each layer, but this must satisfy the first constraint (the reduced bit width must not be lower than the minimum bit width corresponding to each layer). If a layer has already reached its minimum bit width, that layer is skipped. Second, the current bandwidth congestion level is read. When the bandwidth congestion level is detected to be greater than the preset congestion level (i.e., severe data bus congestion), forced bit width reduction is only performed on non-gradient-sensitive layers, reducing the bit width by 1 bit for each layer, again with the first constraint as the lower limit. Gradient-sensitive layers are unaffected in this case to protect their critical feature transmission. For example, in a financial risk control model, the recurrent layer processing time-series transaction data is a gradient-sensitive layer (minimum bit width 12 bits), while the fully connected layer that ultimately outputs the fraud probability is a non-sensitive layer (minimum bit width 8 bits). When the hardware-aware training step detects excessive memory usage, it simultaneously reduces the bit width of the recurrent layer and the fully connected layer (if the lower limit is not reached) to quickly release memory. When only bandwidth congestion is detected, it only reduces the bit width of the fully connected layer, because this layer has a large data transmission volume and is not sensitive to precision, while the bit width of the recurrent layer remains unchanged to ensure that the model's ability to recognize patterns in transaction sequences is not affected.

[0027] The hardware-aware and non-aware dual-mode design in this embodiment enables the model compression process to maintain a stable, progressive reduction in bit width (non-aware steps) in most training steps, while intervening in hardware status monitoring at a lower frequency (hardware-aware steps) to respond promptly to resource constraints. The scenario-specific forced compression strategy (globally reducing to address memory pressure, and selectively reducing non-sensitive layers to address bandwidth pressure) makes the compression action more precise and avoids over-compression of sensitive layers.

[0028] Step 103: Train the compressed target model for the current training step.

[0029] The compressed target model refers to the target model after reducing the bit width of some layers in step 102. Specifically, the compressed target model output from step 102 is loaded into the current training process, using the same batch of input data as the previous training step, and forward propagation, loss calculation, and backpropagation operations are continued. For example, in medical image segmentation tasks, when the model is compressed and retrained, it relearns how to accurately identify lesion contours with low bit width representation. Through continuous training in this step, the model can gradually adapt to different quantization accuracies of each layer, eventually achieving diagnostic performance close to that of a full-accuracy model. In financial transaction fraud detection tasks, the compressed model needs to maintain high sensitivity to abnormal traffic. This step uses quantization-aware training to enable the model to distinguish between normal and fraudulent transactions even with low accuracy. Through continuous training after compression in this step, the model maintains a low bit width representation while its loss function converges to a lower level, avoiding the problem of significant accuracy drops in the traditional "train first, compress later" model, and ensuring the usability of the compressed model in actual deployment.

[0030] By applying the technical solution of this embodiment, the learning rate decay rate and bit width descent rate are coupled and controlled, synchronizing the model compression process with the training convergence process and avoiding model accuracy collapse caused by sudden bit width changes. Dynamic bit width allocation, combining gradient sensitivity with hardware parameters, prioritizes the protection of critical network layers in the compression strategy while responding in real-time to hardware resource constraints, improving training stability and resource utilization. Further quantization-aware training of the compressed model allows it to gradually adapt to mixed-precision representation, ultimately achieving effective compression while maintaining high performance. This method is particularly suitable for applications such as healthcare and fintech, which have stringent requirements for both model accuracy and real-time inference.

[0031] This application has at least the following beneficial effects: 1. By dynamically controlling the bit width decay rate according to the learning rate decay rate during training, the quantization process is coordinated with the model training process. The quantization pace is accelerated when the model tends to converge, and the quantization pace is slowed down in the early stage of model learning, avoiding the defect of the fixed bit width scheme in traditional methods that cannot adapt to the training process.

[0032] 2. By acquiring the current hardware device parameters (including the minimum precision bit width supported by the hardware and the calculation latency at each precision bit width), the hardware characteristics are incorporated into the decision-making basis for bit width allocation, so that the quantized model can fully adapt to the computing power of the target hardware, reduce format conversion overhead, and improve hardware execution efficiency.

[0033] 3. By calculating the gradient sensitivity of each layer, we can identify network layers with different sensitivities to quantization errors. We can prioritize reducing the bit width of layers with low sensitivity, thereby compressing the model data while maximizing the model accuracy.

[0034] 4. The bit width allocation decision is made dynamically during the training process, eliminating the need to repeatedly evaluate the model after quantization to determine the optimal bit width, thus reducing computational complexity.

[0035] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another model training method based on gradient sensitivity and hardware awareness is provided, such as... Figure 2 As shown, training the compressed target model for the current training step includes: Step 1031: Input the training samples of the current training step into the compressed target model, and obtain the quantization error corresponding to each weight of the target layer during the forward propagation process.

[0036] Training samples refer to data instances used for model training. In the healthcare field, they can be represented as medical images or electronic medical records, while in the fintech field, they can be represented as transaction flow sequences or customer behavior feature vectors. Quantization error refers to the difference tensor between the original floating-point weights and the weights obtained by quantization and dequantization at the current bit width of each layer, reflecting the information lost due to reduced precision.

[0037] In this step, a batch of training samples is first read from the training dataset. For medical image diagnosis tasks, this sample can be a computed tomography image containing lung nodules; for financial fraud detection tasks, this sample can be a set of payment records containing fields such as transaction time, amount, and merchant category. This sample is then input into the compressed target model, and forward propagation computation begins. When the computation flows through the target layer with reduced bit width, the following operations are performed within that layer: the current floating-point weight tensor of the target layer is obtained; the floating-point weight tensor is sequentially quantized and dequantized to quantize it into integers and then dequantize it back to floating-point numbers; the floating-point weight tensor and the dequantization result are calculated as the initial quantization error; L2 normalization is performed on the initial quantization error to obtain the quantization error. The shape of this error tensor is completely consistent with the shape of the layer weight tensor, containing spatial structure information such as height, width (for convolutional layers) or feature dimension (for fully connected layers). Specifically, a copy of the currently maintained floating-point weight tensor is first read from the memory region of the target layer. This copy is continuously updated during training, always maintaining full precision, and is used for gradient calculation and weight updates. Then, based on the current bit width allocated to the layer (e.g., 12-bit, 10-bit, or 8-bit during progressive compression), quantization is performed on the floating-point weight tensor: determining the minimum and maximum values ​​of the layer's weights, linearly mapping the floating-point range to 2 power-of-bit-width integer intervals, and rounding each floating-point value to the nearest integer. After quantization, dequantization is performed: the integer values ​​are restored to floating-point numbers using the same linear mapping, yielding the dequantized result. This dequantized result has the exact same shape as the original floating-point weight tensor, but its numerical precision is significantly reduced. Element-by-element subtraction between the original floating-point weight tensor and the dequantized result yields the initial quantization error tensor. This error tensor reflects the specific value lost by each weight parameter at the current bit width. Next, the system performs L2 normalization on the initial quantization error tensor: calculating the square root of the sum of squares of all elements in the tensor, and then dividing each element in the tensor by this square root to obtain the normalized quantization error. The overall norm of the standardized error is 1, in order to eliminate the problem of inconsistent error magnitudes caused by differences in the range of weight values ​​between different layers.

[0038] It should be noted that when an abnormally small L2 norm of an initial quantization error tensor is detected (e.g., less than one-thousandth of the historical average), the L2 normalization step is automatically skipped, the initial quantization error is directly treated as a zero tensor, and the compensation process for the current layer is terminated early to save computational resources. This is because an extremely small error indicates that the current bit width has almost no impact on the layer, and no additional compensation is required.

[0039] Step 1032: Generate the compensation tensor corresponding to the quantization error through a bidirectional long short-term memory network.

[0040] Among them, the bidirectional long short-term memory network is a variant of the recurrent neural network, which contains two independent memory units in the forward propagation direction (processing the sequence from front to back) and the backward propagation direction (processing the sequence from back to front); the compensation tensor is a tensor with the same spatial size as the original error tensor but with a different number of channels, generated by the network after analyzing the quantization error, which is used to guide subsequent error repair; each memory unit contains a forget gate, an input gate and an output gate, which can selectively remember or forget information in the sequence.

[0041] For example, the quantization error is a three-dimensional matrix with the height, width, and number of channels of the target layer as dimensions; generating the compensation tensor corresponding to the quantization error through a bidirectional long short-term memory network includes: determining the length of a two-dimensional feature vector based on the product of the height and the width; determining the correspondence between each element in the two-dimensional feature vector and the height and the width; mapping the data in the quantization error to the two-dimensional feature vector based on the correspondence; inputting the two-dimensional feature vector into the bidirectional long short-term memory network to obtain the initial compensation tensor corresponding to the two-dimensional feature vector, wherein the initial compensation tensor is a two-dimensional matrix with the length of the two-dimensional feature vector and twice the number of hidden units as dimensions; and restoring the initial compensation tensor to a three-dimensional matrix with the height, the width, and twice the number of hidden units as dimensions based on the correspondence between each element in the initial compensation tensor and the corresponding correspondence, to obtain the compensation tensor corresponding to the quantization error.

[0042] First, a 3D quantization error matrix is ​​obtained, with the shape of height × width × number of channels. The product of height and width is calculated to obtain the length of the 2D feature vector, denoted as the sequence length. Then, the correspondence between each position index in this 2D feature vector and the (height coordinates, width coordinates) in the original 3D matrix is ​​established: typically, row-major order is used, i.e., all height rows are traversed first, and within each row, the width columns are traversed from left to right, arranging the channel dimension vectors (length equal to the number of channels) at each spatial position sequentially to form a 2D feature vector with a length equal to the sequence length, where each element is itself a channel dimension vector. This 2D feature vector (shape equal to sequence length × number of channels) is input into a bidirectional long short-term memory network. The forward branch of this network reads the channel dimension vectors at each time step in ascending order of index, outputting a hidden state vector at each time step with a dimension equal to the number of hidden units; the backward branch reads the vectors in descending order of index, similarly outputting a hidden state vector with a dimension equal to the number of hidden units. The output vectors of the two branches at each time step are concatenated along the channel dimension to obtain the initial compensation vector for that time step, with a dimension twice the number of hidden units. After traversing all time steps, a two-dimensional matrix is ​​obtained, with a shape of sequence length × twice the number of hidden units; this is the initial compensation tensor. For example, in a medical image diagnostic model, the quantization error feature map of a certain convolutional layer has a size of 64×64×128 (height 64, width 64, number of channels 128). The calculated sequence length is 4096 (64×64). The 4096 spatial locations are flattened in row-major order, and the 128-dimensional error vectors at each location are arranged sequentially to form a 4096×128 input matrix. The forward branch of the bidirectional LSTM reads vectors 1 to 4096, and the backward branch reads vectors 4096 to 1. Each branch has 256 hidden units, so the output dimension after concatenation at each time step is 512. The final initial compensation tensor has a shape of 4096×512.

[0043] Next, based on the previously established correspondence, the initial compensation tensor is restored to a three-dimensional matrix. Specifically, for each row index (corresponding to a spatial location) in the initial compensation tensor, its height and width coordinates in the original feature map are calculated based on the row index, and the 512-dimensional vector on that row is placed in the channel dimension of the corresponding coordinates in the three-dimensional matrix. After all rows have been processed, a three-dimensional matrix with a shape of height × width × 512 is obtained, which is the final compensation tensor.

[0044] Through spatial flattening and bidirectional temporal modeling in this step, the original two-dimensional feature map, which lacked sequential relationships, is transformed into data with a sequential structure. This allows the long short-term memory network's context modeling capability to capture the long-range spatial dependencies of quantization errors. The combination of forward and backward branches ensures that each spatial location receives information from its upper and left sides, as well as from its lower and right sides, making the generated compensation tensor contain complete context-aware capabilities. The restoration operation maintains the same spatial dimensions as the input, facilitating subsequent channel concatenation with the original weights.

[0045] Step 1033: Perform channel concatenation on the quantization error and the compensation tensor to obtain a concatenation matrix, and perform convolution calculation on the concatenation vector to restore the concatenation vector to an error compensation matrix that matches the number of channels of the target layer.

[0046] Channel concatenation refers to merging two tensors with the same shape, height, and width but different numbers of channels into a new tensor along the channel dimension; convolution calculation involves sliding a 1×1 convolution kernel on the concatenation matrix, performing a weighted summation of all input channels at each position to obtain a value for one output channel; the error compensation matrix refers to the output tensor after 1×1 convolution dimensionality reduction, which has the same number of channels as the original target layer, and can be directly used to correct the original weights.

[0047] In this step, the quantized weight tensor (i.e., the compressed target layer weights, with a shape of height x width x original number of channels) and the compensation tensor (with a shape of height x width x 512) are obtained. A channel concatenation operation is performed on these two tensors: each channel of the quantized weight tensor is arranged side-by-side with all 512 channels of the compensation tensor, generating a new concatenation matrix with a shape of height x width x (original number of channels plus 512). Next, a 1×1 convolutional layer is constructed. The number of input channels of this convolutional layer is equal to the number of channels in the concatenation matrix (original number of channels plus 512), and the number of output channels is equal to the original number of channels (the same as the number of channels in the quantized weight tensor). This convolutional layer contains a trainable weight matrix with a shape of output channel number x input channel number, and an optional bias vector. The system applies this 1×1 convolutional layer to each spatial location on the concatenation matrix: for position (i,j), a feature vector of length (original number of channels plus 512) is extracted from that position and multiplied by the weight matrix of the convolutional layer to obtain an output vector of length equal to the original number of channels. After repeating this calculation process across all spatial locations, a new tensor with a shape equal to height multiplied by width multiplied by the original number of channels is obtained; this is the error compensation matrix. This matrix is ​​essentially a pointwise linear transformation of the quantized weight tensor, incorporating the original quantized weight information and the contextual repair information provided by the compensation tensor. For example, in a financial risk control model, the weight matrix of a fully connected layer can be considered as a 1×1×feature dimension (for simplicity, it can be considered a one-dimensional vector). Its quantized weight tensor is 1×1×128, and its compensation tensor is 1×1×512; concatenating them yields 1×1×640. A 1×1 convolution maps this 640-dimensional vector back to 128 dimensions, obtaining the error compensation matrix. Each element in this matrix is ​​a weighted combination of the original quantized weights and the repair information.

[0048] Step 1034: Perform weight compensation on the target layer according to the error compensation matrix to complete the training of the current training step of the compressed target model.

[0049] For example, the step of concatenating the quantization error and the compensation tensor to obtain a concatenated matrix, and then performing convolution calculation on the concatenated matrix to restore it to an error compensation matrix matching the number of channels in the target layer, includes: concatenating the quantization error and the compensation tensor into a three-dimensional matrix with the height, width, and accumulated channel number of the target layer as dimensions, to obtain the concatenated matrix; wherein the accumulated channel number is the sum of the number of channels and the number of double hidden units; and performing convolution calculation on the concatenated matrix to restore it to a three-dimensional matrix with the height, width, and number of channels in the target layer, to obtain the error compensation matrix.

[0050] In this step, the calculated error compensation matrix is ​​used as the new weights for the target layer, replacing the original quantized weights. The specific replacement method depends on the training stage: during forward propagation, the error compensation matrix is ​​directly used in the convolution or matrix multiplication operations of this layer; during backpropagation, the gradient relative to the error compensation matrix is ​​calculated, and the gradient is further propagated back to the parameters of the compensation network (bidirectional long short-term memory network and 1×1 convolutional layer), thereby achieving end-to-end joint training. For example, in medical image segmentation tasks, the weight-compensated convolutional layer can extract lesion edge features more accurately because the compensation matrix compensates for the loss of high-frequency edge information under low bit width. In financial time series prediction tasks, the compensated recurrent layer can better remember long-term trading patterns and avoid memory decay caused by quantization.

[0051] In addition, when it is found that the performance of the compensated model on the validation set does not improve or even declines for two consecutive evaluation periods, a warm start of the compensation network is automatically performed: all hidden states of the bidirectional long short-term memory network are randomly initialized, and the weights of the 1×1 convolutional layer are reset to initial values ​​close to the identity matrix, so that the compensation network can relearn the quantization error distribution from zero and avoid getting trapped in local optima.

[0052] By applying the technical solution of this embodiment, quantization errors are extracted in real time during forward propagation, and a bidirectional long short-term memory network is used to perform temporal modeling of the spatial context of the errors, generating a compensation tensor rich in structural information. Then, through adaptive fusion of channel concatenation and 1×1 convolution, the compensation information is injected into the original quantization weights, ultimately forming an error compensation matrix and applying it to weight replacement. This series of operations enables the model to effectively recover key features lost due to quantization under low-precision representation, avoiding the problem of traditional fixed compensation schemes failing under complex data distributions.

[0053] Furthermore, as Figure 1 In terms of specific implementation, this application provides a model training system based on gradient sensitivity and hardware awareness, such as... Figure 3 As shown, the system includes: server 301 and client 302; The server 301 is configured to train the target model for each training step based on a preset learning rate decay rule. During the training process of each training step, the server performs the following operations: determining the bit width decay rate corresponding to the current training step based on the learning rate decay rate and the preset bit width decay coefficient, and determining the bit width decay amount corresponding to the current training step based on the bit width decay rate; determining the target layer in the target model and reducing its bit width based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width decay amount, with a preset minimum bit width as a constraint, to compress the target model; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model; and training the compressed target model for the current training step. The client 302 is used to receive the target model trained by the server and deploy it locally.

[0054] In one alternative embodiment, the server 301 is further configured to: The training samples of the current training step are input into the compressed target model, and the quantization error corresponding to each weight of the target layer is obtained during the forward propagation process. The compensation tensor corresponding to the quantization error is generated by a bidirectional long short-term memory network. The quantization error and the compensation tensor are concatenated to obtain a concatenation matrix. The concatenation vector is then convolved to restore the concatenation vector to an error compensation matrix that matches the number of channels in the target layer. The target layer is weighted according to the error compensation matrix to complete the training of the compressed target model for the current training step.

[0055] In one alternative embodiment, the server 301 is further configured to: Obtain the current floating-point weight tensor of the target layer; The floating-point weight tensor is sequentially quantized and dequantized to quantize it into an integer and then dequantize it back into a floating-point number. The floating-point weight tensor and the dequantization result are then used as the initial quantization error. The initial quantization error is calculated using L2 normalization to obtain the quantization error.

[0056] In one optional implementation, the quantization error is a three-dimensional matrix with the height, width, and number of channels of the target layer as dimensions; the server 301 is further configured to: The length of the two-dimensional feature vector is determined by the product of the height and the width, the correspondence between each element in the two-dimensional feature vector and the height and the width is determined, and the data in the quantization error is mapped to the two-dimensional feature vector according to the correspondence. The two-dimensional feature vector is input into a bidirectional long short-term memory network to obtain the initial compensation tensor corresponding to the two-dimensional feature vector. The initial compensation tensor is a two-dimensional matrix with the length of the two-dimensional feature vector and twice the time step as its dimensions. The time step is the number of hidden units in the bidirectional long short-term memory network. Based on the correspondence between each element in the initial compensation tensor and the given information, the initial compensation tensor is restored to a three-dimensional matrix with the height, width, and twice the time step as dimensions, in order to obtain the compensation tensor corresponding to the quantization error.

[0057] In one alternative embodiment, the server 301 is further configured to: The quantization error and the compensation tensor are concatenated into a three-dimensional matrix with the height, width and accumulated channel number of the target layer as dimensions, to obtain the concatenation matrix; wherein, the accumulated channel number is the sum of the number of channels and the number of twice the time steps; The stitched matrix is ​​convolved to restore it into a three-dimensional matrix with the height, width, and number of channels of the target layer as dimensions, thus obtaining the error compensation matrix.

[0058] In one alternative implementation, the server 301 is specifically used for: The bit width decrease variable is accumulated for one training step based on the bit width decrease rate. If the bit width decrease variable is greater than or equal to 1, then the integer part of the bit width decrease variable is determined as the bit width decrease number corresponding to the current training step, and the difference between the bit width decrease variable and the bit width decrease number is updated as the new bit width decrease variable; If the bit width reduction variable is less than 1, then the target layer in the target model is determined according to the gradient sensitivity of each layer in the target model and the current hardware device parameters, and the bit width of the target layer is reduced by using the minimum bit width as a constraint.

[0059] In one optional implementation, the current hardware device parameters include the current video memory utilization rate and the current bandwidth congestion level; the server 301 is specifically used for: Identify whether the current training step is a hardware-aware training step, wherein the hardware-aware training step is determined based on a preset training step interval or a preset time interval; If the current training step is not the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the bit width reduction amount, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced; wherein, the minimum bit width constraint includes a first constraint and a second constraint, the first constraint means that the bit width of the target layer after reduction is not less than the minimum bit width corresponding to the target layer, and the second constraint means that the target layer is the layer with the widest bit width among all layers in the target model that satisfy the first constraint; the minimum bit width of the gradient-sensitive layer of the target model is the first minimum bit width, the minimum bit width of the non-gradient-sensitive layer of the target model is the second minimum bit width, the first minimum bit width is greater than the second minimum bit width, and the gradient sensitivity of the gradient-sensitive layer is greater than a preset sensitivity; If the current training step is the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the number of bit width reductions, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced. When the current memory occupancy rate is greater than the preset occupancy rate, the bit width of all layers is reduced by one bit with the first constraint as a constraint. When the current bandwidth congestion level is greater than the preset congestion level, the non-gradient-sensitive layer is reduced by one bit with the first constraint as a constraint.

[0060] It should be noted that other corresponding descriptions of the functional units involved in the gradient-sensitivity and hardware-aware model training system provided in this application embodiment can be found in [reference]. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0061] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 4 As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0062] Those skilled in the art will understand that Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0064] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0065] It should be noted that the user personal information involved in the embodiments of this application is all authorized (with the knowledge and consent) by the relevant parties or fully authorized by all parties, and the executing entity can obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals. It should be noted that if any software tools or components other than those of this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0067] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model training method based on gradient sensitivity and hardware awareness, characterized in that, The method includes: The target model is trained in each training step based on a preset learning rate decay rule. The following operations are performed during the training process of each training step: Based on the learning rate decay rate and the preset bit width decay coefficient corresponding to the current training step, determine the bit width decay rate corresponding to the current training step, and determine the bit width decay amount corresponding to the current training step based on the bit width decay rate. Based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width reduction amount, the target layer in the target model is determined and its bit width is reduced to compress the target model, with a preset minimum bit width as a constraint; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model. Train the compressed target model for the current training step.

2. The method according to claim 1, characterized in that, The training of the compressed target model for the current training step includes: The training samples of the current training step are input into the compressed target model, and the quantization error corresponding to each weight of the target layer is obtained during the forward propagation process. The compensation tensor corresponding to the quantization error is generated by a bidirectional long short-term memory network. The quantization error and the compensation tensor are concatenated to obtain a concatenation matrix. The concatenation vector is then convolved to restore the concatenation vector to an error compensation matrix that matches the number of channels in the target layer. The target layer is weighted according to the error compensation matrix to complete the training of the current training step of the compressed target model.

3. The method according to claim 2, characterized in that, The step of obtaining the quantization error corresponding to each weight of the target layer during the forward propagation process includes: Obtain the current floating-point weight tensor of the target layer; The floating-point weight tensor is sequentially quantized and dequantized to quantize it into an integer and then dequantize it back into a floating-point number. The floating-point weight tensor and the dequantization result are then used as the initial quantization error. The initial quantization error is calculated using L2 normalization to obtain the quantization error.

4. The method according to claim 2, characterized in that, The quantization error is a three-dimensional matrix with the height, width, and number of channels of the target layer as dimensions; the generation of the compensation tensor corresponding to the quantization error through a bidirectional long short-term memory network includes: The length of the two-dimensional feature vector is determined by the product of the height and the width, the correspondence between each element in the two-dimensional feature vector and the height and the width is determined, and the data in the quantization error is mapped to the two-dimensional feature vector according to the correspondence. The two-dimensional feature vector is input into a bidirectional long short-term memory network to obtain the initial compensation tensor corresponding to the two-dimensional feature vector. The initial compensation tensor is a two-dimensional matrix with the length of the two-dimensional feature vector and twice the time step as its dimensions. The time step is the number of hidden units in the bidirectional long short-term memory network. Based on the correspondence between each element in the initial compensation tensor and the given information, the initial compensation tensor is restored to a three-dimensional matrix with the height, width, and twice the time step as dimensions, in order to obtain the compensation tensor corresponding to the quantization error.

5. The method according to claim 4, characterized in that, The step of concatenating the quantization error and the compensation tensor to obtain a concatenated matrix, and then performing convolution calculations on the concatenated matrix to restore it to an error compensation matrix matching the number of channels of the target layer, includes: The quantization error and the compensation tensor are concatenated into a three-dimensional matrix with the height, width and accumulated channel number of the target layer as dimensions, to obtain the concatenation matrix; wherein, the accumulated channel number is the sum of the number of channels and the number of twice the time steps; The stitched matrix is ​​convolved to restore it into a three-dimensional matrix with the height, width, and number of channels of the target layer as dimensions, thus obtaining the error compensation matrix.

6. The method according to claim 1, characterized in that, Determining the bit width decrease amount corresponding to the current training step based on the bit width decrease rate includes: The bit width decrease variable is accumulated for one training step based on the bit width decrease rate. If the bit width decrease variable is greater than or equal to 1, then the integer part of the bit width decrease variable is determined as the bit width decrease number corresponding to the current training step, and the difference between the bit width decrease variable and the bit width decrease number is updated as the new bit width decrease variable; If the bit width reduction variable is less than 1, then the target layer in the target model is determined according to the gradient sensitivity of each layer in the target model and the current hardware device parameters, and the bit width of the target layer is reduced by using the minimum bit width as a constraint.

7. The method according to any one of claims 1 to 6, characterized in that, The current hardware device parameters include the current video memory utilization rate and the current bandwidth congestion level; the step of determining the target layer in the target model and reducing its bit width based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width reduction amount, with the minimum bit width as a constraint, includes: Identify whether the current training step is a hardware-aware training step, wherein the hardware-aware training step is determined based on a preset training step interval or a preset time interval; If the current training step is not the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the bit width reduction amount, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced; wherein, the minimum bit width constraint includes a first constraint and a second constraint, the first constraint means that the bit width of the target layer after reduction is not less than the minimum bit width corresponding to the target layer, and the second constraint means that the target layer is the layer with the widest bit width among all layers in the target model that satisfy the first constraint; the minimum bit width of the gradient-sensitive layer of the target model is the first minimum bit width, the minimum bit width of the non-gradient-sensitive layer of the target model is the second minimum bit width, the first minimum bit width is greater than the second minimum bit width, and the gradient sensitivity of the gradient-sensitive layer is greater than a preset sensitivity; If the current training step is the hardware-aware training step, then based on the gradient sensitivity of each layer in the target model and the number of bit width reductions, the target layer in the target model is determined with the minimum bit width as a constraint and the bit width is reduced. When the current memory occupancy rate is greater than the preset occupancy rate, the bit width of all layers is reduced by one bit with the first constraint as a constraint. When the current bandwidth congestion level is greater than the preset congestion level, the non-gradient-sensitive layer is reduced by one bit with the first constraint as a constraint.

8. A model training system based on gradient sensitivity and hardware awareness, characterized in that, The system includes: a server and a client; The server is configured to train the target model for each training step based on a preset learning rate decay rule. During the training process of each training step, the server performs the following operations: determining the bit width decay rate corresponding to the current training step based on the learning rate decay rate and the preset bit width decay coefficient, and determining the bit width decay amount corresponding to the current training step based on the bit width decay rate; determining the target layer in the target model and reducing its bit width based on the gradient sensitivity of each layer in the target model, the current hardware device parameters, and the bit width decay amount, with a preset minimum bit width as a constraint, to compress the target model; the preset minimum bit width represents the minimum settable bit width value of each layer in the target model; and training the compressed target model for the current training step. The client is used to receive the target model trained by the server and deploy it locally.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images