Deep learning model compression method and device based on GPU hardware perception collaborative optimization

Abstract

The application belongs to the technical field of artificial intelligence, and particularly relates to a deep learning model compression method and device based on GPU hardware perception collaborative optimization. The method comprises the following steps: obtaining a deep learning model to be compressed and specification parameters of a target GPU hardware, constructing a roofline performance model of the GPU hardware based on the specification parameters, and analyzing the calculation intensity of each network layer by using the roofline performance model to identify the network layer type; obtaining model parameters of the deep learning model, determining the Fisher importance degree and quantum correlation importance degree of the model parameters, and determining a hierarchical hardware perception adjustment factor based on the network layer type and the roofline performance model; dynamically allocating weight coefficients according to the network layer type and the depth, weighting and fusing the Fisher importance degree, the quantum correlation importance degree and the hardware perception adjustment factor to obtain the collaborative importance degree of the model parameters; and performing pruning on the model parameters based on the collaborative importance degree.

Description

[0001] Priority Statement

[0002] This application claims partial priority to Chinese Patent Application No. 202510288277.7 (filed March 12, 2025). The priority basis covers the model parameter importance evaluation methods disclosed in the earlier application, including the calculation of Fisher importance and the generation of quantum correlation importance. The GPU hardware-aware optimization content added in this application does not depend on the priority. Technical Field

[0003] This application belongs to the field of artificial intelligence technology, specifically relating to a deep learning model compression method and apparatus based on GPU hardware perception collaborative optimization. Background Technology

[0004] With the widespread application of deep learning models in fields such as computer vision and natural language processing, the model size has increased dramatically, leading to storage and computational bottlenecks when deployed on resource-constrained devices such as mobile GPUs. Therefore, model compression techniques aimed at reducing the number of parameters while maintaining model performance have become a research hotspot.

[0005] Early model compression techniques achieved compression by removing parameters with smaller absolute weights. However, this simplistic approach ignores the structural dependencies of parameters and hardware characteristics, leading to a significant decrease in model accuracy or failure to achieve practical speedup after pruning.

[0006] To overcome the aforementioned problems, existing technologies have developed pruning methods based on parameter importance, such as using Fisher information matrices to evaluate the sensitivity of parameters to model loss (Fisher importance), thereby achieving more refined pruning. These methods optimize pruning decisions through gradient information, improving accuracy preservation to some extent. Furthermore, some studies have introduced hardware-aware elements, such as parallelized pruning for GPU architectures, to improve inference efficiency.

[0007] However, these techniques still have significant drawbacks: First, the importance assessment dimension is too singular, relying excessively on local indicators such as Fisher importance, and lacking analysis of the global correlation between parameters, resulting in damage to the integrity of the model structure after pruning; second, hardware awareness is disconnected from importance assessment: the pruning strategy is not dynamically adjusted based on the GPU roofline performance model, resulting in a mismatch between the theoretical compression ratio and the actual acceleration effect; finally, the weight allocation strategy is rigid and cannot adapt to the characteristics of different network layers, limiting the universality and efficiency of the method. Summary of the Invention

[0008] This application proposes a deep learning model compression scheme based on GPU hardware-aware collaborative optimization, aiming to solve the problems of single importance evaluation dimension, disconnect between hardware awareness and importance evaluation, and rigid weight allocation strategy in existing deep learning model compression schemes.

[0009] The first aspect of this application provides a deep learning model compression method based on GPU hardware-aware collaborative optimization, including:

[0010] Obtain the specifications of the deep learning model to be compressed and the target GPU hardware, construct the roofline performance model of the GPU hardware based on the specifications, and use the roofline performance model to analyze the computational intensity of each network layer. Identify the network layer type based on the computational intensity, wherein the network layer type includes computationally intensive and memory-intensive.

[0011] Obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model;

[0012] The weight coefficients are dynamically allocated according to the network layer type and depth. The Fisher importance, quantum correlation importance and hardware-aware adjustment factor are then weighted and fused to obtain the co-importance of the model parameters.

[0013] The model parameters are pruned based on the collaborative importance to compress the deep learning model.

[0014] In some embodiments of this application, the specifications include memory bandwidth, peak computing power, and cache hierarchy.

[0015] In some embodiments of this application, the step of analyzing the computational intensity of each network layer using the roofline performance model and identifying the network layer type based on the computational intensity includes:

[0016] The computational strength of network layer i is calculated based on the following formula. :

[0017]

[0018] in, The number of floating-point operations for network layer i. This represents the total memory read / write volume of network layer i.

[0019] When the AI ​​value is less than 5, the network layer is classified as a memory-intensive layer; when the AI ​​value is greater than 20, the network layer is classified as a computation-intensive layer.

[0020] In some embodiments of this application, determining the Fisher importance of the model parameters includes:

[0021] The parameters are grouped according to their dependencies on the model parameters;

[0022] Calculate the Hessian matrix for each set of parameters. The elements of the Hessian matrix are composed of the second-order partial derivatives of the loss function with respect to the parameters.

[0023] Eigenvalues ​​are extracted from the Hessian matrix to generate Fisher importance, where Fisher importance is used to characterize the statistical sensitivity of parameters to model loss.

[0024] In some embodiments of this application, determining the quantum correlation importance of the model parameters includes:

[0025] The model parameters are mapped to quantum states using the following formula:

[0026] ,

[0027] in , , For the i-th model parameter;

[0028] The correlation strength of the parameters is calculated based on the mapping results. The formula for calculating the correlation strength is as follows:

[0029] ,

[0030] Summing all correlation strengths and dividing by the number of samples N generates the quantum correlation importance, i.e.: ,

[0031] The quantum correlation importance is used to characterize the global correlation between parameters.

[0032] In some embodiments of this application, determining the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model includes:

[0033] For compute-intensive layers, the layer hardware-aware adjustment factor is calculated using the following formula:

[0034] R_hw(i) = 1.0 + 0.5 × log(1 + AI_i), where AI_i is the computational intensity of the i-th layer; for memory-intensive layers, the layer hardware-aware adjustment factor is calculated using the following formula:

[0035] R_hw(i) = 0.5 + 0.5 × memory_efficiency, where memory_efficiency is the memory efficiency factor, and the calculation formula is memory_efficiency = exp(-AI_i / 5);

[0036] The hierarchical hardware-aware adjustment factor is shared across the entire network layer, rather than being a single parameter.

[0037] In some embodiments of this application, the dynamically allocated weighting coefficients include:

[0038] Weight coefficients α, β, and γ are assigned according to the network layer type and depth ratio, where α is the weight coefficient for Schere importance, β is the weight coefficient for quantum correlation importance, and γ is the weight coefficient for hardware-aware modulation factor, satisfying α + β + γ = 1; where:

[0039] For early convolutional layers in the ResNet series networks with a depth ratio of 0-33%, set α=0.30, β=0.30, and γ=0.40;

[0040] For deep convolutional layers in the ResNet series networks with a depth ratio of 66-100%, set α=0.50, β=0.30, and γ=0.20;

[0041] For the early convolutional layers of the VGG network, set α=0.25, β=0.35, and γ=0.40.

[0042] In some embodiments of this application, pruning the model parameters based on the collaborative importance includes:

[0043] Based on the requirement of target compression rate, for each network layer, the pruning rate target of the layer is set according to its layer type and depth ratio, wherein the layer type includes computationally intensive layers and memory-intensive layers, and the depth ratio is based on the total network depth.

[0044] Within each layer, the parameters are sorted in descending order of their collaborative importance.

[0045] Remove the least important model parameters or structural units from the sorting list one by one until the pruning rate target for that layer is reached.

[0046] After pruning, the model accuracy and GPU inference latency of the deep learning model were measured.

[0047] If the accuracy drops beyond the preset threshold or the improvement in inference latency does not meet expectations, the weight coefficients α, β, and γ are adjusted, the collaborative importance is recalculated, and the next round of pruning is performed.

[0048] The iterative pruning process continues until one of the following conditions is met: the target compression ratio is achieved, the accuracy degradation is within an acceptable range, or the improvement in GPU inference latency approaches saturation.

[0049] In some embodiments of this application, the method further includes:

[0050] If the model accuracy drops more than a preset threshold after pruning or the improvement in GPU inference latency does not meet the expected metrics, then a limited number of iterations will be used to fine-tune the model using a portion of the training data.

[0051] Monitor the model's accuracy recovery during the fine-tuning process;

[0052] The fine-tuning process is terminated when the accuracy is restored to an acceptable range or the fine-tuning benefits tend to saturate.

[0053] Verify the GPU inference latency and model accuracy of the final fine-tuned model.

[0054] A second aspect of this application provides a deep learning model compression device based on GPU hardware-aware collaborative optimization, characterized in that it includes:

[0055] The hardware awareness module is used to acquire the specifications of the deep learning model to be compressed and the target GPU hardware, construct a roofline performance model of the GPU hardware based on the specifications, analyze the computational intensity of each network layer using the roofline performance model, and identify the network layer type based on the computational intensity. The network layer type includes computationally intensive and memory-intensive types.

[0056] The importance calculation module is used to obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model.

[0057] The weight fusion module is used to dynamically allocate weight coefficients according to the network layer type and depth, and to weight and fuse the Fisher importance, quantum correlation importance and hardware-aware adjustment factor to obtain the co-importance of the model parameters.

[0058] The pruning module is used to prune the model parameters based on the collaborative importance to compress the deep learning model.

[0059] In summary, the deep learning model compression method and apparatus based on GPU hardware-aware co-optimization provided in the embodiments of this application achieve a comprehensive analysis of parameter importance by integrating Fisher importance, quantum correlation importance, and hardware-aware adjustment factors for multi-dimensional importance co-evaluation. This is because Fisher importance quantifies the local sensitivity of parameters to model loss, quantum correlation importance captures the global structural dependencies between parameters, and the hardware-aware adjustment factor introduces GPU efficiency considerations. The collaboration of these three factors ensures that pruning decisions consider both performance sensitivity and structural integrity, thereby significantly improving the accuracy retention and robustness of the pruned model. Furthermore, the dynamic adjustment technology based on the Roofline model achieves deep integration of hardware awareness and importance evaluation. The Roofline model quantifies the computational intensity of each network layer based on GPU hardware specifications (such as memory bandwidth and computing power) and dynamically generates adjustment factors, enabling the pruning strategy to... Prioritizing optimization for memory-intensive or computationally-intensive layers ensures a high degree of consistency between theoretical compression ratio and actual acceleration effect. Adaptive optimization for different network layer characteristics is achieved by adaptively adjusting weight coefficients α, β, and γ based on network layer type and depth ratio. Since the weight allocation is dynamically adjusted according to the network layer type of the deep learning model, it balances the contributions of local sensitivity, global correlation, and hardware efficiency, thereby enhancing the method's universality and applicability to various network architectures (such as VGG and ResNet), and improving compression efficiency. Furthermore, this application also ensures robust recovery of model performance through iterative pruning and fine-tuning techniques based on threshold dynamic adjustment and a closed-loop accuracy verification system. After pruning, verification is performed by measuring GPU inference latency and model accuracy. If performance does not meet expectations, it is rolled back and optimized through fine-tuning (using a subset of training data for 1-2 epochs), minimizing accuracy loss and achieving end-to-end compression process reliability. Attached Figure Description

[0060] The features and advantages of this application will be more clearly understood by referring to the accompanying drawings, which are illustrative and should not be construed as limiting the application in any way. In the drawings:

[0061] Figure 1 This is a framework diagram of a deep learning model compression algorithm based on GPU hardware-aware collaborative optimization, as shown in this application.

[0062] Figure 2 This is a flowchart illustrating a deep learning model compression method based on GPU hardware-aware collaborative optimization according to some embodiments of this application;

[0063] Figure 3 This is a specific process for fine-tuning a pruned deep learning model, as shown in some embodiments of this application;

[0064] Figure 4 This is a schematic diagram of a deep learning model compression device based on GPU hardware-aware collaborative optimization, according to some embodiments of this application. Detailed Implementation

[0065] In the following detailed description, numerous specific details of this application are illustrated by example to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that this application can be practiced without these details. It should be understood that the terms “system,” “apparatus,” “unit,” and / or “module” used in this application are one way of distinguishing different parts, elements, sections, or components at different levels in a sequential arrangement. However, these terms may be replaced with other expressions if other expressions can achieve the same purpose.

[0066] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly connected to or coupled to or communicate with other devices, units, or modules, or there may be intermediate devices, units, or modules present, unless the context explicitly indicates otherwise. For example, the term "and / or" as used herein includes any one and all combinations of one or more of the relevant listed items.

[0067] The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate that explicitly identified features, integrals, steps, operations, elements, and / or components are included, and such expressions do not constitute an exclusive list, and other features, integrals, steps, operations, elements, and / or components may also be included.

[0068] Referring to the following description and accompanying drawings, these and other features and characteristics, operating methods, functions of related structural elements, combinations of parts, and economics of manufacture of this application can be better understood, wherein the description and drawings form part of the specification. However, it is clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. It is understood that the drawings are not drawn to scale.

[0069] Various structural diagrams are used in this application to illustrate various variations of the embodiments according to this application. It should be understood that the preceding or following structures are not intended to limit this application. The scope of protection of this application is determined by the claims.

[0070] With the successful application of deep learning models in various tasks, the size and complexity of these models are rapidly increasing. Although models with large parameter sizes generally have higher performance, training and inference can be affected to some extent when faced with limited target hardware resources, resulting in performance loss. Deep learning model compression methods are of significant research importance in model optimization and deployment.

[0071] Model compression includes model pruning, model quantization, and knowledge distillation. The compression method described in this application refers to model pruning. Model pruning can significantly reduce the number of model parameters while minimizing performance loss. Pruning is divided into structured pruning and unstructured pruning. Unstructured pruning is hardware-dependent and has limited scalability. Structured pruning is hardware-friendly and suitable for GPU acceleration of models.

[0072] However, traditional pruning methods do not fully consider the characteristics of GPU hardware, resulting in a mismatch between theoretical compression effects and actual inference acceleration; existing importance evaluation methods use the same strategy for different network layers, lacking hierarchical adaptability; at the same time, a single importance index is difficult to fully reflect the true value of the parameters.

[0073] To address the aforementioned issues, this application provides a deep learning model compression method based on GPU hardware-aware collaborative optimization. This method calculates parameter importance through parameter-level evaluation (including parameter statistical sensitivity evaluation and parameter structure correlation evaluation) and hierarchical GPU hardware feature analysis, thereby collaboratively optimizing the parameters to achieve efficient model pruning. The aim is to reduce the number of model parameters while minimizing the loss of model accuracy, ultimately accelerating model inference. Figure 1 This is a framework diagram of a deep learning model compression algorithm based on GPU hardware-aware co-optimization, as shown in this application. In some embodiments, the deep learning model compression method based on GPU hardware-aware co-optimization is as follows: Figure 2 As shown, the specific steps include:

[0074] S210: Obtain the specifications of the deep learning model to be compressed and the target GPU hardware; construct a roofline performance model of the GPU hardware based on the specifications; analyze the computational intensity of each network layer using the roofline performance model; identify the network layer type based on the computational intensity; wherein the network layer type includes computationally intensive and memory-intensive types.

[0075] Specifically, the first step is to perform hardware performance analysis on the target GPU. This includes obtaining the specifications of the target GPU hardware, including memory bandwidth, peak computing power, and cache structure. Then, based on these specifications, a roofline performance model of the GPU hardware is constructed to quantify the computational intensity of each network layer.

[0076] In some embodiments of this application, the computational strength of network layer i The calculation formula is:

[0077]

[0078] in, The number of floating-point operations for network layer i. This represents the total memory read / write volume of network layer i.

[0079] This application identifies the type of a network layer by analyzing the computational intensity (AI value) of each network layer. In some embodiments of this application, when the AI ​​value is less than 5, the network layer is classified as a memory-intensive layer; when the AI ​​value is greater than 20, the network layer is classified as a computation-intensive layer.

[0080] S220, obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model.

[0081] This application evaluates the statistical sensitivity of deep learning model parameters using Fisher importance. Specifically, the parameters are first grouped according to their dependencies, and the Hessian matrix of each group is calculated. Feature values ​​are then extracted from the Hessian matrix to generate Fisher importance. Fisher importance characterizes the statistical sensitivity of parameters to the model loss. This statistical sensitivity dimension evaluates the direct contribution of each parameter to model performance at the gradient optimization level. By quantifying the impact of parameter perturbations on the loss function, the key parameters most sensitive to the model output are identified.

[0082] In some embodiments of this application, the calculation of Fisher importance is specifically as follows:

[0083] The Hessian matrix for calculating the model parameters is given by the following formula:

[0084]

[0085] The specific formula for calculating Fisher information is as follows:

[0086]

[0087] in For the sample size, The i-th sample number, In the sample The Hessian matrix on.

[0088] The Fisher importance, used to characterize the statistical sensitivity of the parameters, is calculated using the following formula:

[0089] I_Fisher

[0090] Where λᵢ is the i-th eigenvalue of the Fisher matrix F.

[0091] This application evaluates the global correlation between parameters in a deep learning model using quantum correlation importance. Specifically, it performs quantum state mapping on the parameters of the model to be pruned, calculates the correlation strength of the parameters, and derives the importance of parameter structural correlation. The global correlation dimension evaluates the global correlation pattern of parameters in the network from the perspective of parameter distribution relationships. It identifies the synergistic effects and dependencies between parameters through quantum state inner product calculation, ensuring the integrity of the network structure after pruning.

[0092] In some embodiments of this application, the calculation of quantum correlation importance is specifically as follows:

[0093] First, parameter quantum state mapping is performed, and the calculation formula is as follows:

[0094]

[0095] in , , Let be the i-th parameter of the model.

[0096] Then, the parameter correlation strength is calculated based on the results of quantum state mapping, using the following formula:

[0097]

[0098] The expansion is as follows:

[0099]

[0100] The final calculated importance of parameter correlation is shown below:

[0101] .

[0102] This application determines hierarchical hardware-aware adjustment factors based on the network layer type and the roofline performance model. These hardware-aware adjustment factors characterize the impact of hardware execution efficiency on model compression decisions.

[0103] In one embodiment of this application, the specific calculation formula for the hierarchical hardware awareness adjustment factor is as follows:

[0104] Computationally dense layers:

[0105] Memory-intensive layer:

[0106] Where: R_hw(i) is the adjustment factor for the i-th layer, which is shared by all parameters of the i-th layer; AI_i is the computational intensity of the i-th layer; and memory_efficiency is the memory efficiency factor, which is calculated as memory_efficiency=exp(-AI_i / 5).

[0107] S230, dynamically allocate weight coefficients according to network layer type and depth, and weight and fuse the Fisher importance, quantum correlation importance and hardware-aware adjustment factor to obtain the co-importance of the model parameters.

[0108] This application prunes the model based on co-importance. Co-importance is a weighted sum of Fisher importance, quantum correlation importance, and hardware-aware modulating factor.

[0109] This application dynamically allocates weight coefficients based on the network layer type and depth ratio. The dynamic allocation of the three weights satisfies the constraint: α+β+γ=1, where α is the Fisher importance weight, β is the quantum correlation importance weight, and γ is the hardware-aware adjustment factor weight.

[0110] In some embodiments of this application, the adaptive rule for weight allocation is as follows:

[0111] If the computational intensity AI of the network layer is greater than 20 (computationally intensive): γ decreases by 0.05-0.10;

[0112] If the computational intensity AI of the network layer is less than 5 (memory-intensive): increase γ by 0.05-0.15.

[0113] If the network layer contains residual connections: increase β by 0.05-0.10.

[0114] If the network layer is close to the classification head (depth > 90%): α increases by 0.05-0.15.

[0115] In other embodiments of this application, different weight allocation strategies are employed for different deep computational neural network models, such as:

[0116] ResNet series networks:

[0117] Early convolutional layers (network depth 0-33%): α=0.30, β=0.30, γ=0.40

[0118] Principle: Early layers extract basic features (edges, textures), which has a significant impact on hardware efficiency; therefore, GPU execution efficiency should be prioritized.

[0119] Intermediate convolutional layers (network depth 33-66%): α=0.40, β=0.40, γ=0.20

[0120] Principle: The intermediate layer learns the semantic features of the middle layer, emphasizing both parameter sensitivity and structural relevance, thus balancing performance and structure.

[0121] Deep convolutional layers (network depth 66-100%): α=0.50, β=0.30, γ=0.20

[0122] Principle: Deep learning of high-level semantics has a significant impact on the final classification, and more attention is paid to the performance contribution of parameters.

[0123] Fully connected layer: α=0.40, β=0.20, γ=0.40

[0124] Principle: Fully connected layers have a large number of parameters and are memory-intensive, balancing hardware efficiency and performance, with relatively weak structural dependencies.

[0125] VGG series networks:

[0126] Early convolutional layers (network depth 0-33%): α=0.25, β=0.35, γ=0.40

[0127] Principle: VGG has a simple structure but many layers. Early layers focused more on structural integrity and hardware efficiency.

[0128] Intermediate convolutional layers (network depth 33-66%): α=0.35, β=0.40, γ=0.25

[0129] Principle: The intermediate layer connects the upper and lower layers, and its structural interrelationship is of utmost importance.

[0130] Deep convolutional layers (network depth 66-100%): α=0.45, β=0.35, γ=0.20

[0131] Principle: Deep semantic features are important; appropriately increasing the Fisher weights is crucial.

[0132] Fully connected layer: α=0.40, β=0.20, γ=0.40

[0133] Principle: VGG's FC layer parameters have a very high weighting, and hardware efficiency has a significant impact.

[0134] Weighting adjustment principles:

[0135] Computationally intensive layers: Reduce the γ value (0.1-0.2), as GPU utilization is already high in these layers.

[0136] Memory-intensive layers: Increase the γ value (0.3-0.5) and prioritize pruning for significant acceleration.

[0137] Residual connection layer: Improves the β value (0.4-0.5) and maintains structural integrity.

[0138] Classification head layer: Increase the α value (0.5-0.6) to ensure classification performance.

[0139] S240, prune the model parameters based on the collaborative importance to compress the deep learning model.

[0140] In some embodiments of this application, pruning model parameters based on collaborative importance to compress the model includes:

[0141] First, set the overall target compression rate based on the application scenario, such as reducing the number of parameters by 50-70%, as a global constraint for the pruning process.

[0142] Based on the overall target compression ratio requirement, for each network layer, the pruning rate target for that layer is set according to its layer type (computation-intensive or memory-intensive) and depth ratio (based on the total network depth).

[0143] Within each layer, the parameters are sorted in descending order of their collaborative importance.

[0144] Remove the least important model parameters or structural units from the sorting list one by one until the pruning rate target for that layer is reached.

[0145] After pruning, the model accuracy and GPU inference latency of the deep learning model were measured.

[0146] If the accuracy drops beyond the preset threshold or the improvement in inference latency does not meet expectations, the weight coefficients α, β, and γ are adjusted, the collaborative importance is recalculated, and the next round of pruning is performed.

[0147] The iterative pruning process continues until one of the following conditions is met: the target compression ratio is achieved, the accuracy degradation is within an acceptable range, or the improvement in GPU inference latency approaches saturation.

[0148] In some embodiments, the target pruning rate is set as follows: 40-50% for early convolutional layers in ResNet, 60-70% for deep convolutional layers; and 65-75% for fully connected layers in VGG. Those skilled in the art can adjust this according to actual needs.

[0149] This application achieves collaborative pruning by integrating the importance assessment results from three dimensions.

[0150] This collaborative mechanism ensures that:

[0151] Fisher importance dimension ensures that the performance loss of the model is minimized after pruning;

[0152] The quantum correlation dimension ensures the integrity of the network structure after pruning;

[0153] Hardware efficiency measures ensure the actual acceleration of GPU inference after pruning.

[0154] In some embodiments of this disclosure, the results of various metrics of the deep learning model are verified after pruning, and the verification results are used as feedback to iteratively prune the model.

[0155] In some embodiments of this application, the method further includes:

[0156] Fine-tuning the pruned deep learning model further restores its accuracy.

[0157] Figure 3 This describes the specific process for fine-tuning a pruned deep learning model, as shown in some embodiments of this application. For example... Figure 3 As shown, fine-tuning includes:

[0158] Measure the performance metrics of the pruned deep learning model, such as GPU inference latency and model accuracy;

[0159] If the performance does not meet the expected targets based on the validation results, the model will be fine-tuned.

[0160] The fine-tuning process uses a subset of the training set and iterates for 1-2 epochs.

[0161] After fine-tuning, verify the GPU inference latency and model accuracy again. If the performance still does not meet the standard, repeat the fine-tuning process.

[0162] Iteratively fine-tune until performance recovers to an acceptable range or the fine-tuning benefits approach saturation.

[0163] Through fine-tuning, the pruned deep learning model can recover most of the original model's performance. Ultimately, the deep learning model is compressed with an acceptable performance loss, enabling deployment on performance-constrained devices.

[0164] Figure 4 This is a schematic diagram of a deep learning model compression device based on GPU hardware-aware collaborative optimization, according to some embodiments of this application. Figure 4 As shown, the deep learning model compression device 400 includes a hardware awareness module 410, an importance calculation module 420, a weight fusion module 430, and a pruning module 440. Wherein:

[0165] The hardware awareness module 410 is used to acquire the specifications of the deep learning model to be compressed and the target GPU hardware, construct a roofline performance model of the GPU hardware based on the specifications, analyze the computational intensity of each network layer using the roofline performance model, and identify the network layer type based on the computational intensity, wherein the network layer type includes computationally intensive and memory-intensive.

[0166] Importance calculation module 420 is used to obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model.

[0167] The weight fusion module 430 is used to dynamically allocate weight coefficients according to the network layer type and depth, and to weight and fuse the Fisher importance, quantum correlation importance and hardware-aware adjustment factor to obtain the co-importance of the model parameters.

[0168] The pruning module 440 is used to prune the model parameters based on the collaborative importance to compress the deep learning model.

[0169] In summary, the deep learning model compression method and apparatus based on GPU hardware-aware co-optimization provided in the embodiments of this application achieve a comprehensive analysis of parameter importance by integrating Fisher importance, quantum correlation importance, and hardware-aware adjustment factors for multi-dimensional importance co-evaluation. This is because Fisher importance quantifies the local sensitivity of parameters to model loss, quantum correlation importance captures the global structural dependencies between parameters, and the hardware-aware adjustment factor introduces GPU efficiency considerations. The collaboration of these three factors ensures that pruning decisions consider both performance sensitivity and structural integrity, thereby significantly improving the accuracy retention and robustness of the pruned model. Furthermore, the dynamic adjustment technology based on the Roofline model achieves deep integration of hardware awareness and importance evaluation. The Roofline model quantifies the computational intensity of each network layer based on GPU hardware specifications (such as memory bandwidth and computing power) and dynamically generates adjustment factors, enabling the pruning strategy to... Prioritizing optimization for memory-intensive or computationally-intensive layers ensures a high degree of consistency between theoretical compression ratio and actual acceleration effect. Adaptive optimization for different network layer characteristics is achieved by adaptively adjusting weight coefficients α, β, and γ based on network layer type and depth ratio. Since the weight allocation is dynamically adjusted according to the network layer type of the deep learning model, it balances the contributions of local sensitivity, global correlation, and hardware efficiency, thereby enhancing the method's universality and applicability to various network architectures (such as VGG and ResNet), and improving compression efficiency. Furthermore, this application also ensures robust recovery of model performance through iterative pruning and fine-tuning techniques based on threshold dynamic adjustment and a closed-loop accuracy verification system. After pruning, verification is performed by measuring GPU inference latency and model accuracy. If performance does not meet expectations, it is rolled back and optimized through fine-tuning (using a subset of training data for 1-2 epochs), minimizing accuracy loss and achieving end-to-end compression process reliability.

[0170] Although the subject matter described herein is provided in the general context of execution on a computer system in conjunction with an operating system and applications, those skilled in the art will recognize that other implementations can also be executed in conjunction with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will understand that the subject matter described herein can be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframes, etc., and can also be used in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may reside on both local and remote memory storage devices.

[0171] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this application and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this application should be included within the protection scope of this application. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A deep learning model compression method based on GPU hardware-aware collaborative optimization, characterized in that, include: Obtain the specifications of the deep learning model to be compressed and the target GPU hardware, construct the roofline performance model of the GPU hardware based on the specifications, and use the roofline performance model to analyze the computational intensity of each network layer. Identify the network layer type based on the computational intensity, wherein the network layer type includes computationally intensive and memory-intensive. Obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model; The weight coefficients are dynamically allocated according to the network layer type and depth. The Fisher importance, quantum correlation importance and hardware-aware adjustment factor are then weighted and fused to obtain the co-importance of the model parameters. The model parameters are pruned based on the collaborative importance to compress the deep learning model; The determination of Fisher importance of the model parameters includes: The parameters are grouped according to their dependencies on the model parameters; Calculate the Hessian matrix for each set of parameters. The elements of the Hessian matrix are composed of the second-order partial derivatives of the loss function with respect to the parameters. Eigenvalues ​​are extracted from the Hessian matrix to generate Fisher importance, wherein the Fisher importance is used to characterize the statistical sensitivity of parameters to model loss; Determining the quantum correlation importance of the model parameters includes: The model parameters are mapped to quantum states using the following formula: ， in, , , For the i-th model parameter, This represents the quantum state corresponding to the i-th parameter. This represents the phase angle of the i-th parameter; The correlation strength of the parameters is calculated based on the mapping results. The formula for calculating the correlation strength is as follows: ， Summing all correlation strengths and dividing by the number of samples N generates the quantum correlation importance, i.e.: , The quantum correlation importance is used to characterize the global correlation between parameters.

2. The method according to claim 1, characterized in that... : The specifications include memory bandwidth, peak computing power, and cache hierarchy.

3. The method according to claim 1, characterized in that, The step of analyzing the computational intensity of each network layer using the roofline performance model and identifying the network layer type based on the computational intensity includes: The computational intensity AI_i of network layer i is calculated based on the following formula: in, The number of floating-point operations for network layer i. This represents the total memory read / write volume of network layer i. When the AI ​​value is less than 5, the network layer is classified as a memory-intensive layer; when the AI ​​value is greater than 20, the network layer is classified as a computation-intensive layer.

4. The method according to claim 3, characterized in that, The determination of the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model includes: For compute-intensive layers, the layer hardware-aware adjustment factor is calculated using the following formula: R_hw(i) = 1.0 + 0.5 × log(1 + AI_i), where AI_i is the computational intensity of the i-th layer; for memory-intensive layers, the layer hardware-aware adjustment factor is calculated using the following formula: R_hw(i)=0.5+0.5×memory_efficiency, Where memory_efficiency is the memory efficiency factor, and the calculation formula is memory_efficiency=exp(-AI_i / 5); The hierarchical hardware-aware adjustment factor is shared across the entire network layer, rather than being a single parameter.

5. The method according to claim 1, characterized in that, The dynamic allocation of weight coefficients based on network layer type and depth includes: Weight coefficients α, β, and γ are assigned according to the network layer type and depth ratio, where α is the weight coefficient for Schere importance, β is the weight coefficient for quantum correlation importance, and γ is the weight coefficient for hardware-aware modulation factor, satisfying α + β + γ = 1; where: For early convolutional layers in the ResNet series networks with a depth ratio of 0-33%, set α=0.30, β=0.30, and γ=0.40; For deep convolutional layers in the ResNet series networks with a depth ratio of 66-100%, set α=0.50, β=0.30, and γ=0.20; For the early convolutional layers of the VGG network, set α=0.25, β=0.35, and γ=0.

40.

6. The method according to claim 1, characterized in that, The pruning of the model parameters based on the collaborative importance includes: Based on the requirement of target compression rate, for each network layer, the pruning rate target of the layer is set according to its layer type and depth ratio, wherein the layer type includes computationally intensive layers and memory-intensive layers, and the depth ratio is based on the total network depth. Within each layer, the parameters are sorted in descending order of their collaborative importance. Remove the least important model parameters or structural units from the sorting list one by one until the pruning rate target for that layer is reached. After pruning, the model accuracy and GPU inference latency of the deep learning model were measured. If the accuracy drops beyond the preset threshold or the improvement in inference latency does not meet expectations, the weight coefficients α, β, and γ are adjusted, the collaborative importance is recalculated, and the next round of pruning is performed. The iterative pruning process continues until one of the following conditions is met: the target compression ratio is achieved, the accuracy degradation is within an acceptable range, or the improvement in GPU inference latency approaches saturation.

7. The method according to claim 1, characterized in that, The method further includes: If the model accuracy drops more than a preset threshold after pruning or the improvement in GPU inference latency does not meet the expected metrics, then a limited number of iterations will be used to fine-tune the model using a portion of the training data. Monitor the model's accuracy recovery during the fine-tuning process; The fine-tuning process is terminated when the accuracy is restored to an acceptable range or the fine-tuning benefits tend to saturate. Verify the GPU inference latency and model accuracy of the final fine-tuned model.

8. A deep learning model compression device based on GPU hardware-aware collaborative optimization, characterized in that, include: The hardware awareness module is used to acquire the specifications of the deep learning model to be compressed and the target GPU hardware, construct a roofline performance model of the GPU hardware based on the specifications, analyze the computational intensity of each network layer using the roofline performance model, and identify the network layer type based on the computational intensity. The network layer type includes computationally intensive and memory-intensive types. The importance calculation module is used to obtain the model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model. The weight fusion module is used to dynamically allocate weight coefficients according to the network layer type and depth, and to weight and fuse the Fisher importance, quantum correlation importance and hardware-aware adjustment factor to obtain the co-importance of the model parameters. A pruning module is used to prune the model parameters based on the collaborative importance to compress the deep learning model; The determination of Fisher importance of the model parameters includes: The parameters are grouped according to their dependencies on the model parameters; Calculate the Hessian matrix for each set of parameters. The elements of the Hessian matrix are composed of the second-order partial derivatives of the loss function with respect to the parameters. Eigenvalues ​​are extracted from the Hessian matrix to generate Fisher importance, wherein the Fisher importance is used to characterize the statistical sensitivity of parameters to model loss; Determining the quantum correlation importance of the model parameters includes: The model parameters are mapped to quantum states using the following formula: ， in, , , For the i-th model parameter, This represents the quantum state corresponding to the i-th parameter. This represents the phase angle of the i-th parameter; The correlation strength of the parameters is calculated based on the mapping results. The formula for calculating the correlation strength is as follows: ， Summing all correlation strengths and dividing by the number of samples N generates the quantum correlation importance, i.e.: , The quantum correlation importance is used to characterize the global correlation between parameters.

Images