Deep learning model compression method and device based on GPU hardware perception collaborative optimization

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a deep learning model compression method and device based on GPU hardware perception collaborative optimization. The method comprises the following steps: obtaining a deep learning model to be compressed and specification parameters of target GPU hardware, constructing a roof line performance model of the GPU hardware based on the specification parameters, and analyzing the calculation strength of each network layer by using the roof line performance model to identify the type of the network layer; obtaining model parameters of the deep learning model, determining Fisher importance and quantum correlation importance of the model parameters, and determining hierarchical hardware perception regulatory factors based on the network layer type and the roof line performance model; according to a network layer type and a depth dynamic allocation weight coefficient, carrying out weighted fusion on the Fisher importance, the quantum correlation importance and the hardware perception adjustment factor to obtain a collaborative importance of the model parameters; and pruning the model parameters based on the collaborative importance degree.

Description

[0001] CLAIM OF PRIORITY

[0002] This application claims partial priority of Chinese patent application No. 202510288277.7 (filing date 2025 / 03 / 12). The priority basis covers the model parameter importance evaluation method disclosed in the prior application, including the calculation of Fisher importance and the generation of quantum correlation importance. The newly added GPU hardware-aware optimization content of this application does not depend on the priority. TECHNICAL FIELD

[0003] The present application belongs to the field of artificial intelligence technology, and specifically relates to a deep learning model compression method and device based on GPU hardware-aware collaborative optimization. BACKGROUND

[0004] With the wide application of deep learning models in computer vision and natural language processing, the model size has increased dramatically, leading to storage and computing bottlenecks when deploying on resource-constrained devices such as mobile GPUs. Therefore, model compression techniques aimed at reducing parameter quantity while maintaining model performance have become a research hotspot.

[0005] Early model compression techniques achieve compression by removing parameters with small absolute weight values. However, this simple method ignores the structural dependency of parameters and hardware characteristics, resulting in a significant decrease in model accuracy after pruning or inability to actually accelerate.

[0006] To overcome the above problems, existing technologies have developed pruning methods based on parameter importance, such as using Fisher information matrix to evaluate the sensitivity of parameters to model loss (Fisher importance), thereby achieving more refined pruning. This method optimizes pruning decisions through gradient information, improving the accuracy retention rate to some extent. In addition, some research has introduced hardware-aware elements, such as parallel pruning for GPU architecture, to improve inference efficiency.

[0007] However, these technologies still have significant defects: first, the importance evaluation dimension is single, relying too much on local indicators such as Fisher importance, lacking global correlation analysis between parameters, resulting in damage to the structural integrity of the pruned model; second, hardware awareness is disconnected from importance evaluation: unable to dynamically adjust pruning strategies based on GPU roofline performance model (Roofline model), resulting in a mismatch between theoretical compression rate and actual acceleration effect; finally, the weight allocation strategy is rigid and cannot adapt to different network layer characteristics, limiting the universality and efficiency of the method. SUMMARY

[0008] The embodiment of the application provides a deep learning model compression scheme based on GPU hardware perception collaborative optimization, aiming at solving the problems of single evaluation dimension of importance, disconnection between hardware perception and importance evaluation, and rigid weight distribution strategy of an existing deep learning model compression scheme.

[0009] The first aspect of the embodiment of the application provides a deep learning model compression method based on GPU hardware perception collaborative optimization, comprising:

[0010] Obtaining a deep learning model to be compressed and specification parameters of a target GPU hardware, constructing a roofline performance model of the GPU hardware based on the specification parameters, analyzing the calculation intensity of each network layer by using the roofline performance model, and identifying the network layer type according to the calculation intensity, wherein the network layer type includes a computation-intensive type and a memory-intensive type;

[0011] Obtaining model parameters of the deep learning model, determining the Fisher importance and quantum correlation importance of the model parameters, and determining a hierarchical hardware perception adjustment factor based on the network layer type and the roofline performance model;

[0012] According to the network layer type and the depth, a weight coefficient is dynamically allocated, the Fisher importance, the quantum correlation importance and the hardware perception adjustment factor are weighted and fused to obtain the collaborative importance of the model parameters;

[0013] Pruning the model parameters based on the collaborative importance to compress the deep learning model.

[0014] In some embodiments of the application, the specification parameters include memory bandwidth, peak computing capacity and cache hierarchy.

[0015] In some embodiments of the application, the analysis of the calculation intensity of each network layer by using the roofline performance model and the identification of the network layer type according to the calculation intensity include:

[0016] The calculation intensity of the network layer i is calculated based on the following formula :

[0017]

[0018] wherein, is the number of floating point operations of the network layer i, is the total amount of memory read and write of the network layer i;

[0019] When the AI value is less than 5, the network layer is classified as a memory-intensive layer; when the AI value is greater than 20, the network layer is classified as a computation-intensive layer.

[0020] In some embodiments of the application, the determination of the Fisher importance of the model parameters includes:

[0021] grouping the parameters according to their dependence on the model parameters;

[0022] computing a Hessian matrix for each group of parameters, elements of the Hessian matrix being composed of second-order partial derivatives of the loss function with respect to the parameters;

[0023] extracting eigenvalues from the Hessian matrix to generate Fisher importances, wherein the Fisher importances are used to characterize statistical sensitivity of the parameters to the model loss.

[0024] In some embodiments of the present application, determining the quantum correlation importance of the model parameters comprises:

[0025] mapping the model parameters into quantum states according to the following formula:

[0026] ,

[0027] wherein , , is the i-th model parameter;

[0028] computing parameter correlation strengths based on the mapping results according to the following formula:

[0029] ,

[0030] summing all the correlation strengths and dividing by the sample number N to generate the quantum correlation importance, i.e.: ,

[0031] wherein the quantum correlation importance is used to characterize global correlation among the parameters.

[0032] In some embodiments of the present application, the determining the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model comprises:

[0033] for a compute-intensive layer, the hierarchical hardware-aware adjustment factor is computed by the following formula:

[0034] R_hw(i) = 1.0 + 0.5 x log(1 + AI_i), wherein AI_i is the computation intensity of the i-th layer; for a memory-intensive layer, the hierarchical hardware-aware adjustment factor is computed by the following formula:

[0035] R_hw(i) = 0.5 + 0.5 x memory_efficiency, wherein memory_efficiency is a memory efficiency factor, and the computation formula is memory_efficiency = exp(-AI_i / 5);

[0036] The hierarchical hardware-aware adjustment factor is shared by the entire network layer rather than a single parameter.

[0037] In some embodiments of the present application, the dynamically allocated weight coefficients include:

[0038] The weight coefficients a, b, g are allocated according to the network layer type and depth proportion, where a is the weight coefficient of the Shor importance, b is the weight coefficient of the quantum correlation importance, and g is the weight coefficient of the hardware-aware adjustment factor, and a + b + g = 1; wherein:

[0039] For early convolutional layers of the ResNet series network with a depth proportion of 0-33%, a = 0.30, b = 0.30, and g = 0.40 are set;

[0040] For deep convolutional layers of the ResNet series network with a depth proportion of 66-100%, a = 0.50, b = 0.30, and g = 0.20 are set;

[0041] For early convolutional layers of the VGG network, a = 0.25, b = 0.35, and g = 0.40 are set.

[0042] In some embodiments of the present application, the pruning of the model parameters based on the collaborative importance includes:

[0043] Based on the requirement of the target compression rate, for each network layer, according to its layer type and depth proportion, the pruning rate target of the layer is set, wherein the layer type includes a computation-intensive layer and a memory-intensive layer, and the depth proportion is divided based on the total network depth;

[0044] In each layer, the parameters are sorted in descending order according to the collaborative importance;

[0045] From the sorted list, the model parameters or structural units with the lowest importance are removed in turn until the pruning rate target of the layer is reached;

[0046] The model accuracy and GPU inference delay of the deep learning model are measured after pruning;

[0047] If the accuracy drops more than a preset threshold or the inference delay improvement does not meet expectations, adjust the weight coefficients a, b, g, recalculate the collaborative importance, and perform the next round of pruning;

[0048] Iterate the pruning process until one of the following conditions is met: the target compression rate is reached, the accuracy drop is within an acceptable range, or the GPU inference delay improvement tends to saturate.

[0049] In some embodiments of the present application, the method further includes:

[0050] If the model accuracy drops more than a preset threshold or the GPU inference delay improvement does not reach the expected index after pruning, the model is fine-tuned using part of the training data for a limited number of iterations;

[0051] The accuracy recovery of the model during the fine-tuning process is monitored.

[0052] When the accuracy recovers to an acceptable range or the fine-tuning benefit tends to be saturated, the fine-tuning process is terminated.

[0053] The GPU inference delay and model accuracy of the final fine-tuned model are verified.

[0054] The second aspect of the embodiments of the application provides a deep learning model compression device based on GPU hardware perception collaborative optimization, characterized in that it comprises:

[0055] A hardware perception module is configured to obtain a deep learning model to be compressed and specification parameters of a target GPU hardware, construct a roofline performance model of the GPU hardware based on the specification parameters, analyze the calculation intensity of each network layer using the roofline performance model, and identify the network layer type according to the calculation intensity, wherein the network layer type includes a computation-intensive and a memory-intensive type.

[0056] An importance calculation module is configured to obtain model parameters of the deep learning model, determine the Fisher importance and quantum correlation importance of the model parameters, and determine a hierarchical hardware perception adjustment factor based on the network layer type and the roofline performance model.

[0057] A weight fusion module is configured to dynamically allocate weight coefficients according to the network layer type and the depth, and weight fuse the Fisher importance, the quantum correlation importance and the hardware perception adjustment factor to obtain the collaborative importance of the model parameters.

[0058] A pruning module is configured to prune the model parameters based on the collaborative importance to compress the deep learning model.

[0059] In summary, the deep learning model compression method and device based on GPU hardware perception collaborative optimization provided by the embodiments of the present application realize comprehensive analysis of parameter importance through multi-dimensional importance collaborative evaluation by integrating Fisher importance, quantum correlation importance and hardware perception adjustment factor. Because Fisher importance quantifies the local sensitivity of parameters to model loss, quantum correlation importance captures the global structural dependency between parameters, and the hardware perception adjustment factor introduces the consideration of GPU efficiency, the collaborative of the three ensures that the pruning decision considers both performance sensitivity and structural integrity, thereby significantly improving the accuracy retention rate and robustness of the pruned model. Through the dynamic adjustment technology based on the Roofline model, the deep integration of hardware perception and importance evaluation is realized. Because the Roofline model quantifies the computational intensity of each network layer based on the GPU hardware specifications (such as memory bandwidth and computing power), and dynamically generates an adjustment factor, the pruning strategy is optimized for memory-intensive or computationally-intensive layers, thereby ensuring a high degree of consistency between theoretical compression rate and actual acceleration effect. Through adaptive adjustment of weight coefficients α, β, and γ based on network layer type and depth ratio, adaptive optimization of different network layer characteristics is realized. Because the weight distribution is dynamically adjusted according to the type of network layer in the deep learning model, the contributions of local sensitivity, global correlation and hardware efficiency are balanced, thereby enhancing the universality of the method and improving the compression efficiency for various network architectures (such as VGG and ResNet). In addition, through iterative pruning and fine-tuning technology based on threshold dynamic adjustment and accuracy verification closed loop, the robust recovery of model performance is ensured. Because the performance is verified after pruning by measuring GPU inference delay and model accuracy, if the performance does not meet expectations, it is rolled back and optimized through fine-tuning (using a training subset for 1-2 epochs), thereby minimizing accuracy loss and realizing end-to-end compression process reliability. BRIEF DESCRIPTION OF DRAWINGS

[0060] The features and advantages of the present application will be more clearly understood through reference to the following drawings, which are presented as illustrative and should not be construed as limiting the present application, in which:

[0061] Figure 1 is a deep learning model compression algorithm framework based on GPU hardware perception collaborative optimization shown in the present application;

[0062] Figure 2 is a flowchart of a deep learning model compression method based on GPU hardware perception collaborative optimization according to some embodiments of the present application;

[0063] Figure 3 is a specific process of fine-tuning a pruned deep learning model according to some embodiments of the present application;

[0064] Figure 4 is a schematic diagram of a deep learning model compression device based on GPU hardware-aware collaborative optimization according to some embodiments of the present application. DETAILED DESCRIPTION

[0065] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the application. However, it will be apparent to one ordinarily skilled in the art that the application can be practiced without these details. In other instances, well-known methods have not been described in detail in order to avoid unnecessarily obscuring the present application. It will be appreciated that the terms "system," "device," "unit," and / or "module" as used herein are used in a generic sense and are not intended to limit the aspects of the present application to a particular device, unit, part, or assembly.

[0066] It should be understood that when a device, unit, or module is referred to as being "on", "connected to", or "coupled to" another device, unit, or module, it can be directly on, connected, or coupled to the other device, unit, or module, or intervening devices, units, or modules can be present. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0067] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0068] These and other features and characteristics of the present application, as well as the methods of operation and functions of the related elements of the structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the present application. As such, the

[0069] Various structural diagrams have been used in the present application to illustrate various modifications according to embodiments of the present application. It should be understood that the foregoing or following structures are not intended to limit the present application. The scope of protection of the present application is defined by the claims.

[0070] With the successful application of deep learning models in various tasks, the size and complexity of the models are also rapidly increasing. Although large-scale parameter models usually have higher performance, in the face of limited target hardware resources, the training and inference of the model will be affected to a certain extent, causing performance loss of the model. Deep learning model compression methods have important research significance in model optimization and deployment.

[0071] Model compression includes model pruning, model quantization, and knowledge distillation. The compression method of the present application refers to model pruning. Model pruning can significantly reduce the number of model parameters while minimizing the loss of model performance. Pruning is divided into structural pruning and non-structural pruning. Non-structural pruning relies on specific hardware and has limited scalability. Structural pruning is hardware-friendly and suitable for GPU acceleration of models.

[0072] However, the traditional pruning method does not fully consider the GPU hardware characteristics, resulting in a mismatch between the theoretical compression effect and the actual inference acceleration; the existing importance evaluation method uses the same strategy for different network layers, lacking layer-level adaptability; and a single importance index is difficult to comprehensively reflect the true value of the parameters.

[0073] To solve the above problems, the present application provides a deep learning model compression method based on GPU hardware perception collaborative optimization. The method calculates the parameter importance through parameter-level evaluation (including parameter statistical sensitivity evaluation and parameter structure correlation evaluation) and layer-level GPU hardware feature analysis, and then collaboratively optimizes the parameters to achieve efficient pruning of the model, aiming to reduce the number of model parameters while minimizing the loss of model accuracy, and ultimately achieve inference acceleration of the model. Figure 1 is the deep learning model compression algorithm framework based on GPU hardware perception collaborative optimization shown in the present application. In some embodiments, the deep learning model compression method based on GPU hardware perception collaborative optimization as shown in Figure 2 includes the following steps:

[0074] S210, obtaining a deep learning model to be compressed and specification parameters of target GPU hardware, constructing a roofline performance model of the GPU hardware based on the specification parameters, and analyzing the calculation intensity of each network layer using the roofline performance model, and identifying the network layer type according to the calculation intensity, wherein the network layer type includes computation-intensive and memory-intensive.

[0075] Specifically, first, the hardware performance of the target GPU is analyzed. This includes obtaining the specification parameters of the target GPU hardware, including memory bandwidth, peak computing power, and cache structure, and then constructing a roofline performance model (Roofline model) of the GPU hardware based on the specification parameters, which is used to quantify the calculation intensity of each network layer.

[0076] In some embodiments of the present application, the calculation intensity of the network layer i The calculation formula is:

[0077]

[0078] Wherein, is the number of floating point operations of the network layer i, is the total amount of memory read and write of the network layer i;

[0079] The present application identifies the type of the network layer by analyzing the calculation intensity (AI value) of each network layer. In some embodiments of the present application, when the AI value is less than 5, the network layer is classified as a memory-intensive layer; when the AI value is greater than 20, the network layer is classified as a calculation-intensive layer.

[0080] S220, obtaining the model parameters of the deep learning model, determining the Fisher importance and quantum correlation importance of the model parameters, and determining the hierarchical hardware-aware adjustment factor based on the network layer type and the roof line performance model.

[0081] The present application evaluates the statistical sensitivity of the deep learning model parameters by Fisher importance. Specifically, first, the parameters are grouped according to the dependency of the model parameters, the Hessian matrix of each group of model parameters is calculated, and the eigenvalues are extracted from the Hessian matrix to generate the Fisher importance. Fisher importance is used to represent the statistical sensitivity of parameters to model loss. The statistical sensitivity dimension evaluates the direct contribution of each parameter to the model performance from the optimization gradient level, identifies the key parameters most sensitive to the model output by quantifying the influence of parameter perturbation on the loss function.

[0082] In some embodiments of the present application, the calculation of Fisher importance is specifically:

[0083] The Hessian matrix of the model parameters is calculated, and the specific formula is as follows:

[0084]

[0085] The Fisher information is calculated, and the specific formula is as follows:

[0086]

[0087] Wherein is the number of samples, is the number of the i-th sample, is the Hessian matrix on the sample .

[0088] The Fisher importance representing the statistical sensitivity of the parameters is calculated, and the formula is as follows:

[0089] I_Fisher

[0090] where λi is the i-th eigenvalue of the Fisher matrix F.

[0091] The application evaluates the global correlation between the parameters of a deep learning model through quantum correlation importance. Specifically, the parameters of the model to be pruned are mapped to quantum states, the correlation strength between the parameters is calculated, and the importance of the structural correlation between the parameters is obtained. The global correlation dimension evaluates the global correlation pattern of the parameters in the network from the parameter distribution relationship level, identifies the synergistic effect and dependency relationship between the parameters through quantum state inner product calculation, and ensures the integrity of the network structure after pruning.

[0092] In some embodiments of the application, the calculation of quantum correlation importance is specifically:

[0093] First, the parameter quantum state mapping is performed, and the calculation formula is as follows:

[0094]

[0095] where , , is the i-th parameter of the model.

[0096] Then, the correlation strength between the parameters is calculated based on the results of the quantum state mapping, and the calculation formula is as follows:

[0097]

[0098] The expansion formula is as follows:

[0099]

[0100] Finally, the importance of the correlation between the parameters is calculated as follows:

[0101] .

[0102] The application determines a hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model. The hardware-aware adjustment factor is used to represent the influence of hardware execution efficiency on model compression decision.

[0103] In an embodiment of the application, the specific calculation formula of the hierarchical hardware-aware adjustment factor is as follows:

[0104] Computationally intensive layer:

[0105] Memory-intensive layer:

[0106] Wherein: R_hw(i) is the adjustment factor for the ith layer, all parameters of the layer share this adjustment factor, AI_i is the calculation intensity of the ith layer, and the memory_efficiency is a memory efficiency factor, and the calculation formula is memory_efficiency=exp(-AI_i / 5);

[0107] In S230, the Fisher importance, the quantum correlation importance, and the hardware-aware adjustment factor are weighted and fused according to the network layer type and the depth to obtain the collaborative importance of the model parameter.

[0108] The present application prunes the model based on the collaborative importance. The collaborative importance is the weighted sum of the Fisher importance, the quantum correlation importance, and the hardware-aware adjustment factor.

[0109] The present application dynamically allocates the weight coefficients according to the network layer type and the depth ratio. The dynamic allocation of the three weights satisfies the constraint: α+β+γ=1, wherein α is the Fisher importance weight, β is the quantum correlation importance weight, and γ is the hardware-aware adjustment factor weight.

[0110] In some embodiments of the present application, the adaptive rule of weight allocation is:

[0111] If the calculation intensity AI of the network layer is greater than 20 (computation-intensive): γ is reduced by 0.05-0.10;

[0112] If the calculation intensity AI of the network layer is less than 5 (memory-intensive): γ is increased by 0.05-0.15

[0113] If the network layer contains residual connection: β is increased by 0.05-0.10

[0114] If the network layer is close to the classification head (depth>90%): α is increased by 0.05-0.15

[0115] In some other embodiments of the present application, different weight allocation strategies are adopted for different deep calculation neural network models, such as:

[0116] ResNet series network:

[0117] Early convolutional layer (network depth 0-33%): α=0.30, β=0.30, γ=0.40

[0118] Principle: Early layers extract basic features (edges, textures), and hardware efficiency has a great impact, so GPU execution efficiency is preferred

[0119] Intermediate convolutional layer (network depth 33-66%): α=0.40, β=0.40, γ=0.20

[0120] Principle: middle layers learn middle-level semantic features, both parameter sensitivity and structural relevance, balance performance and structure

[0121] Deep convolutional layers (network depth 66-100%): α=0.50, β=0.30, γ=0.20

[0122] Principle: deep layers learn high-level semantics, which have a large impact on final classification, and focus more on the performance contribution of parameters

[0123] Fully connected layers: α=0.40, β=0.20, γ=0.40

[0124] Principle: fully connected layers have large parameter quantities and are memory-intensive, both hardware efficiency and performance, and relatively weak structure dependence

[0125] VGG series network:

[0126] Early convolutional layers (network depth 0-33%): α=0.25, β=0.35, γ=0.40

[0127] Principle: VGG structure is simple but has many layers, early layers focus more on structural integrity and hardware efficiency

[0128] Middle convolutional layers (network depth 33-66%): α=0.35, β=0.40, γ=0.25

[0129] Principle: middle layers are both preceding and following, with the most important structural relevance

[0130] Deep convolutional layers (network depth 66-100%): α=0.45, β=0.35, γ=0.20

[0131] Principle: deep semantic features are important, and Fisher weights are appropriately increased

[0132] Fully connected layers: α=0.40, β=0.20, γ=0.40

[0133] Principle: VGG's FC layer parameters account for a large proportion, and hardware efficiency weights are large

[0134] Weight adjustment principles:

[0135] Computationally intensive layers: reduce γ value (0.1-0.2), as these layers have high GPU utilization

[0136] Memory-intensive layers: increase γ value (0.3-0.5), and prefer pruning to obtain significant acceleration

[0137] Residual connection layers: increase β value (0.4-0.5) to maintain structural integrity

[0138] Classification head layer: increase alpha value (0.5-0.6) to ensure classification performance

[0139] S240, pruning the model parameters based on the collaborative importance to compress the deep learning model.

[0140] In some embodiments of the present application, pruning the model parameters based on the collaborative importance to compress the model includes:

[0141] First, set the overall target compression rate according to the application scenario, for example, reduce the parameter amount by 50-70%, as a global constraint for the pruning process.

[0142] Based on the requirement of the overall target compression rate, for each network layer, set the pruning rate target of the layer according to its layer type (computation-intensive or memory-intensive) and depth ratio (based on the total network depth division);

[0143] In each layer, sort the parameters in descending order according to the collaborative importance;

[0144] Remove the model parameters or structural units with the lowest importance from the sorted list one by one until the pruning rate target of the layer is reached;

[0145] Measure the model accuracy and GPU inference delay of the deep learning model after pruning;

[0146] If the accuracy decreases by more than a preset threshold or the inference delay improvement does not meet expectations, adjust the weight coefficients α, β, γ, recalculate the collaborative importance and perform the next round of pruning;

[0147] Iterate the pruning process until one of the following conditions is met: the target compression rate is reached, the accuracy decrease is within an acceptable range, or the GPU inference delay improvement tends to saturate.

[0148] In some embodiments, the pruning rate target is set with reference to: ResNet early convolutional layer 40-50%, deep convolutional layer 60-70%; VGG fully connected layer 65-75%. Those skilled in the art can adjust according to actual needs.

[0149] The present application finally completes collaborative pruning by integrating the importance evaluation results of the three dimensions.

[0150] This collaborative mechanism ensures:

[0151] The Fisher importance dimension ensures that the performance loss of the pruned model is minimized;

[0152] The quantum correlation dimension ensures the integrity of the network structure after pruning;

[0153] The hardware efficiency dimension ensures the actual acceleration of GPU inference after pruning.

[0154] In some embodiments of the present disclosure, the pruned deep learning model is verified for various index results, and the verification results are used as feedback to iteratively prune the model.

[0155] In some embodiments of the present disclosure, the method further comprises:

[0156] The pruned deep learning model is fine-tuned to further restore the accuracy of the model.

[0157] Figure 3 is a specific process of fine-tuning the pruned deep learning model according to some embodiments of the present disclosure. As shown in Figure 3 The fine-tuning includes:

[0158] The performance indicators of the pruned deep learning model are measured, such as GPU inference delay and model accuracy;

[0159] According to the verification results, if the performance does not meet the expected indicators, the model is fine-tuned;

[0160] The fine-tuning process uses a partial subset of the training set for 1-2 epochs of iteration;

[0161] After fine-tuning, the GPU inference delay and model accuracy are verified again, and if the performance is still not up to standard, the fine-tuning process is repeated;

[0162] Iterative fine-tuning is performed until the performance is restored to an acceptable range or the fine-tuning benefit tends to be saturated.

[0163] Through fine-tuning, the pruned deep learning model can restore most of the performance of the original model. Finally, the deep learning model completes model compression with acceptable performance loss and can be deployed on performance-limited devices.

[0164] Figure 4 is a schematic diagram of a deep learning model compression device based on GPU hardware-aware collaborative optimization according to some embodiments of the present disclosure. As shown in Figure 4 The deep learning model compression device 400 includes a hardware-aware module 410, an importance calculation module 420, a weight fusion module 430, and a pruning module 440. Among them:

[0165] The hardware-aware module 410 is configured to obtain a deep learning model to be compressed and specification parameters of a target GPU hardware, construct a roofline performance model of the GPU hardware based on the specification parameters, analyze the calculation intensity of each network layer using the roofline performance model, and identify the network layer type according to the calculation intensity, wherein the network layer type includes computation-intensive and memory-intensive;

[0166] The importance calculation module 420 is configured to acquire model parameters of the deep learning model, determine Fisher importance and quantum correlation importance of the model parameters, and determine a hierarchical hardware-aware adjustment factor based on the network layer type and the Roofline performance model.

[0167] The weight fusion module 430 is configured to dynamically allocate weight coefficients according to the network layer type and the depth, and fuse the Fisher importance, the quantum correlation importance and the hardware-aware adjustment factor to obtain cooperative importance of the model parameters.

[0168] The pruning module 440 is configured to prune the model parameters based on the cooperative importance to compress the deep learning model.

[0169] In summary, the deep learning model compression method and device based on GPU hardware-aware cooperative optimization provided by the embodiments of the present application realize comprehensive analysis of parameter importance through multi-dimensional importance cooperative evaluation by integrating Fisher importance, quantum correlation importance and hardware-aware adjustment factor. Because the Fisher importance quantifies the local sensitivity of parameters to model loss, the quantum correlation importance captures the global structural dependency between parameters, and the hardware-aware adjustment factor introduces the consideration of GPU efficiency, the three cooperate to ensure that the pruning decision considers both performance sensitivity and structural integrity, thereby significantly improving the accuracy retention rate and robustness of the pruned model. Through the dynamic adjustment technology based on the Roofline model, the deep integration of hardware awareness and importance evaluation is realized. Because the Roofline model quantifies the computational intensity of each network layer based on the GPU hardware specifications (such as memory bandwidth and computing power), and dynamically generates an adjustment factor, the pruning strategy is optimized for memory-intensive or computation-intensive layers, thereby ensuring high consistency between theoretical compression rate and actual acceleration effect. Through adaptive adjustment of weight coefficients (α, β, γ) based on network layer type and depth, adaptive optimization of different network layer characteristics is realized. Because the weight allocation is dynamically adjusted according to the network layer type of the deep learning model, the contributions of local sensitivity, global correlation and hardware efficiency are balanced, thereby enhancing the universality of the method, which is applicable to various network architectures (such as VGG and ResNet), and improving the compression efficiency. In addition, through the iterative pruning and fine-tuning technology based on threshold dynamic adjustment and accuracy verification closed loop, the present application ensures the robust recovery of model performance. Because the model is verified after pruning by measuring GPU inference delay and model accuracy, if the performance does not meet the expectation, it is rolled back and optimized through fine-tuning (using a training subset for 1-2 epochs), thereby minimizing the accuracy loss and realizing the reliability of the end-to-end compression process.

[0170] Although the subject matter described herein is provided in the general context of computer-executable instructions of a program module being executed by a computer system on a computing device, those skilled in the art will recognize that programs modules can be implemented in cooperation with other programs modules or not, and computer system or systems can be implemented as a combination of computing devices executing programs modules, as at least partially embodied in a microprocessor-based or in programmable consumer electronics, as at least partially embodied in a minicomputer, mainframe computer, and the like. Still further, those skilled in the art will appreciate that one or more computer systems with associated computer program modules / members can be deployed, while software modules implemented as computer program modules can be stored on a computer usable medium almost anywhere on a given computer system — from a hard disk to a computer data storage media. Also, those skilled in the art will appreciate that the subject matter described herein can be implemented as a computer program product, i.e., a computer program tangibly embodied in a computer readable storage medium.

[0171] Those skilled in the art can realize also that the exemplary units and method steps described in connection with the embodiments disclosed herein can be carried out by electronic hardware, computer software, or combinations of both. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0172] It should be understood that the above specific embodiments are only used to illustrate or explain the principles of the present application, and do not constitute a limitation on the present application. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present application shall be included in the protection scope of the present application. In addition, the claims of the present application are intended to cover all variations and modifications falling within the scope and boundary of the appended claims, or the equivalent forms of such scope and boundary.

Claims

1. A method for deep learning model compression based on GPU hardware-aware collaborative optimization, characterized in that, The method comprises the following steps: obtaining a deep learning model to be compressed and specification parameters of target GPU hardware, constructing a roofline performance model of the GPU hardware based on the specification parameters, analyzing the calculation intensity of each network layer by using the roofline performance model, and identifying the network layer type according to the calculation intensity, wherein the network layer type comprises a computation-intensive layer and a memory-intensive layer; obtaining model parameters of the deep learning model, determining the Fisher importance and quantum correlation importance of the model parameters, and determining a hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model; dynamically allocating weight coefficients according to the network layer type and depth, and weighting and fusing the Fisher importance, quantum correlation importance and hardware-aware adjustment factor to obtain the cooperative importance of the model parameters; pruning the model parameters based on the cooperative importance to compress the deep learning model.

2. The method of claim 1, wherein : The specification parameters comprise memory bandwidth, peak calculation capability and cache hierarchy.

3. The method of claim 1, wherein, The calculation intensity of each network layer is analyzed by using the roofline performance model, and the network layer type is identified according to the calculation intensity, which comprises the following steps: the calculation intensity AI_i of the network layer i is calculated based on the following formula: ； wherein, is the number of floating point operations of network layer i, is the total memory read / write of network layer i; when the AI value is less than 5, the network layer is classified as a memory-intensive layer; when the AI value is greater than 20, the network layer is classified as a computation-intensive layer.

4. The method of claim 1, wherein, The Fisher importance of the model parameters comprises the following steps: grouping the parameters according to the dependency of the model parameters; calculating the Hessian matrix of each group of parameters, and the elements of the Hessian matrix are composed of the second-order partial derivatives of the loss function with respect to the parameters; extracting eigenvalues from the Hessian matrix to generate the Fisher importance, wherein the Fisher importance is used to represent the statistical sensitivity of the parameters to the model loss.

5. The method of claim 1, wherein, The quantum correlation importance of the model parameters comprises the following steps: mapping the model parameters to quantum states according to the following formula: ， wherein , , is the ith model parameter; calculating the parameter correlation strength based on the mapping result, and the correlation strength calculation formula is: ， Summing over all the correlation strengths and dividing by the number of samples N, the quantum correlation importance is generated, i.e. , wherein the quantum correlation importance is used to represent the global correlation between parameters.

6. The method of claim 3, wherein, The hierarchical hardware-aware adjustment factor is determined based on the network layer type and the roofline performance model, which comprises the following steps: for the computation-intensive layer, the hierarchical hardware-aware adjustment factor is calculated by the following formula: R_hw(i)=1.0+0.5×log(1+AI_i), wherein AI_i is the calculation intensity of the i-th layer; for the memory-intensive layer, the hierarchical hardware-aware adjustment factor is calculated by the following formula: R_hw(i)=0.5+0.5×memory_efficiency, wherein memory_efficiency is a memory efficiency factor, and the calculation formula is memory_efficiency=exp(-AI_i / 5); wherein the hierarchical hardware-aware adjustment factor is shared for the entire network layer, rather than a single parameter.

7. The method of claim 1, wherein, The weight coefficients are dynamically allocated according to the network layer type and depth, which comprises the following steps: The weight coefficients α, β, and γ are allocated according to the network layer type and depth proportion, where α is the weight coefficient of Fisher importance, β is the weight coefficient of quantum correlation importance, and γ is the weight coefficient of hardware-aware adjustment factor, and α+β+γ=1; wherein: For the early convolutional layers of the ResNet series network with a depth proportion of 0-33%, α=0.30, β=0.30, and γ=0.40 are set; For the deep convolutional layers of the ResNet series network with a depth proportion of 66-100%, α=0.50, β=0.30, and γ=0.20 are set; For the early convolutional layers of the VGG network, α=0.25, β=0.35, and γ=0.40 are set.

8. The method of claim 1, wherein, The pruning of the model parameters based on the collaborative importance includes: Based on the demand for target compression rate, for each network layer, the pruning rate target of the layer is set according to its layer type and depth proportion, where the layer type includes computation-intensive layers and memory-intensive layers, and the depth proportion is divided based on the total network depth; In each layer, the parameters are sorted in descending order according to the collaborative importance; The model parameters or structural units with the lowest importance are removed from the sorted list one by one until the pruning rate target of the layer is reached; The model accuracy and GPU inference delay of the deep learning model are measured after pruning; If the accuracy decreases by more than a preset threshold or the inference delay improvement does not meet the expected value, the weight coefficients α, β, and γ are adjusted, the collaborative importance is recalculated, and the next round of pruning is performed; The iterative pruning process is continued until one of the following conditions is met: the target compression rate is reached, the accuracy decrease is within an acceptable range, or the GPU inference delay improvement tends to be saturated.

9. The method of claim 1, wherein, The method further includes: If the model accuracy decreases by more than a preset threshold or the GPU inference delay improvement does not meet the expected value after pruning, the model is fine-tuned using part of the training data for a limited number of iterations; The accuracy recovery of the model during the fine-tuning process is monitored; When the accuracy is restored to an acceptable range or the fine-tuning benefit tends to be saturated, the fine-tuning process is terminated; The GPU inference delay and model accuracy of the final fine-tuned model are verified.

10. A deep learning model compression device based on GPU hardware-aware collaborative optimization, characterized in that, It includes: A hardware-aware module for obtaining a deep learning model to be compressed and the specification parameters of a target GPU hardware, constructing a roofline performance model of the GPU hardware based on the specification parameters, analyzing the computation intensity of each network layer using the roofline performance model, and identifying the network layer type according to the computation intensity, where the network layer type includes computation-intensive and memory-intensive layers; An importance calculation module for obtaining the model parameters of the deep learning model, determining the Fisher importance and quantum correlation importance of the model parameters, and determining the hierarchical hardware-aware adjustment factor based on the network layer type and the roofline performance model; A weight fusion module for dynamically allocating weight coefficients according to the network layer type and depth, and weighting and fusing the Fisher importance, quantum correlation importance, and hardware-aware adjustment factor to obtain the collaborative importance of the model parameters; A pruning module for pruning the model parameters based on the collaborative importance to compress the deep learning model.

Images