An AI model compression method and system driven by dynamic threshold

Abstract

The present application relates to the technical field of artificial intelligence, and provides a dynamic threshold driven AI model compression method and system. The dynamic threshold driven AI model compression method comprises designing an adaptive and dynamically adjusted pruning threshold computer mechanism. The mechanism discards the traditional mode relying on fixed experience parameters, and adopts a more robust median absolute deviation method in statistics as the basis to automatically calculate a data-driven initial pruning threshold for each network layer. The above dynamic pruning mechanism is embedded into a multi-stage progressive execution framework. Between each stage, the system performs a lightweight model fine-tuning, ensuring that the model can maintain stable performance even when the impact of high sparsity is high, thereby effectively solving the technical problem of performance drop.

Description

Technical Field

[0001] This invention relates to the fields of convolutional neural networks and artificial intelligence, and in particular to a dynamic threshold-driven AI model compression method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Artificial intelligence (AI) technology has become a key measure to improve customs supervision efficiency and strengthen national border security. AI systems can perform rapid and accurate automated analysis of massive amounts of trade documents and identify high-risk transactions. However, to achieve this goal, AI models must be deployable on various resource-constrained hardware devices, such as servers in port business systems and mobile inspection terminals used by frontline customs officers. These devices are often limited by computing power, memory size, and power consumption, making it impossible to support the most accurate, but also typically largest and most complex, deep learning models currently available.

[0004] Therefore, lightweighting high-precision AI models to enable them to run efficiently on resource-constrained devices is an urgent and necessary requirement in the current construction of smart customs, but existing technologies still have significant shortcomings in this regard.

[0005] Existing compression model methods generally lack sufficient adaptability and intelligence. Many techniques, especially pruning methods, often rely on fixed empirical rules or global hyperparameters that require extensive experimentation and repeated tuning by domain experts when deciding which network parameters to remove. This "one-size-fits-all" static strategy ignores the complexity and heterogeneity of the internal structure of AI models. It cannot implement differentiated and refined compression strategies based on the functional differences of different network layers in the model (e.g., some layers are responsible for capturing grammatical structures and lexical associations, while others are responsible for understanding deep semantics and logical relationships) and their unique weight distribution characteristics, making it difficult to achieve an optimal balance between compression efficiency and performance preservation.

[0006] Under high compression ratio targets, the performance bottlenecks of existing technologies become particularly prominent. To achieve significant inference acceleration on edge devices, models often need to be deeply compressed, for example, removing more than 70% of the parameters. However, many methods that employ a "one-off" compression strategy suffer a sharp or even catastrophic drop in model performance under such drastic changes to the model structure. This directly impacts the accuracy of identifying high-risk transactions and the accuracy of commodity classification, which is unacceptable in security-critical scenarios. Although large-scale retraining can recover some of the lost accuracy, its high time and computational resource costs contradict the real-world requirements of rapid response and agile deployment in customs operations, making this technical path extremely difficult to implement in practice. Summary of the Invention

[0007] To address the technical problems mentioned above, this invention provides a dynamic threshold-driven AI model compression method and system. This invention integrates adaptive computation, multi-dimensional dynamic adjustment, and progressive execution, aiming to achieve efficient model compression in a refined and performance-friendly manner, thereby reducing the computational and storage costs of hardware devices.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] The first aspect of the present invention provides a dynamic threshold-driven AI model compression method.

[0010] A dynamic threshold-driven AI model compression method, comprising:

[0011] Obtain the large language model to be compressed and the customs business calibration dataset, and define a sparsity rate sequence that progresses from low to high;

[0012] Following the sparsity rate sequence, at each pruning stage, all pruningable layers in the large language model are traversed. The customs business calibration dataset is input into the large language model to obtain the input activation matrix of each layer. Based on the weight matrix and input activation matrix of each layer, the importance score matrix of each layer is calculated. Based on the importance score matrix of each layer, a baseline threshold is calculated. A dynamic factor is introduced to adjust the coefficient of the baseline threshold, and the final dynamic threshold of each layer is calculated. Based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated. According to the outlier ratio of each layer, the initial sparsity rate of each layer is calculated, constraints are applied, and the sparsity rate index of each layer is obtained. Based on the sparsity rate index of each layer, the number of weights that need to be subtracted from each layer is calculated.

[0013] Select a threshold in the importance score matrix to create a binary mask matrix; if the importance score of a certain layer is greater than or equal to the threshold, retain the position corresponding to the mask; otherwise, remove the position corresponding to the mask; thus obtaining the compressed large language model.

[0014] Furthermore, the sparsity rate sequence represents the sparsity rate that the large language model needs to achieve at each pruning stage, and the sparsity rate sequence includes the target sparsity rate.

[0015] Furthermore, before traversing the large language model, the process includes defining key business metrics for evaluating the performance of the large language model, including: F1 score for high-risk message recognition, accuracy of commodity HS code classification, and document processing latency; these key business metrics are used to evaluate the performance of the pruned large language model.

[0016] Furthermore, based on the importance score matrix of each layer, a baseline threshold is calculated; a dynamic multiplier is introduced to adjust the coefficient of the baseline threshold, and the final dynamic threshold for each layer is calculated; the method includes:

[0017] Based on the importance score matrix of each layer, calculate the median and MAD value;

[0018] Calculate the baseline threshold based on the median and MAD value;

[0019] Introducing dynamic multiples ,in, It is a base multiple; , , All represent adjustment factors;

[0020] By constraining the dynamic multiple within a set range, the final dynamic multiple is obtained.

[0021] Calculate the final dynamic threshold for each layer ;in, This represents the median; Indicates the MAD value; This indicates the final dynamic multiple.

[0022] Furthermore, based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated; the method includes:

[0023] Iterate through each score in the importance score matrix of each layer and compare it with the final dynamic threshold of each layer. If the score is greater than the final dynamic threshold, it is identified as an outlier. Count the total number of outliers in each layer.

[0024] The outlier ratio for each layer is calculated based on the ratio of the total number of outliers to the total number of parameters in that layer.

[0025] Furthermore, based on the outlier ratio of each layer, the initial sparsity of each layer is calculated, constraints are applied, and the sparsity index of each layer is obtained; the method includes:

[0026] Based on the outlier ratio of each layer, an initial sparsity rate is set for each layer that is inversely proportional to its importance. ;in, express l Outlier ratio of the layer;

[0027] Constrain the initial sparsity of each layer to Within this layer, the sparsity index is obtained for each layer; among them, λ represents the target sparsity; λ represents the hyperparameter.

[0028] Furthermore, a threshold is selected in the importance score matrix to create a binary mask matrix; the method includes:

[0029] Sort all importance scores in the importance score matrix and select the first one. A number of scores are used as thresholds; among which, express l The number of weights that need to be subtracted from the layer;

[0030] Create a binary mask matrix with the exact same dimensions as the weight matrix of that layer in each layer.

[0031] A second aspect of the present invention provides a dynamic threshold-driven AI model compression system.

[0032] A dynamic threshold-driven AI model compression system, comprising:

[0033] The preparation module is configured to: acquire the large language model to be compressed and the customs business calibration dataset, and define a sparse rate sequence that progresses from low to high.

[0034] The traversal calculation module is configured to: traverse all pruningable layers in the large language model according to the sparsity rate sequence at each pruning stage; input the customs business calibration dataset into the large language model to obtain the input activation matrix of each layer; calculate the importance score matrix of each layer based on the weight matrix and input activation matrix of each layer; calculate the baseline threshold based on the importance score matrix of each layer; introduce a dynamic factor to adjust the coefficient of the baseline threshold and calculate the final dynamic threshold of each layer; filter outliers in the importance score matrix of each layer based on the final dynamic threshold of each layer and calculate the outlier ratio of each layer; calculate the initial sparsity rate of each layer based on the outlier ratio of each layer, apply constraints to obtain the sparsity rate index of each layer; and calculate the number of weights to be subtracted from each layer based on the sparsity rate index of each layer.

[0035] The model compression module is configured to: select a threshold in the importance score matrix and create a binary mask matrix; if the importance score of a certain layer is greater than or equal to the threshold, retain the position corresponding to the mask; otherwise, remove the position corresponding to the mask; and obtain the compressed large language model.

[0036] A third aspect of the present invention provides a computer device comprising:

[0037] A processor, adapted to execute computer programs;

[0038] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the dynamic threshold-driven AI model compression method described in the first aspect above.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and to execute steps in the dynamic threshold-driven AI model compression method as described in the first aspect above.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] This invention designs an adaptive, dynamically adjustable pruning threshold calculation mechanism. This mechanism abandons the traditional model that relies on fixed empirical parameters and is based on the more statistically robust Median Absolute Deviation (MAD) method to automatically calculate a data-driven initial pruning threshold for each network layer. This threshold is not static but is dynamically adjusted in real time and intelligently based on the following three dimensions of information: (1) Data distribution characteristics: The system analyzes the statistical characteristics of the weights of each layer, such as skewness and kurtosis, to perceive their distribution pattern and adjust the conservatism of pruning accordingly; (2) Hierarchical structure characteristics: The system can identify the position (such as shallow or deep) and type (such as attention layer or MLP layer) of the layer in the network and automatically adopt a more cautious pruning strategy for more important and sensitive layers; (3) Model macroscopic performance: Through a closed-loop feedback mechanism, the aggressiveness of pruning is linked to the real-time performance of the model on the validation set (such as accuracy and loss value). When the performance decreases, the system will automatically raise the pruning threshold.

[0042] This invention embeds the aforementioned dynamic pruning mechanism into a multi-stage progressive execution framework. To avoid the drastic impact on the model caused by a "one-time" high-ratio pruning, this invention decomposes the overall compression target into multiple progressive, smooth stages (e.g., sparsity from 20% to 50%, then to 70%). Between each stage, the system performs a lightweight model fine-tuning, giving the network sufficient "breathing" and "recovery" time to adapt to the new sparse structure. This iterative "pruning-recovery-re-pruning" cycle ensures that the model's performance remains stable even under high sparsity impacts, effectively solving the technical problem of sudden performance drops and reducing the computational and storage costs of hardware devices. Attached Figure Description

[0043] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0044] Figure 1 This is a flowchart illustrating a dynamic threshold-driven AI model compression method according to an embodiment of the present invention;

[0045] Figure 2This is a flowchart of another embodiment of the dynamic threshold-driven AI model compression method shown in the embodiments of the present invention;

[0046] Figure 3 This is a structural diagram of a dynamic threshold-driven AI model compression system shown in an embodiment of the present invention;

[0047] Figure 4 This is a structural diagram of a computer device shown in an embodiment of the present invention. Detailed Implementation

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0050] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0051] As introduced in the background section, existing model compression technologies, when applied to intelligent text analysis scenarios requiring high precision and reliability, exhibit significant and irreconcilable shortcomings in three key dimensions: adaptability, fine-grained processing, and performance maintenance at high compression rates. Regarding adaptability, current mainstream methods generally exhibit a "static" and "rigid" characteristic. They mostly rely on preset, globally unified compression rules or hyperparameters that require repeated manual trial and error adjustments. This "one-size-fits-all" strategy cannot dynamically adapt to the differences in weight distribution brought about by different analysis models (such as different backbone networks) and different datasets (such as different types of trade documents and commodity descriptions), making them inadequate in complex practical applications and unable to automatically find the optimal balance between compression and performance. In terms of fine-grained processing, existing technologies often treat neural network models as a homogeneous whole, lacking in-depth insight into the functional heterogeneity within the model. A deep model used for customs document understanding has different levels and modules playing drastically different roles—from parsing vocabulary and grammar to understanding complex trade terms and logic, the sensitivity and importance of pruning vary drastically from part to part. However, existing methods fail to establish an effective mechanism to perceive and utilize these internal characteristics, hindering differentiated, "surgical" compression of the model and thus limiting its compression potential without sacrificing critical functionality. Maintaining performance at high compression ratios also presents significant challenges. Achieving substantial speed improvements on edge devices often requires extremely high model compression ratios. However, many aggressive compression methods, once a certain threshold is reached, cause a precipitous drop in model accuracy, which is fatal for security-critical customs inspections. Solutions relying on large-scale retraining, with their high resource and time costs, run counter to the demands of agile business deployments.

[0052] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is: how to break through the limitations of current technological paradigms and construct a new lightweight model method that integrates intelligent perception, dynamic adjustment, and progressive recovery. This method must be able to comprehensively address the challenges at the three levels mentioned above: it needs to automatically perceive and adapt to the microscopic characteristics within the model (adaptability); it needs to implement optimal and differentiated processing for different modules based on these characteristics (refinement); and it needs to guide the model to robustly achieve extremely high compression rates in a smooth and performance-friendly manner, ultimately effectively supporting the reliable and efficient large-scale deployment of high-performance customs business text analysis AI models on various resource-constrained hardware platforms.

[0053] Figure 1 This is a flowchart illustrating a dynamic threshold-driven AI model compression method according to an embodiment of the present invention; see also... Figure 1 The method includes:

[0054] Obtain the large language model to be compressed and the customs business calibration dataset, and define a sparsity rate sequence that progresses from low to high;

[0055] Following the sparsity rate sequence, at each pruning stage, all pruningable layers in the large language model are traversed. The customs business calibration dataset is input into the large language model to obtain the input activation matrix of each layer. Based on the weight matrix and input activation matrix of each layer, the importance score matrix of each layer is calculated. Based on the importance score matrix of each layer, a baseline threshold is calculated. A dynamic factor is introduced to adjust the coefficient of the baseline threshold, and the final dynamic threshold of each layer is calculated. Based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated. According to the outlier ratio of each layer, the initial sparsity rate of each layer is calculated, constraints are applied, and the sparsity rate index of each layer is obtained. Based on the sparsity rate index of each layer, the number of weights that need to be subtracted from each layer is calculated.

[0056] Select a threshold in the importance score matrix to create a binary mask matrix; if the importance score of a certain layer is greater than or equal to the threshold, retain the position corresponding to the mask; otherwise, remove the position corresponding to the mask; thus obtaining the compressed large language model.

[0057] This invention provides a groundbreaking solution for language model compression in demanding scenarios such as customs operations by combining a unique dynamic adaptive pruning mechanism with a multi-stage progressive framework. Its core dynamic MAD threshold mechanism can perceive in real time the distribution characteristics of customs text data, hierarchical differences within the model, and changes in key business indicators (such as risk identification F1 scores). This intelligently "tailors" the optimal, differentiated compression strategy for the model, completely eliminating the reliance on fixed empirical parameters and the "one-size-fits-all" drawbacks of traditional methods. More importantly, its multi-stage progressive pruning and lightweight inter-stage fine-tuning "pruning-healing" mode effectively avoids the performance collapse risk caused by "one-time" high-ratio compression. This allows customs business language models to achieve sparsity rates of 70% or even higher and several times faster inference while maximizing their audit accuracy, truly achieving a "win-win" situation of processing speed and business precision. Ultimately, the method's high degree of automation and efficient performance recovery capabilities not only accelerate the iteration and deployment of customs AI applications, but also enable high-performance models to run on lower-cost hardware, significantly reducing the overall application cost of building smart customs.

[0058] Figure 2 This is a flowchart of another embodiment of the dynamic threshold-driven AI model compression method shown in this invention; see also... Figure 2 The method includes the following steps:

[0059] Step 1: Load a large, uncompressed language model that has been fine-tuned using customs business data (such as historical customs declarations, trade regulations, manifest information, etc.). Simultaneously, prepare a representative calibration dataset, which should include text samples of various customs declarations, invoices, contracts, etc.

[0060] In some embodiments, the specific implementation process of step one includes:

[0061] Select and load a high-starting-point base model, and build a high-fidelity calibration dataset.

[0062] Regarding the base model, we selected to load a large, uncompressed language model that has been deeply optimized for customs operations. This model is based on a general-purpose model and has been specifically fine-tuned using massive amounts of anonymized historical customs data (such as customs declarations, trade regulations, and commodity catalogs). The reason for choosing such a domain-specific model as the starting point is that its internal parameters have learned and solidified a deep understanding of customs-specific terminology, document formats, and business logic. We loaded its original, dense version containing all parameters. The performance of this version represents the "ceiling" that subsequent compression aims to achieve and serves as a performance benchmark for measuring the compression effect.

[0063] Regarding the calibration dataset, a representative set of text samples that fully reflects the real-world business environment needs to be prepared. This dataset is not used for training, but rather serves as a "probe" and "measuring stick" for subsequent pruning steps. Its core purpose is to simulate the data flow that the model will process in actual work, so that the dynamic contribution of each part of the model in handling typical customs operations can be accurately evaluated during pruning. Therefore, although the dataset is not large in scale, it must be carefully constructed to ensure the diversity and representativeness of its content. It should include samples from various real-world business scenarios, such as compliant customs declarations, risky documents with logical contradictions or non-standard declarations, and descriptive texts of key regulated commodities.

[0064] Step 2: Define the final target sparsity. And define a multi-stage, progressive sparsity path. For example, S = {0.2, 0.5, 0.7}.

[0065] In some embodiments, the specific implementation process of step two includes:

[0066] Traditional pruning methods, when faced with high sparsity targets (e.g., 70%), typically employ a "one-step" strategy, removing a large number of model parameters at once. However, both research and practice have shown that such drastic structural changes can severely impact large language models, easily leading to performance "collapse" and a precipitous drop in accuracy. To mitigate this risk, this invention employs a multi-stage progressive pruning strategy. Its core principle is to decompose a large, potentially destructive pruning task into a series of smaller, manageable, and sequential subtasks. By gradually increasing the sparsity in stages, with opportunities for adjustments and recovery between stages, the model is guided to smoothly reach its final high-sparse state, maximizing its high performance in customs business processing.

[0067] The specific operation of this step is to define a clear, progressively increasing sequence of sparsity rates. This sequence not only defines the final compression target but also outlines the intermediate states that must be passed to achieve that target.

[0068] Example path: With a final sparsity of 70% as the goal, a typical progressive path can be defined as: S = {0.2, 0.5, 0.7}, which means that the entire pruning process will be divided into three main stages. First, the model will be pruned to a sparsity of 20%; then, based on the 20% sparsity model, it will be further pruned to 50%; finally, based on the 50% sparsity model, it will be pruned to 70% to achieve the final goal.

[0069] Configurability: This path is fully configurable. Users can flexibly adjust the number of stages and the specific sparsity value of each stage through parameters such as `--progressive_stages` to achieve different pruning strategies, such as conservative or aggressive pruning, according to different models or business needs.

[0070] Step 3: Conduct a comprehensive performance evaluation of the original model on an independent, tagged customs business validation set. Key business metrics should include: F1 score for high-risk message identification, accuracy of commodity HS code classification, and document processing latency. Record this benchmark as the basis for subsequent performance comparisons.

[0071] In some embodiments, the specific implementation process of step three includes:

[0072] The performance of the original model is comprehensively evaluated on an independent, labeled customs business validation set. This benchmark represents the model's best performance within the current business capabilities. The ultimate goal of all subsequent pruning and fine-tuning operations is to achieve a high compression ratio while bringing the model's various business metrics as close as possible to this initial benchmark. Without a benchmark, it is impossible to scientifically judge the performance of the compressed model. This provides an initial basis for dynamic adjustments; one of the core aspects of this invention is that the pruning strategy is sensitive to model performance. Before the pruning cycle begins, the system needs to know the performance level of the original model (e.g., whether it is a "high-performance" model or a "medium-performance" model) so that a reasonable, initial conservative or aggressive strategy can be set in the first round of pruning. For example, a model with extremely high initial performance can withstand a more aggressive initial pruning strategy.

[0073] This invention focuses on the following three key indicators directly related to customs operations:

[0074] A. High-Risk Message Identification – F1 Score: In tasks involving imbalanced samples (where high-risk messages are far fewer than compliant messages) such as risk identification, simply using accuracy can be misleading. The F1 score, which considers both precision and recall, is a more reliable evaluation metric. In customs operations, it's crucial to avoid misclassifying a high-risk target (requiring high recall) and also to avoid misclassifying a large number of compliant documents as high-risk, thus increasing unnecessary manual review burden (requiring high precision). Precision: Recall rate: F1 score: In the formula, TP Indicates a true positive result; FP This indicates a false positive. FN This indicates a false negative.

[0075] B. Product HS Code Classification - Accuracy: For the task of automatic classification of product HS codes, the accuracy rate is an intuitive and core indicator that evaluates the model's ability to correctly classify product descriptions into thousands of categories.

[0076] C. Document Processing Latency: This metric is directly related to the deployment efficiency of the model and is one of the most direct motivations for model compression. It measures the time required for the model to process a single business request (such as reviewing a customs declaration).

[0077] On the target deployment hardware, the model is run multiple times to infer a single standard-length message and calculate its average time (in milliseconds).

[0078] After the evaluation is completed, the values ​​of all metrics will be recorded to form a baseline performance file. This file is not only the final report card of the project, but will also serve as the initial performance data input to the dynamic MAD module in the next pruning cycle, thereby initiating the entire adaptive pruning process.

[0079] Step 4: Entering a phased loop, the system starts the main loop and performs pruning stage by stage according to the preset sparsity path S. In each stage... Set the target sparsity rate for the current stage. .

[0080] In some embodiments, the specific implementation process of step four includes:

[0081] The model is subjected to only one limited structural change and has the opportunity to recover its performance before moving to the next stage (as described in step eleven). This iterative approach of "small steps and quick iterations" ensures that the model can gradually adapt to the increasing sparsity, thereby maintaining relatively stable performance throughout the process.

[0082] The system reads the sparsity path defined in step two, for example, S = {0.2, 0.5, 0.7}. The loop will iterate through this sequence for the number of iterations. In each loop (stage)... At the beginning of the process, the system will retrieve the current sparsity value from path S. (For example, 0.2 in the first loop and 0.5 in the second), and set it as the global target sparsity for the current stage.

[0083] Step 5: Calculate the hierarchy importance score, in the stage At the beginning, the system traverses every pruningable layer in the model. The customs document data used for calibration is input into the model to obtain the input activation X for each layer. Then, each weight is calculated. Importance score.

[0084] In some embodiments, the specific implementation process of step five includes:

[0085] The system first traverses all pruningable layers in the current model. (In the Transformer architecture, this mainly refers to the self-attention layer and the linear layers in the feedforward network.) Subsequently, the "calibration dataset" (i.e., customs document samples) prepared in step one is input into the model in batches for a forward propagation.

[0086] During the forward propagation process, the system acquires and records data from each layer. The input data, i.e., the input activation matrix This matrix represents the actual data signals received by each layer when the model processes typical customs business texts.

[0087] After obtaining the weight matrix of each layer and corresponding input activation Then, the system will assign weights to each weight in that layer. Calculate its importance score The score is calculated using the following formula:

[0088]

[0089] in, Indicates the first The first in the layer Outlier scores by weight; This represents the input feature associated with the weight. The L2 norm is used to reflect the overall magnitude of the input channel across all samples; This represents the absolute value of the weight itself, used to measure the strength of its parameter.

[0090] After this step is completed, for each pruning layer in the model Each of these will result in a corresponding importance score matrix. This matrix is ​​compared to the original weight matrix. While maintaining the same dimensions, the internal values ​​are no longer static weights, but rather scores that dynamically reflect the importance of each weight in handling customs operations. This series of importance score matrices serves as the direct input and basis for judgment in subsequent implementation of all advanced pruning strategies, such as dynamic threshold calculation and allocation of hierarchical sparsity.

[0091] Step 6: Calculate the dynamic pruning threshold for the importance score matrix of each layer. The core of this invention, dynamic threshold calculation, is performed to dynamically and adaptively calculate a pruning threshold for each layer to distinguish between important and unimportant weights. First, the median score and median absolute deviation (MAD) are calculated. Then, considering data distribution (skewness of score distribution), layer characteristics (whether the layer is an attention layer or a feedforward network layer), and business performance feedback (the decrease in F1 score or HS coding classification accuracy for high-risk identification after pruning in the previous stage), a dynamic threshold multiplier is calculated. When key business metrics decline, the system will automatically adopt a more conservative threshold.

[0092] In some embodiments, the specific implementation process of step six includes:

[0093] (1) Calculation of basic threshold

[0094] For each layer of importance score matrix obtained from step five The calculation process for this step is as follows:

[0095] First, calculate a basic, data-driven statistical threshold.

[0096] Calculate the median: Calculate the fractional matrix median .

[0097] Calculate the MAD value: The MAD value is calculated according to its definition, using the following formula:

[0098]

[0099] in, This represents the score of outlier values ​​for each weight in this layer. This represents the median of the weight scores for that layer. MAD indicates the average deviation of most weight scores from the "typical value" (median) within the layer.

[0100] Establish a baseline threshold: Set a baseline threshold based on a fixed multiple (e.g., 2 times), as shown in the following formula:

[0101]

[0102] In the formula, This represents the baseline threshold.

[0103] (2) Dynamic threshold adjustment

[0104] Next, the fixed multiplier "2" is upgraded to a dynamically calculated multiplier. This dynamic multiplier is obtained by multiplying multiple adjustment factors, as shown in the following formula:

[0105]

[0106] in, Indicates a dynamic multiple; It is a base multiple (e.g., 2.0); , , All of these represent adjustment factors.

[0107] Final threshold calculation: To prevent extreme values, the calculated threshold is... It will be forcibly constrained within a reasonable range of [0.5, 4.0], as shown in the formula:

[0108]

[0109] In this formula, `clip` is a numerical clipping function used to limit the input variable to a specified range; it represents the dynamic multiplier. When the value is below 0.5, it is truncated to 0.5; when it is above 4.0, it is truncated to 4.0; if it is within the interval [0.5, 4.0], the original value is kept unchanged.

[0110] in, This indicates the final dynamic multiple.

[0111] Final dynamic threshold:

[0112] in, This represents the final dynamic threshold.

[0113] Step 7: Quantify the importance of each level. Based on the final dynamic threshold, calculate the importance of each level. outlier ratio This serves as a quantitative indicator of the importance of this layer in handling customs text transactions.

[0114] In some embodiments, the specific implementation process of step seven includes:

[0115] The system will iterate through the importance score matrix. Each score in the algorithm is compared with the final dynamic threshold of that layer. The system compares scores. If a score exceeds the threshold, it is identified as an "outlier." The system then calculates the total number of outliers for that layer using the following formula:

[0116] ,in, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; This represents the total weight of this layer. It is calculated... The system can then obtain the proportion of weights considered "highly important" or "abnormal" within that layer, which is used to generate a layer sparsity allocation matrix in the future, ensuring that highly sensitive layers receive a more conservative pruning strategy.

[0117] Divide the total number of outliers by the total number of parameters in this layer. This will give you the outlier ratio for that layer. It is usually expressed as a percentage, and the formula is:

[0118]

[0119] After the system has traversed all pruningable layers in the model, the final output of this step is an outlier ratio vector R = [R 1 , R 2 , ..., R nThis vector clearly depicts the importance distribution map of the entire model, for example, R = [1.5%, 2.3%, 0.8%, ...].

[0120] Step 8: Assign layer sparsity based on the importance index of all layers. For each layer Assign a specific pruning thinning rate The principle is: the layer that is more important for customs business text processing ( Higher), cut less ( Lower).

[0121] In some embodiments, the specific implementation process of step eight includes:

[0122] By intelligently allocating a limited "pruning budget," more parameters are retained in the critical layers that contribute the most to model performance, while less important layers are compressed more drastically. Compared to uniform pruning across all layers, this differentiated strategy better maintains the model's business processing accuracy under high sparsity. The fundamental principle is: the more important a layer is in processing customs business text (i.e., the higher its outlier ratio), the lower the pruning sparsity allocated to it should be.

[0123] For each prunable layer in the model (In practice, this usually refers to a Transformer module), its final sparsity. It was calculated through a three-step process:

[0124] (1) Input outlier ratio vector R = [R 1 , R 2 , ..., R n (From step seven) and the global target sparsity at the current stage (From step four).

[0125] (2) Set an inverse relationship for the initial sparsity rate. Based on the core principle, set an initial sparsity rate for each layer that is inversely proportional to its importance. This relation ensures The higher the level, the higher its initial... The lower the value.

[0126] (3) Constraints and Adjustments. Directly using the above relationships may result in excessively high or low sparsity rates in some layers, thus affecting the stability of the model. Therefore, a constraint mechanism needs to be introduced for adjustment.

[0127] Introducing the hyperparameter λ: A hyperparameter λ is used to define a sparsity around the global objective. The range of fluctuation.

[0128] The final sparsity of each layer It must be constrained to the following interval: .

[0129] After applying constraints, the system performs normalization to ensure the final sparsity across all levels. After weighting by parameters, the result is exactly equal to the global target sparsity at the current stage. .

[0130] After this step is completed, the final output is a hierarchical sparsity distribution vector. = [S 1 , S 2 , ...,S n This vector constitutes a detailed, directly executable pruning plan. For example, when the global objective... When the value is 0.5, the output plan might be [0.45, 0.42, 0.53, ...], clearly indicating how much weight should be removed from each level in the next operation.

[0131] Step 9: Perform pruning operations based on the calculated sparsity of each layer. The model was pruned using the relatively mature Wanda pruning method, removing the weights with the lowest importance scores in a corresponding proportion.

[0132] In some embodiments, the specific implementation process of step nine includes:

[0133] For each prunable layer in the model l The pruning operation shall be performed according to the following procedure:

[0134] (1) Input: This step requires two results from the previous step: the importance score matrix of this layer. (From step five) The specific sparsity index assigned to this layer (From step eight).

[0135] (2) Determine the number of branches to be pruned within the layer: based on the allocated sparsity. Calculate the number of weights that need to be pruned in this layer. The formula is: .

[0136] in, yes l Total number of parameters in the layer It is the sparsity rate assigned to this layer.

[0137] (3) In the importance score matrix Find a threshold in the middle. (tau), such that the number of weights with scores below this threshold is exactly equal to... This is typically done by sorting all the importance scores for that layer and selecting the [number]th [level]. The threshold is implemented using a number of scores.

[0138] (4) Generate pruning mask: Create a mask that corresponds to the weight matrix of this layer. Binary mask matrices with identical dimensions If the importance score Then the corresponding position of the mask (Retain), if importance score Then the corresponding position of the mask (Remove).

[0139] (5) The original weight matrix With the generated binary mask matrix To perform element-wise multiplication, the formula is: .

[0140] in, Indicates the first The original weight matrix of the layer, containing all parameters of that layer; This is a binary mask matrix of the same dimension, used to indicate whether each weight is preserved. When When, the weight of the corresponding position is retained; when When this happens, the corresponding weight is set to zero, indicating that it is being pruned. (Symbol) This represents the Hadamard product, i.e., element-wise multiplication. The result of the product. The weight matrix after pruning retains important weights and masks redundant parameters, thereby achieving sparsity.

[0141] After the system has traversed and completed operations on all pruning layers, the output of this step is a new model with a sparser structure. The network architecture and number of layers remain unchanged, but its internal weight matrix has been modified to include a large number of zero values ​​calculated using a dynamic, progressive strategy. This new model has achieved the target sparsity for this stage. And is ready to undergo the next phase of performance evaluation.

[0142] Step 10: Evaluate the phased business performance and complete the phase. After pruning, the model's F1 score and classification accuracy were immediately re-evaluated on the customs business validation set, and performance changes were recorded.

[0143] In some embodiments, the specific implementation process of step ten includes:

[0144] By evaluating the model after each stage, the trajectory of its performance change as the sparsity rate increases can be clearly tracked. This provides complete process data for ultimately determining the effectiveness of the method.

[0145] The results of this assessment will serve as a crucial feedback signal, directly influencing the next round of pruning decisions. This enables the entire pruning process to "perceive" the consequences of its actions and to "reflect" and "adjust," which is fundamental to achieving intelligent pruning.

[0146] The system uses the currently pruned model to completely recalculate all key business metrics defined in step three on the validation set. The pruning intensity for the next round is adjusted: the currently calculated performance metrics (such as Accuracyi) will be used as input when calculating the dynamic threshold in the next stage (i+1) step six. For example, if the current evaluation finds a significant drop in accuracy Accuracyi, then the performance feedback factor F will be adjusted when calculating the dynamic threshold multiple in the next round. perf This will automatically increase, thus making the next round of pruning strategy more conservative.

[0147] Step 11: Perform inter-stage fine-tuning. If the current stage is not the final stage, perform a lightweight, short-term fine-tuning of the model on the calibrated customs document data. This step aims to adapt the model to the new sparse structure and restore the ability to understand customs-specific text patterns temporarily lost due to pruning.

[0148] In some embodiments, the specific implementation process of step eleven includes:

[0149] To ensure lightweight and efficient fine-tuning, parameter-efficient fine-tuning (PEFT) techniques are typically employed. A typical and effective method is LoRA (Low-Rank Adaptation). LoRA freezes the model's original pruned weights and injects two small, trainable low-rank matrices into specific layers of the model (such as attention layers). During fine-tuning, only the parameters of these newly added small matrices are updated, significantly reducing computational and memory overhead.

[0150] Step 12: Looping and Iteration. After fine-tuning, the system enters the next stage of pruning loop, repeating steps 5 to 11 until all stages are completed.

[0151] The core principle of this step is state evolution iteration. Unlike simple repetitive execution, the multi-stage loop of this invention is a process with memory and evolution. The input of each loop is the output of the previous loop after its complete execution (i.e., the model after pruning and fine-tuning). This means that before entering the next more aggressive pruning stage, the system no longer faces the original dense model, but a completely new sparse model that has adapted to the current sparsity and whose performance has been partially recovered.

[0152] Through this precise iterative control, the present invention ensures that its progressive strategy is executed in an orderly manner, achieving a smooth transition from a dense model to a highly sparse model.

[0153] This invention compares the core performance of four methods—the original model, the traditional Wanda, OWL (one-time), and the complete version of this invention—through experiments. The specific results are shown in Table 1.

[0154] Table 1 Core Performance Comparison

[0155]

[0156] As shown in Table 1, under the same sparsity ratio (70%), the method of this invention significantly improves the core performance of the model while maintaining a high compression ratio. Compared with the traditional Wanda method and the OWL (one-time) method, the full version of this invention improves the F1 score for high-risk identification by 0.20 and 0.09 respectively, and the accuracy of HS coding classification by 6.2% and 2.8%, while maintaining the same speedup ratio for single-document processing (2.6×). This indicates that the method of this invention effectively suppresses the performance loss caused by excessive pruning without sacrificing inference speed, demonstrating superior stability and robustness.

[0157] This invention compares the performance curves of OWL (one-time method) and this invention (complete version) under different sparsity ratios through experiments. The specific results are shown in Table 2.

[0158] Table 2 Comparison of performance curves under different sparsity ratios

[0159]

[0160] As shown in Table 2, the method of this invention exhibits higher stability and accuracy preservation under different sparsity rates. As the sparsity rate increases from 50% to 80%, the performance of the traditional OWL (One-Time) method gradually decreases, exhibiting a significant performance crash at 80% sparsity (from 0.76 to 0.52); while the complete version of this invention maintains high performance under the same conditions (0.88→0.79), with a smoother curve decline. This indicates that the present invention possesses stronger robustness and scalability under high sparsity conditions, effectively avoiding model degradation caused by excessive pruning, and achieving a balance between high compression and high performance.

[0161] The above combination Figure 1 , Figure 2 The dynamic threshold-driven AI model compression method provided in the embodiments of the present invention has been described in detail. Next, the dynamic threshold-driven AI model compression system provided in the embodiments of the present invention will be described in conjunction with the accompanying drawings.

[0162] Figure 3 This is a schematic diagram of the structure of a dynamic threshold-driven AI model compression system according to an embodiment of the present invention, with reference to... Figure 3 The system described in this invention includes:

[0163] The preparation module is configured to: acquire the large language model to be compressed and the customs business calibration dataset, and define a sparse rate sequence that progresses from low to high.

[0164] The traversal calculation module is configured to: traverse all pruningable layers in the large language model according to the sparsity rate sequence at each pruning stage; input the customs business calibration dataset into the large language model to obtain the input activation matrix of each layer; calculate the importance score matrix of each layer based on the weight matrix and input activation matrix of each layer; calculate the baseline threshold based on the importance score matrix of each layer; introduce a dynamic factor to adjust the coefficient of the baseline threshold and calculate the final dynamic threshold of each layer; filter outliers in the importance score matrix of each layer based on the final dynamic threshold of each layer and calculate the outlier ratio of each layer; calculate the initial sparsity rate of each layer based on the outlier ratio of each layer, apply constraints to obtain the sparsity rate index of each layer; and calculate the number of weights to be subtracted from each layer based on the sparsity rate index of each layer.

[0165] The model compression module is configured to: select a threshold in the importance score matrix and create a binary mask matrix; if the importance score of a certain layer is greater than or equal to the threshold, retain the position corresponding to the mask; otherwise, remove the position corresponding to the mask; and obtain the compressed large language model.

[0166] In some embodiments, the sparsity rate sequence represents the sparsity rate that the large language model needs to achieve at each pruning stage, and the sparsity rate sequence includes a target sparsity rate.

[0167] In some embodiments, before traversing the large language model, the method further includes: defining key business metrics for evaluating the performance of the large language model, including: F1 score for high-risk message recognition, accuracy of commodity HS code classification, and document processing latency; the key business metrics are used to evaluate the performance of the pruned large language model.

[0168] In some embodiments, a baseline threshold is calculated based on the importance score matrix of each layer; a dynamic multiplier is introduced to adjust the coefficient of the baseline threshold, and the final dynamic threshold for each layer is calculated; including:

[0169] Based on the importance score matrix of each layer, calculate the median and MAD value;

[0170] Calculate the baseline threshold based on the median and MAD value;

[0171] Introducing dynamic multiples ,in, It is a base multiple; , , All represent adjustment factors;

[0172] By constraining the dynamic multiple within a set range, the final dynamic multiple is obtained.

[0173] Calculate the final dynamic threshold for each layer ;in, This represents the median; Indicates the MAD value; This indicates the final dynamic multiple.

[0174] In some embodiments, outliers in the importance score matrix of each layer are filtered based on the final dynamic threshold of each layer, and the outlier ratio of each layer is calculated; including:

[0175] Iterate through each score in the importance score matrix of each layer and compare it with the final dynamic threshold of each layer. If the score is greater than the final dynamic threshold, it is identified as an outlier. Count the total number of outliers in each layer.

[0176] The outlier ratio for each layer is calculated based on the ratio of the total number of outliers to the total number of parameters in that layer.

[0177] In some embodiments, based on the outlier ratio of each layer, an initial sparsity rate for each layer is calculated, constraints are applied, and a sparsity rate index for each layer is obtained; including:

[0178] Based on the outlier ratio of each layer, an initial sparsity rate is set for each layer that is inversely proportional to its importance. ;in, express l Outlier ratio of the layer;

[0179] Constrain the initial sparsity of each layer to Within this layer, the sparsity index is obtained for each layer; among them, λ represents the target sparsity; λ represents the hyperparameter.

[0180] In some embodiments, a threshold is selected in the importance score matrix to create a binary mask matrix; including:

[0181] Sort all importance scores in the importance score matrix and select the first one. A number of scores are used as thresholds; among which, express l The number of weights that need to be subtracted from the layer;

[0182] Create a binary mask matrix with the exact same dimensions as the weight matrix of that layer in each layer.

[0183] The dynamic threshold-driven AI model compression system according to embodiments of the present invention can correspond to performing the method described in the embodiments of the present invention, and the above and other operations and / or functions of each module of the dynamic threshold-driven AI model compression system are respectively for implementing Figure 1 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0184] See Figure 4 The diagram shows the structure of a computer device, which includes a processor, a communication interface, and a computer-readable storage medium. The processor, communication interface, and computer-readable storage medium are connected via a bus or other means. The communication interface is used to receive and send data. The computer-readable storage medium can be stored in the computer device's memory. The computer-readable storage medium stores computer programs, including program instructions, and the processor executes the program instructions stored in the computer-readable storage medium. The processor (or CPU, Central Processing Unit) is the computing and control core of the computer device, adapted to implement one or more instructions, specifically adapted to load and execute one or more instructions to achieve the corresponding steps in the embodiment of the dynamic threshold-driven AI model compression method.

[0185] This embodiment provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device.

[0186] Furthermore, this storage space also contains one or more instructions suitable for loading and execution by the processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM memory or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0187] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes one or more instructions stored in the computer-readable storage medium to implement the corresponding steps in the above-described embodiment of the dynamic threshold-driven AI model compression method.

[0188] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0189] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0193] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic threshold-driven AI model compression method, characterized in that, include: Obtain the large language model to be compressed and the customs business calibration dataset, and define a sparsity rate sequence that progresses from low to high; According to the sparsity rate sequence, all pruningable layers in the large language model are traversed at each pruning stage. The customs business calibration dataset is input into the large language model to obtain the input activation matrix of each layer. Based on the weight matrix and input activation matrix of each layer, calculate the importance score matrix of each layer; Calculate the baseline threshold based on the importance score matrix of each layer; Introduce a dynamic multiplier, adjust the coefficient of the baseline threshold, and calculate the final dynamic threshold for each layer; Based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated. Based on the outlier ratio of each layer, the initial sparsity of each layer is calculated, constraints are applied, and the sparsity index of each layer is obtained. Based on the sparsity index of each layer, the number of weights that need to be subtracted from each layer is calculated. Select a threshold from the importance score matrix to create a binary mask matrix; If the importance score of a certain layer is greater than or equal to the threshold, the position corresponding to the mask is retained; otherwise, the position corresponding to the mask is removed, resulting in a compressed large language model.

2. The dynamic threshold-driven AI model compression method according to claim 1, characterized in that, The sparsity rate sequence represents the sparsity rate that the large language model needs to achieve at each pruning stage, and the sparsity rate sequence includes the target sparsity rate.

3. The AI ​​model compression method driven by dynamic threshold according to claim 1, characterized in that, Before traversing the large language model, the process also includes: defining key business metrics for evaluating the performance of the large language model, including: F1 score for high-risk message recognition, accuracy of commodity HS code classification, and document processing latency; these key business metrics are used to evaluate the performance of the pruned large language model.

4. The AI ​​model compression method driven by dynamic threshold according to claim 1, characterized in that, Calculate the baseline threshold based on the importance score matrix of each layer; A dynamic multiplier is introduced to adjust the coefficient of the baseline threshold, and the final dynamic threshold for each layer is calculated; the method includes: Based on the importance score matrix of each layer, calculate the median and MAD value; Calculate the baseline threshold based on the median and MAD value; Introducing dynamic multiples ,in, It is a base multiple; , , All represent adjustment factors; By constraining the dynamic multiple within a set range, the final dynamic multiple is obtained. Calculate the final dynamic threshold for each layer ;in, This represents the median; Indicates the MAD value; This indicates the final dynamic multiple.

5. The dynamic threshold-driven AI model compression method according to claim 1, characterized in that, Based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated; the method includes: Iterate through each score in the importance score matrix of each layer and compare it with the final dynamic threshold of each layer. If the score is greater than the final dynamic threshold, it is identified as an outlier. Count the total number of outliers in each layer. The outlier ratio for each layer is calculated based on the ratio of the total number of outliers to the total number of parameters in that layer.

6. The AI ​​model compression method driven by dynamic threshold according to claim 1, characterized in that, Based on the outlier ratio of each layer, the initial sparsity of each layer is calculated, constraints are applied, and the sparsity index of each layer is obtained; the method includes: Based on the outlier ratio of each layer, an initial sparsity rate is set for each layer that is inversely proportional to its importance. ;in, express l Outlier ratio of the layer; Constrain the initial sparsity of each layer to Within this layer, the sparsity index is obtained for each layer; among them, λ represents the target sparsity; λ represents the hyperparameter.

7. The dynamic threshold-driven AI model compression method according to claim 1, characterized in that, A threshold is selected from the importance score matrix to create a binary mask matrix; the method includes: Sort all importance scores in the importance score matrix and select the first one. A number of scores are used as thresholds; among which, express l The number of weights that need to be subtracted from the layer; Create a binary mask matrix with the exact same dimensions as the weight matrix of that layer in each layer.

8. A dynamic threshold-driven AI model compression system, characterized in that, include: The preparation module is configured to: acquire the large language model to be compressed and the customs business calibration dataset, and define a sparse rate sequence that progresses from low to high. The traversal computation module is configured to: traverse all pruningable layers in the large language model at each pruning stage according to the sparsity rate sequence, input the customs business calibration dataset into the large language model, and obtain the input activation matrix of each layer; Based on the weight matrix and input activation matrix of each layer, calculate the importance score matrix of each layer; Calculate the baseline threshold based on the importance score matrix of each layer; Introduce a dynamic multiplier, adjust the coefficient of the baseline threshold, and calculate the final dynamic threshold for each layer; Based on the final dynamic threshold of each layer, outliers in the importance score matrix of each layer are filtered out, and the outlier ratio of each layer is calculated. Based on the outlier ratio of each layer, the initial sparsity of each layer is calculated, constraints are applied, and the sparsity index of each layer is obtained. Based on the sparsity index of each layer, the number of weights that need to be subtracted from each layer is calculated. The model compression module is configured to: select a threshold in the importance score matrix and create a binary mask matrix; If the importance score of a certain layer is greater than or equal to the threshold, the position corresponding to the mask is retained; otherwise, the position corresponding to the mask is removed, resulting in a compressed large language model.

9. A computer device, characterized in that, A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the steps of the dynamic threshold-driven AI model compression method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the steps of the dynamic threshold-driven AI model compression method as described in any one of claims 1-7.

Images