Embedded platform-oriented semantic preserving type parameter compression method and system

Abstract

The invention provides a semantic preserving type parameter compression method and system oriented to an embedded platform, and relates to the technical field of model lightweight. Structured pruning is guided by calculating channel-level and network-level semantic importance, mixing precision quantification is carried out on determined bit width constraints, finally, precision is recovered through feature-level distillation, and the compression degree and recognition precision can be effectively considered.

Description

Technical Field

[0001] This invention relates to the field of lightweight model technology, specifically to a semantically preservative parameter compression method and system for embedded platforms. Background Technology

[0002] With the development of the Internet of Things (IoT) and edge computing, deploying deep learning models on resource-constrained embedded devices (such as MCUs, DSPs, FPGAs, or low-power NPUs) has become a mainstream trend, widely applied in vertical fields such as industrial inspection and autonomous driving perception terminals. These vertical fields exhibit a significant duality in their core demands: on the one hand, they require models with extremely high professional semantic recognition accuracy to meet reliability requirements in specific scenarios (such as controlling the false negative rate in industrial defect detection and accurate analysis of medical data); on the other hand, embedded devices face strict constraints in hardware resources. Flash storage space is typically only in the megabyte to hundreds of megabyte range, SRAM memory is mostly in the tens of kilobytes to several megabytes range, and their computing power is far lower than that of cloud servers, making it difficult to support the storage and computational needs of the original deep learning models. Therefore, lightweighting of the model becomes a core prerequisite for embedded deployment.

[0003] To address the challenges of deploying deep learning models on embedded devices, the industry has developed a series of model compression technologies. The core objective is to reduce the number of model parameters, storage requirements, and computational complexity while maintaining a certain level of accuracy. These technologies mainly include the following three key categories: 1. Network pruning: Based on metrics such as weight magnitude and gradient contribution, redundant connections, neurons, or channels in the neural network are identified and removed. This reduces ineffective computational units, resulting in model slimming. Common methods include structured pruning (pruning by channel or layer dimension) and unstructured pruning (randomly pruning individual weight connections). 2. Parameter Quantization: Converting high-precision 32-bit floating-point (FP32) parameters in the model into low-bit integers (such as INT8, INT4, or even binary). This reduces memory usage and computational overhead by lowering the data storage bit width. Mainstream solutions include uniform quantization, non-uniform quantization, and adaptive quantization. 3. Knowledge Distillation: Constructing a "teacher-student" network architecture. Utilizing the output distribution, intermediate features, or attention mechanisms of a well-trained, high-precision large model (teacher network), guides the learning of a more streamlined small model (student network), enabling the small model to inherit the generalization ability and recognition accuracy of the large model. These techniques can be used individually or in combination, forming the mainstream technology system for lightweight models.

[0004] However, although existing model compression techniques have theoretically reduced the number of parameters and computational load, they still cannot balance compression level and recognition accuracy in the actual deployment of embedded devices in vertical fields. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a semantically preservative parameter compression method and system for embedded platforms, which solves the technical problem that existing model compression technologies cannot balance compression level and recognition accuracy.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a semantically preserving parameter compression method for embedded platforms, comprising: The base model is optimized and trained using a vertical domain calibration dataset to obtain a domain-adapted large model; The computational domain adapts the channel-level and network-level semantic importance of large models and constructs hardware lookup tables; Based on semantic importance and hardware lookup tables, hardware-aware structured pruning is performed on the domain adaptation model to obtain a pruned, simplified model. Under a defined bit width constraint, semantically preserving mixed-precision quantization is performed on the simplified model to obtain an initial lightweight model; The initial lightweight model is subjected to feature-level semantic compensation distillation to obtain a lightweight model with restored accuracy, which is then used as the final lightweight large model. The lightweight optimized model is then subjected to edge device hardware adaptation processing to generate a hardware-compatible model file. The model file is then deployed to the target edge device to complete the rapid deployment of the large model.

[0007] Preferably, the computational domain adapts to the channel-level and network-level semantic importance of large models, including: in, Indicates category Weighting coefficients are set for key categories within a vertical industry. For non-priority categories, set ; This indicates that the domain-adaptive large model is adapted to the input samples. At that time, the loss function affects the parameters The first gradient, i.e. ; The first step in adapting a large model to the domain The first layer The semantic importance of each channel; Represents the vertical domain calibration dataset Belongs to the category The sample set, This represents the total number of sample categories in the vertical domain calibration dataset; Domain Adaptation Large Model Network hierarchical semantic importance score The calculation formula is as follows: in, For domain adaptation, the first in the large model Total number of channels in the layer.

[0008] Preferably, the step of performing hardware-aware structured pruning on the domain adaptation model based on semantic importance and a hardware lookup table to obtain a pruned, simplified model includes: An optimization target model is constructed based on semantic importance and a hardware lookup table. The optimization target model is solved to obtain the optimal solution. Based on the optimal solution, a hardware-aware structured pruning operation is performed on the domain adaptation model to obtain a pruned simplified model. The optimal solution of the optimization target model satisfies the following: maximizing the retention of semantic scores while satisfying the target latency.

[0009] Preferably, the optimization target model includes the objective function and constraints; The objective function is as follows: The constraints are as follows: in, As a mask, it represents the first... Layer Should each channel be retained? The first step in adapting a large model to the domain The first layer The semantic importance of each channel; Delay for the goal; For domain adaptation, the first in the large model Total number of channels in the layer; The total number of network layers for domain-adaptive large models; The first one is obtained based on the constructed hardware lookup table. The measured inference time of a layer at a specific pruning rate.

[0010] Preferably, the step of performing semantically preserving mixed-precision quantization on the simplified model under a defined bit-width constraint to obtain an initial lightweight model includes: Hierarchical adaptive bit width selection based on KL divergence determines the bit width of each level in the simplified model, specifically including: Define the set of optional bit widths for each layer of the simplified model. bit; The first step in defining the simplified model Layer in position width Quantization perturbation ; A comprehensive consideration of the semantic importance and quantization perturbation of each layer in the simplified model Under the premise of meeting the overall model compression rate or hardware storage constraints, high bit width is allocated to layers with large quantization perturbation and high semantic importance, and low bit width is allocated to layers with small quantization perturbation or low semantic importance. Under a defined bit width constraint, symmetric quantization is performed, and a learnable truncation threshold is introduced for the activation values. The numerical mapping is performed using the following quantization formula: in, This represents the original parameter values ​​in the simplified model; This represents the quantized parameter value; To be based on bit width A defined scaling factor.

[0011] Preferably, the step of performing feature-level semantic compensation distillation on the initial lightweight model to obtain the final lightweight large model after accuracy restoration includes: A teacher-student network is constructed, with the teacher network serving as a domain-adapted large model and the student network as an initial lightweight model. The high-dimensional features output by the teacher network guide the student network in feature alignment. During the guidance process, the following total loss function is used to fine-tune the initial lightweight model: in, For mission losses; For knowledge distillation loss; and The teacher network and the student network are respectively in the 1st Feature maps of key layers; It is a lightweight linear transformation layer used to match the feature dimensions of the student network and the teacher network; The set is automatically selected based on the highest response value among the semantic importance of network layers in the domain-adapted large model.

[0012] Preferably, the logical structure of the hardware lookup table is represented as follows: {(operator type, input dimension, number of output channels, hardware configuration): physical latency}.

[0013] Secondly, the present invention provides a semantically preserving parameter compression system for embedded platforms, the semantically preserving parameter compression system for embedded platforms being used to perform the semantically preserving parameter compression method for embedded platforms as described above.

[0014] Thirdly, the present invention provides a computer-readable storage medium, characterized in that it stores a computer program for semantically preserving parameter compression for an embedded platform, wherein the computer program causes a computer to execute the semantically preserving parameter compression method for an embedded platform as described above.

[0015] Fourthly, the present invention provides an electronic device, comprising: One or more processors; Memory; and One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing semantically preserving parameter compression methods for embedded platforms as described above.

[0016] (III) Beneficial Effects This invention provides a semantically preservative parameter compression method and system for embedded platforms. Compared with existing technologies, it has the following advantages: This invention guides structured pruning by calculating the semantic importance at the channel level and network layer level, performs mixed precision quantization based on the determined bit width constraints, and finally restores the precision through feature-level distillation, which can effectively balance the degree of compression and the recognition accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a block diagram of a semantically preservative parameter compression method for embedded platforms according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This application provides a semantically preservative parameter compression method and system for embedded platforms, which solves the technical problem that existing model compression technologies cannot balance compression level and recognition accuracy. It achieves that the feature extraction capability for specific interest categories does not degrade during the compression process, thus balancing compression level and recognition accuracy.

[0021] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows: Existing model compression techniques cannot balance compression levels and recognition accuracy, specifically in the following ways: 1. Poor hardware adaptability of unstructured pruning: Most existing mainstream pruning schemes based on weight magnitude are unstructured pruning, resulting in a random and sparse distribution of the pruned weight matrix, lacking regularity. Embedded hardware (especially CPUs based on SIMD instruction sets and NPUs using systolic array architectures) relies on the continuity and regularity of data for computation, and cannot effectively utilize such random sparsity to optimize computational efficiency. Not only does it fail to significantly reduce inference latency, but it may also increase additional computational costs due to the indexing and addressing overhead of sparse data, leading to actual deployment performance falling short of expectations. 2. Lack of semantic preservation mechanisms leads to accuracy loss: The optimization objective of general compression algorithms is based on the statistical characteristics of the global dataset, without fully considering the special distribution patterns of data in vertical domains, such as the long-tail distribution of industrial defect samples and specific pathological features of medical images. For example, simply pruning based on the L1 / L2 norm can easily prune channels or neurons that play a decisive role in the identification of specific "key semantics" (such as minor defects or early lesions), causing the false positive and false negative rates of the model to soar on core samples, failing to meet the accuracy requirements of vertical domains.

[0022] 3. Lack of specificity in quantization strategies: Traditional quantization schemes often use a single fixed bit width, without considering the differences in the sensitivity of different layers of the neural network to accuracy (e.g., the feature extraction layer is more sensitive to quantization errors, while the output layer has a higher fault tolerance). At the same time, they ignore the differences in bit width throughput efficiency of different computing units in embedded hardware, which makes it impossible for the quantization scheme to achieve the optimal balance between accuracy loss and compression ratio. 4. Decoupling of compression from hardware characteristics leads to poor actual deployment results: Most compression algorithms only optimize theoretical computational load without fully considering the actual hardware characteristics of embedded devices, such as memory access costs, hardware instruction set constraints (e.g., supported bit width types, operation instruction efficiency), and differences in storage architecture. As a result, although the compressed model has excellent theoretical indicators, its actual energy efficiency ratio (the ratio of computing power consumption to recognition accuracy) on specific chips is far lower than expected, resulting in insufficient deployment compatibility and practicality.

[0023] To address the aforementioned issues, this invention proposes a semantically preservative parameter compression method and system for embedded platforms. It guides structured pruning by calculating semantic importance, performs mixed precision quantization based on determined bit-width constraints, and finally restores precision through feature-level distillation.

[0024] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0025] This invention provides a semantically preservative parameter compression method for embedded platforms, such as... Figure 1 As shown, it includes: S1. Use the vertical domain calibration dataset to optimize and train the base model to obtain a domain-adapted large model; S2. Adapt the semantic importance of channel-level and network-level semantics to large models in the computing domain and construct hardware lookup tables; S3. Based on semantic importance and hardware lookup table, perform hardware-aware structured pruning on the domain adaptation model to obtain a pruned simplified model. S4. Under a defined bit width constraint, perform semantically preservative mixed-precision quantization on the simplified model to obtain the initial lightweight model. S5. Perform feature-level semantic compensation distillation on the initial lightweight model to obtain a lightweight model with restored accuracy as the final lightweight large model. Perform edge device hardware adaptation processing on the lightweight optimized model to generate a hardware-compatible model file. Deploy the model file to the target edge device to complete the rapid deployment of the large model.

[0026] The embodiments of the present invention guide structured pruning by calculating the semantic importance at the channel level and network layer level, and perform mixed precision quantization based on the determined bit width constraints. Finally, the precision is restored by feature-level distillation, which can effectively balance the degree of compression and the recognition accuracy.

[0027] In step S1, the base model is optimized and trained using a vertical domain calibration dataset to obtain a domain-adapted large model. This step uses a conventional model training method, which will not be described in detail here.

[0028] In step S2, the channel-level and network-layer semantic importance of the domain-adapted large model is calculated, and a hardware lookup table is constructed. The specific implementation process is as follows: In this embodiment of the invention, in order to accurately identify parameters in the model that have a low contribution to the core tasks of the vertical domain, a semantic importance evaluation system from theoretical model to engineering implementation is established, as follows: First, we define the semantic importance at the channel level and network layer level. Let the loss function for the domain-adaptive large model be... The parameters are For the first The first layer Each channel, its semantic importance score Theoretically defined as an approximation of the global error increment caused by removing this channel, its expression is: in, Let be the Hessian matrix of the loss function.

[0029] In practical calculations, directly solving for the full Hessian matrix and its inverse involves enormous computational costs, making it difficult to perform quickly on a conventional server. To address this technical challenge, this invention introduces a Fisher information matrix to approximate the above formula. The core of this approximation logic lies in using the first moment of the gradient to approximate the diagonal elements of the Hessian matrix, thereby transforming the complex second-order optimization problem into an efficient first-order computation process.

[0030] Based on the above approximation logic, a specific calculable quantitative index for channel-level semantic importance is obtained, namely, the corrected channel semantic importance is: It identifies the convolutional kernel within each layer that contributes the least to the vertical domain features. The calculation formula is as follows: To ensure that the compressed model does not lose key features in vertical domains (such as defect detection in industrial quality inspection), this embodiment of the invention innovatively introduces a class weighting coefficient in gradient calculation. The corrected channel semantic importance score is: This refers to the category weighting coefficient. For key categories of interest within a vertical domain (such as defect samples or lesion samples), a weighting coefficient is set. For non-priority categories (such as normal background samples), set . Represents the vertical domain calibration dataset Belongs to the category The sample set, This represents the total number of sample categories in the vertical domain calibration dataset.

[0031] : Indicates that the model is based on the input samples At that time, the loss function affects the parameters The first gradient, i.e. .

[0032] Targeting vertical domain calibration datasets The key focus categories (such as "defective samples") are assigned higher weight coefficients, making... This can more accurately reflect the channel's ability to preserve core business semantics, thereby guiding subsequent pruning operations to prioritize the retention of these critical channels.

[0033] Based on the scores obtained for each channel, by analyzing the scores of the first channel... The semantic importance of all channels within a layer is aggregated to define the network layer semantic importance score for that layer. : in, For domain adaptation, the first in the large model Total number of channels in the layer.

[0034] The semantic importance score of network layers reflects the contribution of the overall feature extraction capability to the preservation of semantics in the vertical domain.

[0035] To enable the compression scheme to have hardware awareness, this embodiment of the invention constructs a hardware lookup table through experimental methods, mapping the structural features of the domain-adapted large model to physical-level inference latency. The construction process of the hardware lookup table is as follows: Operator decomposition: Decompose the large domain adaptation model into basic operators (such as convolution, fully connected, activation, etc.).

[0036] Multidimensional parametric measurement: On the target embedded hardware (such as a specific MCU or NPU), exhaustive measurement is performed on the performance of each operator under different input sizes, different number of channels, and different quantization bit widths.

[0037] Data Acquisition: Record the average execution time of each operator configuration under the actual hardware instruction set to ensure that nonlinear performance changes caused by memory alignment or SIMD (Single Instruction Multiple Data) acceleration are captured.

[0038] The completed hardware lookup table (LUT) is represented as a multidimensional mapping database, and its logical structure can be expressed as follows: {(Operator type, input dimension, number of output channels, hardware configuration): Physical latency (ms)} The constructed LUT provides a more realistic performance reference than the theoretical computational cost (FLOPs). In the subsequent step S3, the system can directly index this table to calculate the real-time inference time under different pruning strategies, ensuring that the pruned model achieves optimal semantic preservation while meeting hardware power consumption and speed constraints.

[0039] In step S3, based on semantic importance and the hardware lookup table, a hardware-aware structured pruning operation is performed on the domain adaptation model to obtain a pruned, simplified model. The specific implementation process is as follows: In the structured pruning process, an optimization objective model is introduced, aiming to satisfy the target latency. Under the premise of maximizing the preservation of semantic scores, we avoid the problem that unstructured pruning (such as parametric sparsity) cannot accelerate actual inference because it disrupts continuous data storage and cannot adapt to the hardware SIMD parallelism and storage alignment requirements. The specific explanation is as follows: The objective function for optimizing the target model is as follows: Optimize the constraints of the target model: in: As a mask, it represents the first... Layer Should each channel be retained? It is the measured inference time of this layer at a specific pruning rate, obtained based on a pre-built hardware lookup table (LUT).

[0040] Due to the mask Since the variables are discrete, directly solving this combinatorial optimization problem is NP-hard. To improve the solution efficiency, this embodiment of the invention uses ADMM (Alternating Direction Multiplier Method) to solve the optimization objective model, as detailed below: The discrete mask selection problem is transformed into a continuously optimizable subproblem.

[0041] The complex global constraints are decomposed into: (1) a parameter optimization subproblem that satisfies semantic maximization; and (2) a projection subproblem that satisfies hardware delay constraints. By iterating alternately between the two subproblems, the solution set gradually converges to the optimal mask distribution.

[0042] Finally, the optimized continuous variables are remapped back to binary masks, and physical structured parameter removal is performed.

[0043] The optimization target model in this embodiment is based on channel mask. Modeling strictly adheres to structured pruning—directly pruning entire channels, entire convolution kernel rows / columns, or entire attention heads—ensuring the model maintains a regular, continuous, and compact structure, adapting to hardware storage and parallel computing requirements. This effectively avoids the following issues: traditional unstructured pruning only resets individual weights to zero, resulting in randomly distributed sparse weights. This leads to discontinuous memory access, fragmented data blocks, and sparsity that embedded hardware (such as systolic array-based NPUs) cannot effectively recognize and bypass, resulting in the phenomenon of "fewer parameters but no decrease in inference latency."

[0044] Meanwhile, this optimization target model uses the actual hardware latency measured based on LUTs (Look-Up Tables) as the core constraint, replacing the traditional optimization method that only aims at compression ratio. When searching for pruning paths, the optimization algorithm actively avoids solutions that reduce parameters but result in poor acceleration due to violations of hardware alignment rules (such as non-16-byte alignment, SIMD width mismatch), ensuring that pruning decisions directly serve actual inference acceleration, rather than just pursuing theoretical compression ratio.

[0045] Through this optimization target model, pruning decisions are modeled as a problem of "pursuing the highest semantic quality within the time budget (target latency)". The optimization process automatically selects structured pruning schemes that satisfy latency constraints while maximizing the retention of semantic scores, so that the logical structure of the simplified model is fully matched with the SIMD parallel instruction set, storage alignment rules, and systolic array computing mode of the embedded hardware, achieving an effective transformation from theoretical compression ratio to a doubling of real inference performance.

[0046] In step S4, under the determined bit-width constraint, semantically preserving mixed-precision quantization is performed on the simplified model to obtain the initial lightweight model. The specific implementation process is as follows: The remaining parameters of the simplified model obtained after pruning are quantified. To address the problem of abnormal data distribution in vertical domains, this invention proposes a hierarchical adaptive bit-width selection based on KL divergence, as detailed below: 1. Bit-width search space: Defines the set of possible bit widths for each layer of the simplified model. bit.

[0047] 2. Quantization of noise modeling: Define the first... Layer in position width Quantization perturbation .

[0048] 3. Sensitivity Coupling: A comprehensive balance between the semantic importance of each layer and quantization perturbation. Under the premise of meeting the overall model compression ratio or hardware storage constraints, allocate high bit width (e.g., 8 bits) to layers with large quantization perturbations and high semantic importance, and allocate low bit width (e.g., 4 bits or 2 bits) to layers with small quantization perturbations or low semantic importance.

[0049] Parameterized threshold fine-tuning: Symmetric quantization is performed under a defined bit-width constraint. A learnable truncation threshold is introduced for the activation values. The numerical mapping is performed using the following quantization formula: in, This represents the original parameter values ​​in the simplified model; This represents the quantized parameter value; To be based on bit width A defined scaling factor. By... The network weights are considered to be dynamically updated during the fine-tuning process, which enables the quantization window to automatically lock the effective numerical range of the data in the vertical domain, thereby maximizing the accuracy of key features with a limited bit width.

[0050] In step S5, feature-level semantic compensation distillation is performed on the initial lightweight model to obtain a lightweight model with restored accuracy as the final lightweight large model. The lightweight optimized model is then subjected to edge device hardware adaptation processing to generate a hardware-compatible model file. The model file is then deployed to the target edge device to complete the rapid deployment of the large model.

[0051] To recover the semantic loss caused by pruning and quantization, a teacher-student network is constructed. The teacher is a domain-adapted large model (in specific implementations, the FP32 model can be selected, which represents the highest semantic recognition accuracy in this specific business scenario and serves as a benchmark for subsequent compressed models), and the student is an initial lightweight model. The high-dimensional features output by the teacher network guide the student network in feature alignment, thereby achieving rapid recovery of accuracy.

[0052] This invention employs the following total loss function to fine-tune the initial lightweight model. This total loss function considers not only the conventional Logits loss (including task loss) but also other factors. and knowledge distillation loss It also introduces intermediate layer semantic alignment loss. .

[0053] in: and Teachers and students respectively in the Feature maps of key layers; It is a lightweight linear transformation layer (1x1 convolution) used to match the feature dimensions of the student network (due to the reduction in the number of pruned channels) with those of the teacher network.

[0054] It should be noted that here The selection of the set is not random, but is automatically selected based on the layer with the highest response value in the semantic importance of the network hierarchy.

[0055] The intermediate layer semantic alignment loss penalizes the deviation between student and teacher features in Euclidean space, forcing the student network to fit the higher-order expressive power of the teacher network using the remaining parameter size. Through this semantic compensation mechanism, the student network can recover the subtle vertical domain feature discrimination lost during pruning and quantization to the greatest extent while maintaining a lightweight architecture, thus achieving closed-loop recovery of accuracy.

[0056] By minimizing the total loss, a lightweight model with restored accuracy is obtained as the final lightweight large model. The lightweight optimized model is then subjected to edge device hardware adaptation processing to generate a hardware-compatible model file. The model file is then deployed to the target edge device to complete the rapid deployment of the large model.

[0057] This invention also provides a semantically preserving parameter compression system for embedded platforms, which is used to perform the semantically preserving parameter compression method for embedded platforms as described above.

[0058] This invention also provides a computer-readable storage medium storing a computer program for semantically preserving parameter compression for an embedded platform, wherein the computer program causes a computer to execute the semantically preserving parameter compression method for an embedded platform as described above.

[0059] This invention also provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing semantically preserving parameter compression methods for embedded platforms as described above.

[0060] In summary, compared with existing technologies, it has the following beneficial effects: The embodiments of the present invention guide structured pruning by calculating the semantic importance at the channel level and network layer level, and perform mixed precision quantization based on the determined bit width constraints. Finally, the precision is restored by feature-level distillation, which can effectively balance the degree of compression and the recognition accuracy.

[0061] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantically preservative parameter compression method for embedded platforms, characterized in that, include: The base model is optimized and trained using a vertical domain calibration dataset to obtain a domain-adapted large model; The computational domain adapts the channel-level and network-level semantic importance of large models and constructs hardware lookup tables; Based on semantic importance and hardware lookup tables, hardware-aware structured pruning is performed on the domain adaptation model to obtain a pruned, simplified model. Under a defined bit width constraint, semantically preserving mixed-precision quantization is performed on the simplified model to obtain an initial lightweight model; The initial lightweight model is subjected to feature-level semantic compensation distillation to obtain a lightweight model with restored accuracy, which is then used as the final lightweight large model. The lightweight optimized model is then subjected to edge device hardware adaptation processing to generate a hardware-compatible model file. The model file is then deployed to the target edge device to complete the rapid deployment of the large model.

2. The semantically preservative parameter compression method for embedded platforms as described in claim 1, characterized in that, The semantic importance of channel-level and network-level semantics in the computational domain adaptation of large models includes: in, Indicates category Weighting coefficients are set for key categories within a vertical industry. For non-priority categories, set ; This indicates that the domain-adaptive large model is adapted to the input samples. At that time, the loss function affects the parameters The first gradient, i.e. ; The first step in adapting a large model to the domain The first layer The semantic importance of each channel; Represents the vertical domain calibration dataset Belongs to the category The sample set, This represents the total number of sample categories in the vertical domain calibration dataset; Domain Adaptation Large Model Network hierarchical semantic importance score The calculation formula is as follows: in, For domain adaptation, the first in the large model Total number of channels in the layer.

3. The semantically preservative parameter compression method for embedded platforms as described in claim 1, characterized in that, The step involves performing hardware-aware structured pruning on the domain adaptation model based on semantic importance and a hardware lookup table to obtain a pruned, simplified model, including: An optimization target model is constructed based on semantic importance and a hardware lookup table. The optimization target model is solved to obtain the optimal solution. Based on the optimal solution, a hardware-aware structured pruning operation is performed on the domain adaptation model to obtain a pruned simplified model. The optimal solution of the optimization target model satisfies the following: maximizing the retention of semantic scores while satisfying the target latency.

4. The semantically preservative parameter compression method for embedded platforms as described in claim 3, characterized in that, The optimization target model includes the objective function and constraints; The objective function is as follows: The constraints are as follows: in, As a mask, it represents the first... Layer Should each channel be retained? The first step in adapting a large model to the domain The first layer The semantic importance of each channel; Delay for the goal; For domain adaptation, the first in the large model Total number of channels in the layer; The total number of network layers for domain-adaptive large models; The first one is obtained based on the constructed hardware lookup table. The measured inference time of a layer at a specific pruning rate.

5. The semantically preservative parameter compression method for embedded platforms as described in any one of claims 1 to 4, characterized in that, Under a defined bit-width constraint, semantically preserving mixed-precision quantization is performed on the simplified model to obtain an initial lightweight model, including: Hierarchical adaptive bit width selection based on KL divergence determines the bit width of each level in the simplified model, specifically including: Define the set of optional bit widths for each layer of the simplified model. bit; The first step in defining the simplified model Layer in position width Quantization perturbation ; A comprehensive consideration of the semantic importance and quantization perturbation of each layer in the simplified model Under the premise of meeting the overall model compression rate or hardware storage constraints, high bit width is allocated to layers with large quantization perturbation and high semantic importance, and low bit width is allocated to layers with small quantization perturbation or low semantic importance. Under a defined bit width constraint, symmetric quantization is performed, and a learnable truncation threshold is introduced for the activation values. The numerical mapping is performed using the following quantization formula: in, This represents the original parameter values ​​in the simplified model; This represents the quantized parameter value; To be based on bit width A defined scaling factor.

6. The semantically preservative parameter compression method for embedded platforms as described in any one of claims 1 to 4, characterized in that, The process of performing feature-level semantic compensation distillation on the initial lightweight model to obtain the final lightweight large model after accuracy restoration includes: A teacher-student network is constructed, with the teacher network serving as a domain-adapted large model and the student network as an initial lightweight model. The high-dimensional features output by the teacher network guide the student network in feature alignment. During the guidance process, the following total loss function is used to fine-tune the initial lightweight model: in, For mission losses; For knowledge distillation loss; and The teacher network and the student network are respectively in the 1st Feature maps of key layers; It is a lightweight linear transformation layer used to match the feature dimensions of the student network and the teacher network; The set is automatically selected based on the highest response value among the semantic importance of network layers in the domain-adapted large model.

7. The semantically preservative parameter compression method for embedded platforms as described in any one of claims 1 to 4, characterized in that, The logical structure of the hardware lookup table is represented as follows: {(operator type, input dimension, number of output channels, hardware configuration): physical latency}.

8. A semantically preservative parameter compression system for embedded platforms, characterized in that, The semantically preserving parameter compression system for embedded platforms is used to perform the semantically preserving parameter compression method for embedded platforms as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program for semantically preserving parametric compression for embedded platforms, wherein the computer program causes a computer to execute the semantically preserving parametric compression method for embedded platforms as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing semantically preserving parametric compression methods for embedded platforms as described in any one of claims 1 to 7.

Images