Task-oriented pruning method based on structural space consistency

Abstract

This invention discloses a task-oriented pruning method based on structural space consistency, comprising: collecting a target dataset containing sampled images and noisy images; inputting the sampled images into a teacher model and the noisy images into a pruning model; constructing a Gram matrix; decomposing it to obtain a spatial basis matrix and a channel basis matrix, and truncating them to construct the pruning model basis matrix and the teacher model basis matrix; constructing an affinity matrix and a basis consistency loss; using a minimum subset of the target dataset, based on the singular values ​​of the spatial Gram matrix and the channel Gram matrix output by the teacher model, obtaining the energy probability distribution within the spatial subspace and the channel subspace, calculating the spatial complexity and channel complexity, and obtaining the reconstructed basis consistency loss; and deploying the pruning model to all downstream tasks, completing the construction and deployment of the pruning model for downstream tasks. This invention can alleviate the problems of decreased robustness and insufficient task specificity in the pruning process of existing methods.

Description

Technical Field

[0001] This invention relates to the field of visual basic model pruning and model compression technology, specifically to a task-oriented pruning method based on structural space consistency. Background Technology

[0002] With the development of visual fundamental models, especially the emergence of the DINO series models, the learning ability of visual representations has been significantly improved, enabling visual fundamental models to achieve competitive performance on a variety of downstream visual tasks. However, despite the good performance of visual fundamental models in cross-task transfer, their huge computational overhead still poses a significant challenge to deployment on resource-constrained edge devices. Therefore, efficient compression of visual fundamental models without significantly sacrificing performance has become a research focus.

[0003] Among various model compression paradigms, one-time structured pruning methods that eliminate the need for retraining have been considered a promising approach in recent years. These methods dynamically evaluate parameter importance using the gradient of the loss function, thereby enabling the rapid generation of lightweight subspaces, making them particularly suitable for large-scale vision foundation models where heavy training costs are prohibitive.

[0004] Existing one-time structured pruning methods mostly focus on rigid point-to-point alignment of tokens for pruning. Related methods include LAMP, SNIP, SNOWS, and SnapViT for post-processing of pre-trained models. However, the alignment mechanism of these methods weakens the robustness of the pruned subspace. In contrast, there are also several works that lightweight the visual base model (such as Efficientsam, Theia, Tinysam), which mainly rely on knowledge distillation to transfer the computationally expensive encoder capabilities to lightweight networks. However, this type of method is different from the training-free pruning route of "directly selecting parameters from the visual base model encoder and generating networks with different sparsity".

[0005] Existing training-free, one-time structured pruning methods prune DINOv3 by forcing rigid alignment between tokens from global and local pruning. SnapViT employs a genetic algorithm, specifically xNES, to facilitate global subspace search. It assesses the importance of each block by calculating relevance scores between subspaces, and then determines the importance of each parameter using a local Hessian matrix. The parameters are then sorted and pruned according to given sparsity constraints.

[0006] However, training-free pruning methods still face two key problems: robustness degradation and insufficient task specificity. Existing training-free one-time structured pruning methods often suffer from reduced robustness in practical applications. These methods typically rely on "rigid point-to-point lexical alignment" between the original and pruned models. This local alignment strategy causes the subspace to tend to memorize specific numerical features rather than maintain the global semantic topology. Because these methods emphasize precise lexical matching rather than structural consistency, they struggle to inherit the robust manifold structure of the underlying visual model, leading to representation degradation and increased sensitivity to noise. Furthermore, existing training methods often suffer from insufficient task specificity. Existing training-free methods typically follow a fixed process: optimization on a single general dataset (e.g., ImageNet) followed by deployment of the resulting "general subspace" to all downstream tasks. This paradigm ignores the differences in data distribution between different downstream tasks and the target domain. In edge computing scenarios, model capacity has already been sacrificed for inference speed and memory usage. Further emphasizing task-agnostic shared subspaces will inevitably lead to further compromises in accuracy, resulting in an unsatisfactory accuracy-efficiency tradeoff for specific downstream applications. Summary of the Invention

[0007] The purpose of this invention is to provide a task-oriented pruning method based on structural space consistency. By designing a one-time structured pruning method for training a visual base model for downstream tasks without retraining, this invention addresses the problems of decreased robustness and insufficient task specificity caused by existing methods when pruning on a single general dataset based on rigid point-to-point token alignment. This allows for the rapid acquisition of a subspace that can inherit robust feature representations from DINOv3 without retraining.

[0008] To achieve the above functions, this invention designs a task-oriented pruning method based on structural space consistency, executing the following steps S1-S4 to complete the construction and deployment of a pruning model oriented towards downstream tasks:

[0009] Step S1: Collect the target dataset, which includes sampled images and noisy images of the target object. Construct a teacher model and a pruning model. Input the sampled images into the teacher model and the noisy images into the pruning model. Construct the spatial Gram matrix and the channel Gram matrix for the sampled images and the noisy images respectively. Obtain the spatial basis matrix and the channel basis matrix through singular value decomposition.

[0010] Step S2: Truncate the spatial basis matrix and channel basis matrix of the sampled image and the noisy image respectively to construct the teacher model basis matrix and the pruned model basis matrix; further construct the affinity matrix, basis consistency loss and residual loss;

[0011] Step S3: Using the minimum subset of the target dataset, based on the singular values ​​of the spatial Gram matrix and channel Gram matrix output by the teacher model, obtain the energy probability distribution in the spatial subspace and channel subspace, calculate the spatial complexity and channel complexity, and obtain the reconstructed basis consistency loss based on the basis consistency loss.

[0012] Step S4: Deploy the pruning model to all downstream tasks to complete the construction and deployment of the pruning model for downstream tasks.

[0013] As a preferred technical solution of the present invention: In step S1, the teacher model is constructed based on the native DINOv3 model, and the pruning model is constructed based on the pruning-guided DINOv3 model. The pruning-guided DINOv3 model is based on the native DINOv3 model and is obtained by pruning with a preset sparsity.

[0014] As a preferred technical solution of the present invention, the specific steps of step S1 are as follows:

[0015] Step S1.1: Input the sampled image into the teacher model and the noisy image into the pruning model. After passing through the encoder, the feature matrices of the sampled image and the noisy image are generated as follows:

[0016] ;

[0017] in, The number of lexical units, For channel dimension, Let be the feature matrix of the image, containing the feature matrix of the sampled image and the feature matrix of the noisy image, wherein the feature matrix of the sampled image is represented as... The feature matrix of a noisy image is represented as ;

[0018] Step S1.2: Define the generalized Gram matrix and extract the manifold structure by performing singular value decomposition on the Gram matrix, as follows:

[0019] ;

[0020] in, Represents the Gram matrix. The feature matrix is ​​expanded along the target dimension. It is a singular value diagonal matrix. It is an orthogonal basis matrix;

[0021] Step S1.3: Calculate the spatial Gram matrix based on the Gram matrix as follows:

[0022] ;

[0023] in, The spatial Gram matrix is ​​represented as follows: The spatial Gram matrix of the noisy image output by the pruning model is expressed as... The spatial Gram matrix of the sampled image output by the teacher model is represented as follows: , The feature matrix of the image, The number of lexical units;

[0024] For the space Gram matrix By applying singular value decomposition, the spatial basis matrix is ​​obtained. The spatial basis matrix of the noisy image output by the pruning model is represented as: The spatial basis matrix of the sampled image output by the teacher model is represented as: ;

[0025] Step S1.3: Calculate the channel Gram matrix based on the Gram matrix as follows:

[0026] ;

[0027] in, The channel Gram matrix is ​​represented as follows: The channel Gram matrix of the noisy image output by the pruning model is expressed as... The channel Gram matrix of the sampled image output by the teacher model is represented as follows: , For channel dimensions;

[0028] For channel Gram matrix Singular value decomposition is applied to obtain the channel basis matrix. The channel basis matrix of the noisy image output by the pruning model is represented as follows: The channel basis matrix of the sampled image output by the teacher model is represented as follows: .

[0029] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0030] Step S2.1: For the spatial basis matrix of the noisy image output by the pruning model... Channel basis matrix Perform top-k truncation to obtain the basis matrix of the pruning model. ; the spatial basis matrix of the sampled images output by the teacher model Channel basis matrix Perform top-k truncation to obtain the teacher model basis matrix. ;in, The number of lexical units, The number of principal components truncated for the top-k;

[0031] Step S2.2: Based on the pruning model basis matrix Teacher model basis matrix Define the affinity matrix As shown in the following formula:

[0032] ;

[0033] Based on affinity matrix Define the basis consistency loss As shown in the following formula:

[0034] ;

[0035] in, Let K be the Frobenius norm, and K be the number of principal components. Affinity matrix The element in the i-th row and j-th column;

[0036] Constructing residual loss As shown in the following formula:

[0037] ;

[0038] in, Represents the identity matrix. The feature matrix representing a noisy image; This represents the basis matrix of the teacher model.

[0039] As a preferred technical solution of the present invention, the specific steps of step S3 are as follows:

[0040] Step S3.1: Let Let the singular value of the i-th element of the spatial Gram matrix output by the teacher model be denoted as . Let j represent the singular value of the j-th element of the channel Gram matrix output by the teacher model. Normalize each singular value to obtain the energy probability distribution within the corresponding spatial subspace and channel subspace:

[0041] ;

[0042] ;

[0043] in, express The corresponding energy probability distribution of the spatial subspace, This represents the singular value of the k-th principal component of the spatial Gram matrix output by the teacher model; express The corresponding energy probability distribution of the channel subspace, This represents the singular value of the k-th principal component in the channel Gram matrix output by the teacher model; K is the number of principal components.

[0044] Step S3.2: Calculate the space complexity and channel complexity as follows:

[0045] ;

[0046] ;

[0047] in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; The channel Gram matrix represents the sampled image output by the teacher model;

[0048] Step S3.3: Based on basis consistency loss The reconstructed basis consistency loss is as follows:

[0049] ;

[0050] in, This represents the base consistency loss during reconstruction. Representation basis consistency loss Loss in spatial dimensions, Representation basis consistency loss Loss in the channel dimension; express Dynamic weight loss, express Dynamic weight loss;

[0051] Dynamic weight loss , The calculation is as follows:

[0052] ;

[0053] ;

[0054] in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; This represents the channel Gram matrix of the sampled image output by the teacher model.

[0055] The present invention also designs a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the task-oriented pruning method based on structural space consistency.

[0056] The present invention also designs a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the task-oriented pruning method based on structural space consistency.

[0057] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0058] 1. This invention abandons the traditional "rigid point-to-point term alignment" strategy, which easily leads to model overfitting and noise sensitivity, and innovatively proposes a Gram-anchored subspace consistency alignment method. By designing basis-independent constraints and residual constraints, the pruned subspace is forced to maintain structural consistency with the original DINOv3 model in both spatial and channel dimensions of the Gram subspace. This mechanism enables the lightweight subspace to effectively inherit the robust characteristic manifold structure and anti-interference ability of the original large model.

[0059] 2. To address the unsatisfactory accuracy-efficiency trade-off caused by existing methods using a single general subspace to adapt to all downstream tasks, this invention introduces an adaptive mechanism based on spectral entropy. This mechanism dynamically adjusts the optimization target weights during the pruning process by quantifying the feature information density of the target dataset in both spatial and channel dimensions. This allows the pruning algorithm to adaptively adjust according to the inherent representational requirements of downstream tasks (such as classification, segmentation, and detection).

[0060] 3. This invention provides a one-time pruning scheme for training without retraining, and designs an efficient subspace decomposition and alignment process. Compared with existing mainstream methods, this invention has overwhelming advantages in computational efficiency and resource consumption;

[0061] 4. The method of the present invention can generate subspaces with different sparsity in a very short time, and exhibits stable and excellent performance at all sparsity levels. Attached Figure Description

[0062] Figure 1 This is a flowchart of a task-oriented pruning method based on structural space consistency according to an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram illustrating the process of achieving consistency between the spatial and channel Gram anchoring subspaces according to an embodiment of the present invention;

[0064] Figure 3This is a comparative analysis diagram of the present invention and existing methods according to embodiments of the present invention;

[0065] Figure 4 This is a graph showing the parameter retention rate and overlap rate of the training-free one-time structured pruning and task-specific methods for different blocks provided in the embodiments of the present invention. Detailed Implementation

[0066] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0067] This invention provides a task-oriented pruning method based on structural space consistency, referring to... Figure 1 Perform the following steps S1-S4 to complete the construction and deployment of the pruning model for downstream tasks:

[0068] Step S1: Collect the target dataset, which includes sampled images and noisy images of the target object. Construct a teacher model and a pruning model. Input the sampled images into the teacher model and the noisy images into the pruning model. Construct the spatial Gram matrix and the channel Gram matrix for the sampled images and the noisy images respectively. Obtain the spatial basis matrix and the channel basis matrix through singular value decomposition.

[0069] The teacher model is built based on the native DINOv3 model, and the pruning model is built based on the pruning-guided DINOv3 model. The pruning-guided DINOv3 model is based on the native DINOv3 model and is obtained by pruning with a preset sparsity.

[0070] The strong generalization ability of the DINOv3 model stems from the rich structure and semantic relationships encoded in its dense feature representation. Previous works have forced rigid point-by-point alignment of tokens, but this precise numerical reconstruction imposes overly strict constraints, which may cause the model to overfit to redundant correlations and high-frequency noise. Therefore, this invention proposes to use singular value decomposition to decompose the Gram matrix into subspaces spanned by dense features, and to regularize the consistency of the subspaces anchored to the Gram matrix.

[0071] Step S1 involves subspace decomposition of the Gram matrix and extraction of its dominant subspace basis to avoid the sensitivity of rigid alignment to high-frequency noise; the specific steps are as follows:

[0072] Step S1.1: Input the sampled image into the teacher model and the noisy image into the pruning model. After passing through the encoder, the feature matrices of the sampled image and the noisy image are generated as follows:

[0073] ;

[0074] in, The number of lexical units, For channel dimension, Let be the feature matrix of the image, containing the feature matrix of the sampled image and the feature matrix of the noisy image, wherein the feature matrix of the sampled image is represented as... The feature matrix of a noisy image is represented as ;

[0075] Step S1.2: Define a generalized Gram matrix to characterize the second-order correlation of features along a specific dimension, and extract the manifold structure by performing singular value decomposition on the Gram matrix, as follows:

[0076] ;

[0077] in, Represents the Gram matrix. The feature matrix is ​​expanded along the target dimension. It is a singular value diagonal matrix. It is an orthogonal basis matrix whose column vectors span the dominant subspace of the Gram matrix;

[0078] Step S1.3: To capture the topological relationships between lexical units, features are aggregated along the channel dimension. Based on the Gram matrix, the spatial Gram matrix is ​​calculated as follows:

[0079] ;

[0080] in, The spatial Gram matrix is ​​represented as follows: The spatial Gram matrix of the noisy image output by the pruning model is expressed as... The spatial Gram matrix of the sampled image output by the teacher model is represented as follows: , The feature matrix of the image, The number of lexical units;

[0081] For the space Gram matrix By applying singular value decomposition, the spatial basis matrix is ​​obtained. It identifies the dominant modality of inter-word correlation and can group spatially coherent patches based on feature similarity; the spatial basis matrix of the noisy image output by the pruning model is represented as... The spatial basis matrix of the sampled image output by the teacher model is represented as: ;

[0082] Step S1.3: To capture the semantic relevance between channels, treat the lexical units as samples and calculate the channel Gram matrix according to the Gram matrix as follows:

[0083] ;

[0084] in, The channel Gram matrix is ​​represented as follows: The channel Gram matrix of the noisy image output by the pruning model is expressed as... The channel Gram matrix of the sampled image output by the teacher model is represented as follows: , For channel dimensions;

[0085] For channel Gram matrix Singular value decomposition is applied to obtain the channel basis matrix. This is used to form a compact description of global semantics; the channel basis matrix of the noisy image output by the pruning model is represented as... The channel basis matrix of the sampled image output by the teacher model is represented as follows: .

[0086] Reference Figure 2 For the channel Gram matrix The channel basis matrix can be obtained by decomposition. This decouples abstract concepts from their spatial instances. Therefore, the channel basis matrix... As a compact descriptor of global semantics, it answers the question of "What is described".

[0087] By extracting simultaneously and The alignment between the native teacher model and the pruning model is transformed into the consistency problem between the spatial subspace and the channel subspace.

[0088] Step S2: Truncate the spatial basis matrix and channel basis matrix of the sampled image and the noisy image respectively to construct the teacher model basis matrix and the pruned model basis matrix; further construct the affinity matrix, basis consistency loss and residual loss;

[0089] Considering the non-uniqueness of the singular vectors (basis) obtained from singular value decomposition, directly performing MSE alignment element-by-element on the basis vectors is inappropriate. Therefore, step S2 employs a basis-independent constraint to align the subspace itself, and further proposes a residual constraint to suppress redundant information outside the aligned subspace; the specific steps of step S2 are as follows:

[0090] Step S2.1: For the spatial basis matrix of the noisy image output by the pruning model... Channel basis matrix Perform top-k truncation to obtain the basis matrix of the pruning model. ; the spatial basis matrix of the sampled images output by the teacher model Channel basis matrix Perform top-k truncation to obtain the teacher model basis matrix. ;in, The number of lexical units, The number of principal components truncated for the top-k;

[0091] Step S2.2: Based on the pruning model basis matrix Teacher model basis matrix Define the affinity matrix As shown in the following formula:

[0092] ;

[0093] Based on affinity matrix Define the basis consistency loss Consistency is achieved by maximizing subspace overlap (correlation energy), based on consistency loss. As shown in the following formula:

[0094] ;

[0095] in, Let K be the Frobenius norm, and K be the number of principal components. Affinity matrix The element in the i-th row and j-th column, where i and j represent the basis matrix of the pruning model, respectively. Teacher model basis matrix The index of the corresponding element.

[0096] This embodiment further presents the conclusion of "basic invariance": basis consistency loss. It depends only on the subspace spanned by the basis matrices, and on the basis matrices of the pruning model. Teacher model basis matrix Any orthogonal transformation remains unchanged;

[0097] To prove this conclusion, let's assume... For any orthogonal matrix (i.e. , (The identity matrix). The non-uniqueness of singular value decomposition implies that an effective basis can be the basis matrix of the transformed pruned model. and the transformed teacher model basis matrix And test the basis consistency loss in both cases. Invariance of:

[0098] Case 1: Synchronous Transformation. Consider the scenario where both the original basis and the pruned basis undergo arbitrary rotations. Substituting the transformed basis into the Frobenius norm term yields:

[0099] ;

[0100] Review the unitary invariance of the Frobenius norm (i.e., for any matrix) There exist orthogonal matrices , ,have ), we have the following formula:

[0101] ;

[0102] Case 2: One-sided transformation. In a task-specific pruning setting, the native DINOv3 model acts as a frozen teacher model, with the teacher model's basis matrix... It is fixed ( ) Pruning model basis matrix It evolves during the optimization process and may rotate arbitrarily. Therefore, the objective becomes:

[0103] ;

[0104] because It is orthogonal; multiplication represents isometric rotation, which preserves the Frobenius norm, i.e. Therefore, regardless of the orientation of the pruning base, Both can always align the subspace of the pruning model with the subspace of the teacher model.

[0105] Only The alignment of the main direction is guaranteed, and the feature matrix of the noisy image generated by the pruning model is... It inevitably contains redundant information orthogonal to the target subspace. To explicitly filter out this noise, this invention employs geometric projection, making... This is the orthogonal projection matrix projected onto the native DINOv3 model subspace. The feature matrix of the noisy image generated by the pruning model. It can be uniquely decomposed into two mutually orthogonal components:

[0106] ;

[0107] The first term represents the feature successfully projected onto the robust teacher subspace. The second term corresponds to the residual error or noise located outside the target manifold.

[0108] Construct the following residual loss The energy used to minimize irrelevant information:

[0109] ;

[0110] in, Represents the identity matrix. The feature matrix representing a noisy image; This represents the basis matrix of the teacher model. Suppress activations that are structurally inconsistent with the Gram matrix anchoring of the teacher model, thereby improving feature purity.

[0111] Step S3: Using the minimum subset of the target dataset, based on the singular values ​​of the spatial Gram matrix and channel Gram matrix output by the teacher model, obtain the energy probability distribution in the spatial subspace and channel subspace, calculate the spatial complexity and channel complexity, and obtain the reconstructed basis consistency loss based on the basis consistency loss.

[0112] This invention injects customized attributes for each task into a training-free pruning pipeline. First, instead of performing general pruning on a general dataset (e.g., ImageNet, which ignores different data distributions), this embodiment selects a minimal subset of N samples from the target dataset for pruning. This allows the subspace to capture the latent data manifold of the target domain without labeling. Notably, thanks to the design of this embodiment, even with multiple searches, it maintains lower latency than previous methods. Second, spectral entropy is introduced to adaptively weight the pruning target based on the information density of the feature manifold on the spatial and channel axes. The specific steps of step S3 are as follows:

[0113] Step S3.1: Let Let the singular value of the i-th element of the spatial Gram matrix output by the teacher model be denoted as . Let j represent the singular value of the j-th element of the channel Gram matrix output by the teacher model. Normalize each singular value to obtain the energy probability distribution within the corresponding spatial subspace and channel subspace:

[0114] ;

[0115] ;

[0116] in, express The corresponding energy probability distribution of the spatial subspace, This represents the singular value of the k-th component of the spatial Gram matrix output by the teacher model; express The corresponding energy probability distribution of the channel subspace, This represents the singular value of the k-th component of the channel Gram matrix output by the teacher model; K is the number of principal components.

[0117] Step S3.2: Calculate the space complexity and channel complexity as follows:

[0118] ;

[0119] ;

[0120] in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; The channel Gram matrix represents the sampled image output by the teacher model;

[0121] Theoretically, the spectral decay of the feature covariance matrix reflects the degree of representation collapse within the lexical space. For example, classification tasks optimize global semantics that are insensitive to shifts, which drives spatial lexical units to tend towards consistency, thus concentrating the energy of the spatial Gram matrix on a few dominant principal components (lower spectral density). Conversely, dense prediction tasks (such as semantic segmentation and correspondence matching) should preserve high-frequency topological boundaries and pixel-level semantics. This prompts this invention to penalize representation collapse and force a flatter distribution of singular values ​​(lower scalar values) along the spatial axis. ).

[0122] Step S3.3: Based on basis consistency loss Using a task-aware approach, the reconstructed basis consistency loss is as follows:

[0123] ;

[0124] in, This represents the base consistency loss during reconstruction. Representation basis consistency loss Loss in spatial dimensions, Representation basis consistency loss Loss in the channel dimension; express Dynamic weight loss, express Dynamic weight loss;

[0125] Dynamic weight loss , The calculation is as follows:

[0126] ;

[0127] ;

[0128] in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; This represents the channel Gram matrix of the sampled image output by the teacher model.

[0129] Reference Figure 1 During the pruning process, the gradients of each parameter are calculated by backpropagation, and then the parameters are sorted in descending order of magnitude to select the parameters, and subspaces with different sparsity levels are generated.

[0130] The reconstructed basis consistency loss ensures that the present invention can adaptively adjust for a given downstream task and take into account its potential characterization requirements.

[0131] Step S4: Deploy the pruning model to all downstream tasks to complete the construction and deployment of the pruning model for downstream tasks.

[0132] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the task-oriented pruning method based on structural space consistency.

[0133] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the task-oriented pruning method based on structural space consistency.

[0134] To verify the effectiveness and rationality of the method of the present invention, a comprehensive evaluation was conducted on multiple vision tasks with sparsity levels ranging from 10% to 30%. For broader comparison, this embodiment also evaluated the performance of other state-of-the-art methods at sparsity levels of 40%, 50%, and 60%.

[0135] Reference Figure 3 Existing methods force rigid alignment between CLS tokens, while the method of this invention aligns the subspaces induced by the intrinsic Gram anchoring of DINOv3, thereby enabling the subspaces to inherit robust feature representations of DINOv3.

[0136] The following datasets were used for various tasks: ADE20K and PASCAL VOC 2012 for semantic segmentation; COCO for object detection; NYUv2 for depth estimation; DAVIS-2017 for video object segmentation; FG3DCAR, JODS, and SBD for semantic matching; and ImageNet for image classification. ADE20K is a large semantic segmentation dataset containing 150 categories, with 20,210 / 2,000 / 3,000 images used for training / validation / testing respectively. PASCALVOC 2012 contains 20 semantic categories and one background category, with 1,464 / 1,449 / 1,456 images used for training / validation / testing respectively. COCO has 80 object detection categories, containing 118,287 training images, 5,000 validation images, and 40,000 test images. NYUv2 was split into 795 training images and 654 test images. DAVIS-2017 contains 60 training videos and 30 validation videos. The three datasets used for semantic matching together comprise 400 pairs of test images with densely labeled correspondences. ImageNet contains 1,000 classes, with 1.2 million images allocated for training, 50,000 for validation, and 100,000 for testing.

[0137] In terms of model architecture, the encoder uniformly adopts the DINOv3 model based on the ViT-B / 16 architecture across various sparsity levels. Object detection uses a standard deformable attention decoder. Linear probing is used for classification, semantic segmentation, and depth estimation. Video object segmentation and semantic matching are evaluated through feature matching without introducing any decoder. The performance of classification, semantic segmentation, object detection, and semantic matching is evaluated using Top-1 accuracy, mean Intersection over Union (mIoU), mean accuracy (mAP) at IoU thresholds in the range [0.5, 0.05, 0.95], and PCK@0.05, respectively. Depth estimation uses relative error ARel (lower is better) and accuracy. The performance of video object segmentation is evaluated based on the average region similarity (higher is better). Average profile accuracy ,as well as and average To measure.

[0138] The model was implemented using the PyTorch framework. Pruning was performed on a single GPU, while all other experiments were conducted on four NVIDIA A100 GPUs. For semantic segmentation, the learning rate was set to 1e-3, and the model was trained for 100 epochs. For object detection, the learning rate was 5e-4, and the model was trained for 72 epochs. For depth estimation, the learning rate was 1e-6, and the model was trained over 60 epochs. The batch size for all tasks was set to 16. For image classification, the learning rate was 5e-3, the batch size was 1024, and the model was trained for 20 epochs. Video object segmentation and semantic matching tasks did not require training; the pruned models at different sparsity levels were directly validated on the dataset. In this embodiment, the number of samples N=1000 and the number of principal components K=192 were set.

[0139] This embodiment compares the method of the present invention with other methods on the DAVIS-2017 dataset used for video object segmentation, as shown in Table 1.

[0140] The method of this invention is compared with two categories of methods. The first category is training-based methods, such as HydraViT and EA-ViT. HydraViT stacks attention heads and dynamically changes the embedding dimension and the number of attention heads during training, allowing the overall model to contain multiple subnetworks that can be directly reasoned. EA-ViT starts from a single pre-trained ViT and forms a resilient model that can dynamically select subnetworks of different sizes according to budget by jointly adjusting the multilayer perceptron (MLP) ratio, the number of attention heads, the embedding dimension, and the network depth. The second category is training-free methods, such as LAMP, NVIT, SNIP, SNOWS, and SnapViT. LAMP focuses on measuring the importance of layers in the network, thereby applying different proportions of parameter pruning to each layer. NVIT uses Hessian-aware importance metrics to globally evaluate the sensitivity of different structural components in the ViT and performs iterative structural pruning and parameter reallocation accordingly. SNIP uses connection sensitivity to evaluate the impact of each weight on the loss function and prunes unimportant connections all at once before network training. SNOWS primarily maintains the Hessian matrix to measure the importance of each parameter to performance, while SnapViT measures parameter sensitivity based on the parameter gradient generated by MSE loss, serving as a standard for parameter pruning.

[0141] At various pruning sparseness levels, the method of this invention is , and The proposed method significantly outperforms training-free methods in terms of metrics. Notably, at 20% sparsity, the proposed method improves the corresponding metrics by more than 4.2%, 4.4%, and 4.0%, respectively. Compared to training-based pruning methods, the proposed method also outperforms HydraViT, although it lags slightly behind EA-ViT in performance. These results highlight the superiority of the backbone network obtained using the proposed method.

[0142] Table 1. Application of DAVIS-2017 video object segmentation , and Quantitative results

[0143]

[0144] Table 2 reports the semantic matching results on the FJS dataset. Across all datasets and pruning sparsity levels, the proposed method consistently outperforms existing training-free methods and the training-based HydraViT. For example, at 30% sparsity, the proposed method achieves performance improvements of 5.1%, 12.1%, and 5.5% over training-free methods on FG3DCar, JODS, and PASCAL, respectively, and surpasses HydraViT by 0.9%, 0.6%, and 0.3%. These results demonstrate that the proposed pruning strategy effectively preserves the fine-grained, dense feature matching capabilities of DINOv3.

[0145] Table 2. Quantitative results of semantic matching on the FG3DCar / JODS / SBD (FJS) dataset.

[0146]

[0147] Table 3 compares the performance of the proposed method with other methods on the COCO dataset. The proposed method outperforms the training-free method and achieves competitive performance with the training-based EA-ViT. For example, at 30% sparsity, the proposed method improves the mAP by 2.8% compared to the former, while lagging behind the latter by only 0.4%. These results further demonstrate that the proposed method more effectively preserves the detection capabilities of the DINOv3 model.

[0148] Table 3. Quantitative results of target detection on COCO

[0149]

[0150] Table 4 presents the experimental results comparing the proposed method with other methods on the ADE20K dataset. At sparsity levels of 30%, 20%, and 10%, the proposed method outperforms the training-free method by 1.2%, 1.7%, and 0.8% in mIoU, respectively, and significantly surpasses the training-based HydraViT. These results demonstrate that the proposed method preserves robust semantic information in the subspace pruned from DINOv3.

[0151] Table 4. Quantitative results of semantic segmentation on ADE20K

[0152]

[0153] Table 5 presents the depth estimation results on the NYUv2 dataset and compares the method of this invention with other methods. The method of this invention performs well on ARa and... The proposed method outperforms training-free pruning methods in all aspects, while achieving performance comparable to training-based methods. Furthermore, when pruning an equal number of parameters, the model generated by the proposed method performs closer to the original DINOv3 model than the model generated by training-free pruning methods. These results across multiple dense prediction tasks demonstrate that the proposed method effectively preserves the robust dense feature representation of the original DINOv3 model during the pruning process.

[0154] Table 5. Using ARel / on NYUv2 Quantitative results of depth estimation

[0155]

[0156] Table 6 presents the image classification results of the proposed method on the ImageNet dataset and compares it with other methods. Even without Gram anchoring, the proposed method outperforms training-free pruning methods solely based on basis independence and residual constraints. It achieves gains of 1.8%, 1.2%, and 0.8% at sparsity levels of 10%, 20%, and 30%, respectively. Furthermore, consistent with dense prediction tasks, the proposed method achieves competitive performance compared to training-based pruning methods.

[0157] Table 6. Quantitative results in ImageNet image classification

[0158]

[0159] Table 7 compares the generalization ability of the proposed method with other methods by using a model trained on ADE20K to perform inference on the PASCAL VOC 2012 dataset. The proposed method consistently outperforms the training-free pruning method at all sparsity levels, achieving a significant gain of 1.5% mIoU at 20% sparsity. These results provide strong evidence that the proposed pruning strategy enables the subspace to effectively inherit the generalization ability of the native DINOv3 model.

[0160] Table 7. Quantitative results in ImageNet image classification

[0161]

[0162] Local dense feature alignment Used as a baseline, all ablation experiments were performed on the ADE20K dataset with 20% sparsity.

[0163] Table 8 shows a comparison of the algorithmic complexity of the method of this invention with other methods. Compared with training-free and training-based methods, the method of this invention achieves encouraging performance while significantly reducing pruning time and GPU memory usage. Specifically, it requires only 20.9% of the time and 45.5% of the GPU memory required by SnapViT, while achieving a 1.7% improvement in mIoU. Although the method lags behind training-based pruning methods (such as EA-ViT) by 0.9% in performance, it requires only 0.44% of the time and 28.2% of the GPU memory. These results demonstrate that competitive performance can be achieved with minimal computational overhead.

[0164] Table 8. Complexity analysis using a single A100 GPU under the same settings.

[0165]

[0166] Tables 9 and 10 report the results for each component and different Gram matrices, respectively. In Tables 9 and 10, √ indicates that the corresponding component is retained. Basis-independent constraints, residual constraints, and task-specific adaptability complement each other, enabling the model to achieve excellent experimental performance. Furthermore, combining the spatial Gram matrix and the channel Gram matrix encourages the pruned model to learn spatial contextual information and channel-level semantic representations aligned with the DINOv3 model. The complementary effect of these two Gram matrices allows the model to achieve optimal performance.

[0167] Table 9. Component analysis of basis-independent residual constraints and task-specific constraints on ADE20K with 20% sparsity.

[0168]

[0169] Table 10. Component analysis of spatial and channel Gram anchoring on ADE20 with 20% sparsity.

[0170]

[0171] Tables 11 and 12 present the ablation studies for the sampling size N and the number of principal components K, respectively. The results show that the method of this invention is largely insensitive to the number of samples. For example, increasing the sampling size from 500 to 1000 only brings a small performance improvement of 0.2%. Notably, even with only 500 samples, the method of this invention outperforms SnapViT trained with 1000 samples by 1.5% in mIoU. These findings demonstrate that Gram anchoring with subspace consistency effectively captures the structural properties of datasets even with limited data. As shown in Table 12, the performance improves proportionally with the increase in the number of principal components K. The improvement is particularly significant when k increases from 96 to 192, after which the gain becomes more gradual. Considering the trade-off between computational efficiency and performance, K was set to 192.

[0172] Table 11. Parameter analysis of sample size N under 20% sparsity setting on ADE20K.

[0173]

[0174] Table 12. Parameter analysis of the number of principal components K at 20% sparsity setting on ADE20K.

[0175]

[0176] Figure 4 The parameter retention and overlap rates of the training-free one-time structured pruning method and the method of this invention for different task-specific methods of different blocks are shown. Figure 4 As shown in the left figure, the parameter retention rate of the general method fluctuates greatly between layers 3 and 10, while the method of the present invention shows a more stable change, indicating that the method of the present invention better preserves the fine-grained texture features of the DINOv3 model. Figure 4 The curves in the right figure show that both pruning paradigms pay more attention to the information of the attention layer within a few intermediate blocks.

[0177] In summary, this invention aims to alleviate the robustness degradation and task-specific deficiencies in existing works. Specifically, it constructs Gram anchoring matrices from both spatial and channel perspectives and applies subspace decomposition to extract the corresponding basis. This invention designs basis-independent constraints and residual constraints to enhance subspace consistency between the native DINOv3 model and the pruning-oriented DINOv3 model in both domains, enabling the subspace to inherit robust feature representations from DINOv3. Furthermore, this invention proposes spectral entropy adaptation to quantify the information density of the feature manifold along the spatial and channel dimensions, thereby adapting the pruning objective to specific downstream tasks. Extensive experiments on six tasks across nine datasets demonstrate that this invention achieves state-of-the-art performance with minimal computational overhead compared to previous methods.

[0178] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A task-oriented pruning method based on structural space consistency, characterized in that, Perform the following steps S1-S4 to complete the construction and deployment of the pruning model for downstream tasks: Step S1: Collect the target dataset, which includes sampled images and noisy images of the target object. Construct a teacher model and a pruning model. Input the sampled images into the teacher model and the noisy images into the pruning model. Construct the spatial Gram matrix and the channel Gram matrix for the sampled images and the noisy images respectively. Obtain the spatial basis matrix and the channel basis matrix through singular value decomposition. Step S2: Truncate the spatial basis matrix and channel basis matrix of the sampled image and the noisy image respectively to construct the teacher model basis matrix and the pruned model basis matrix; further construct the affinity matrix, basis consistency loss and residual loss; Step S3: Using the minimum subset of the target dataset, based on the singular values ​​of the spatial Gram matrix and channel Gram matrix output by the teacher model, obtain the energy probability distribution in the spatial subspace and channel subspace, calculate the spatial complexity and channel complexity, and obtain the reconstructed basis consistency loss based on the basis consistency loss. The specific steps of step S3 are as follows: Step S3.1: Let Let the singular value of the i-th element of the spatial Gram matrix output by the teacher model be denoted as . Let j represent the singular value of the j-th element of the channel Gram matrix output by the teacher model. Normalize each singular value to obtain the energy probability distribution within the corresponding spatial subspace and channel subspace: ； ； in, express The corresponding energy probability distribution of the spatial subspace, This represents the singular value of the k-th component of the spatial Gram matrix output by the teacher model; express The corresponding energy probability distribution of the channel subspace, This represents the singular value of the k-th principal component in the channel Gram matrix output by the teacher model; K is the number of principal components. Step S3.2: Calculate the space complexity and channel complexity as follows: ； ； in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; The channel Gram matrix represents the sampled image output by the teacher model; Step S3.3: Based on basis consistency loss The reconstructed basis consistency loss is as follows: ； in, This represents the base consistency loss during reconstruction. Representation basis consistency loss Loss in spatial dimensions, Representation basis consistency loss Loss in the channel dimension; express Dynamic weight loss, express Dynamic weight loss; Dynamic weight loss , The calculation is as follows: ； ； in, Indicate space complexity, Indicates channel complexity; The spatial Gram matrix representing the sampled image output by the teacher model; The channel Gram matrix represents the sampled image output by the teacher model; Step S4: Deploy the pruning model to all downstream tasks to complete the construction and deployment of the pruning model for downstream tasks.

2. The task-oriented pruning method based on structural space consistency according to claim 1, characterized in that, In step S1, the teacher model is constructed based on the native DINOv3 model, and the pruning model is constructed based on the pruning-guided DINOv3 model. The pruning-guided DINOv3 model is based on the native DINOv3 model and is obtained by pruning with a preset sparsity.

3. The task-oriented pruning method based on structural space consistency according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S1.1: Input the sampled image into the teacher model and the noisy image into the pruning model. After passing through the encoder, the feature matrices of the sampled image and the noisy image are generated as follows: ； in, The number of lexical units, For channel dimension, Let be the feature matrix of the image, containing the feature matrix of the sampled image and the feature matrix of the noisy image, wherein the feature matrix of the sampled image is represented as... The feature matrix of a noisy image is represented as ; Step S1.2: Define the generalized Gram matrix and extract the manifold structure by performing singular value decomposition on the Gram matrix, as follows: ； in, Represents the Gram matrix. The feature matrix is ​​expanded along the target dimension. It is a singular value diagonal matrix. It is an orthogonal basis matrix; Step S1.3: Calculate the spatial Gram matrix based on the Gram matrix as follows: ； in, The spatial Gram matrix is ​​represented as follows: The spatial Gram matrix of the noisy image output by the pruning model is expressed as... The spatial Gram matrix of the sampled image output by the teacher model is represented as follows: , The feature matrix of the image, The number of lexical units; For spatial Gram matrix By applying singular value decomposition, the spatial basis matrix is ​​obtained. The spatial basis matrix of the noisy image output by the pruning model is represented as: The spatial basis matrix of the sampled image output by the teacher model is represented as: ; Step S1.3: Calculate the channel Gram matrix based on the Gram matrix as follows: ； in, The channel Gram matrix is ​​represented as follows: The channel Gram matrix of the noisy image output by the pruning model is expressed as... The channel Gram matrix of the sampled image output by the teacher model is represented as follows: , For channel dimensions; For channel Gram matrix Singular value decomposition is applied to obtain the channel basis matrix. The channel basis matrix of the noisy image output by the pruning model is represented as: The channel basis matrix of the sampled image output by the teacher model is represented as: .

4. The task-oriented pruning method based on structural space consistency according to claim 3, characterized in that, The specific steps of step S2 are as follows: Step S2.1: Calculate the spatial basis matrix of the noisy image output by the pruning model. Channel basis matrix Perform top-k truncation to obtain the basis matrix of the pruning model. ; the spatial basis matrix of the sampled images output by the teacher model Channel basis matrix Perform top-k truncation to obtain the teacher model basis matrix. ;in, The number of lexical units, The number of principal components truncated for the top-k; Step S2.2: Based on the pruning model basis matrix Teacher model basis matrix Define the affinity matrix As shown in the following formula: ； Based on affinity matrix Define the basis consistency loss As shown in the following formula: ； in, Let K be the Frobenius norm, and K be the number of principal components. Affinity matrix The element in the i-th row and j-th column; Constructing residual loss As shown in the following formula: ； in, Represents the identity matrix. The feature matrix representing a noisy image; This represents the basis matrix of the teacher model.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the task-oriented pruning method based on structural space consistency as described in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the task-oriented pruning method based on structural space consistency as described in any one of claims 1 to 4.

Images