A structured pruning method for a deep pedestrian search model

Abstract

The application belongs to the technical field of model compression, and relates to a structured pruning method for a deep pedestrian search model, comprising the following steps: S1, preparing an image data set, completing data preprocessing, dividing the data set, and constructing a to-be-pruned pedestrian search model; S2, pre-training the to-be-pruned pedestrian search model until the pedestrian search model converges; S3, initializing a pruning module, setting a target compression scale, and performing coupled layer grouping and initialization; S4, pruning the model: calculating a channel importance measure, determining a prunable network layer in combination with a convolutional layer position and a compression scale; S5, fine-tuning the pruned model, deleting network structures corresponding to channels according to a mask, and training the pruned model until convergence; and S6, inputting a query pedestrian image and a candidate image into the fine-tuned pruned model, and outputting the model as a final inference result. The application improves the performance of the compressed deep pedestrian search model and reduces precision loss.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of model compression, and particularly relates to a structured pruning method for a deep pedestrian search model. BACKGROUND

[0002] The pedestrian search model pruning technology aims to delete redundant parameters in the pedestrian search model, reduce the calculation complexity, improve the inference speed of the pedestrian search model, and reduce the precision loss caused by the reduction of the parameters of the pedestrian search model as much as possible.

[0003] In recent years, with the rapid development of deep learning, the size and parameters of deep neural networks increase dramatically, and the storage and inference costs of pedestrian search models also increase. In order to solve the above problems, researchers introduce the idea of removing redundant calculations into the compression of deep pedestrian search models. Pedestrian search is an important task in the field of computer vision, and has wide applications in intelligent monitoring, smart city, intelligent retail and the like. The core goal is to locate and determine the target pedestrian to be queried from the monitoring image. The task is composed of two sub-tasks of pedestrian detection and pedestrian re-identification, and the pedestrian search model requires a large number of parameters. At the same time, the monitoring terminal, unmanned aerial vehicle / vehicle and other computing resource limited platforms are important application scenarios of the task, and the actual business has a high requirement for the fast response of the system. Therefore, effectively compressing the deep pedestrian search model to reduce resource and time consumption is a key technology for improving the deployability of the pedestrian search model in many fields such as security, transportation and business.

[0004] Most of the existing deep pedestrian search model compression work focuses on designing a lightweight network structure or using a lightweight feature extraction network. For example, in 2019, Li et al. considered that detection is the main factor affecting the speed of the pedestrian search task, and then used a lightweight MobileNet as the backbone network of pedestrian detection. Most of these methods rely on manual design of network structure, and have low flexibility in the face of rapidly updated industrial application scenarios and different hardware devices. In contrast, the pruning method is mostly independent of the original pedestrian search model structure, and uses the floating point calculation amount to regulate the size of the compressed pedestrian search model, and has significantly higher flexibility.

[0005] Existing deep pedestrian search model pruning methods are mainly geared towards general vision tasks and can be broadly categorized into unstructured pruning and structured pruning. Unstructured pruning methods prune by resetting individual weights to zero. Because the weights removed by these methods are spatially irregular, most hardware struggles to accelerate them. Structured pruning methods, on the other hand, use filters and feature map channels as the basic units of pruning, thus enabling large-scale compression and acceleration. A representative method in structured pruning is importance-based pruning. The basic idea is to design indicators such as channel importance based on the contribution of each unit to the pedestrian search model, and select and delete low-importance units. Specifically, loss-based pruning uses the change in the loss function after pruning channels to reflect the importance of feature map channels, and approximates the importance index using the feature gradient values ​​from the network's forward propagation. Absolute value-based pruning uses the norm of the weight matrix as an importance measure, sets a threshold, and deletes all units below the threshold. These general pruning algorithms are mostly designed for computer vision tasks such as classification, detection, and segmentation, and they perform particularly well on pedestrian search models based on ResNet, Faster-RCNN, and other similar algorithms.

[0006] However, for pedestrian search tasks, only a few current works attempt to compress the aforementioned pedestrian search model. For example, some use unstructured pruning to compress the pedestrian search model for the pedestrian re-identification subtask and use distillation methods to fine-tune the compressed model, without considering the pedestrian detection subtask. Compared to general vision tasks, pedestrian search tasks have distinct task characteristics, and directly using general pruning algorithms has certain limitations, resulting in significant optimization space for the compressed pedestrian search model. First, in mainstream end-to-end pedestrian search models, the pedestrian detection and pedestrian re-identification subtasks typically share a backbone network. Since the detection subtask focuses on common pedestrian features while the re-identification subtask focuses on unique pedestrian identity features, they are semantically inconsistent. When using pruning methods based on importance metrics, existing methods do not consider these differences and fail to consistently reflect the contribution of specific channels to both subtasks, leading to the incorrect pruning of channels that are important to both subtasks. Secondly, existing methods are prone to over-pruning at high compression rates, meaning some network layers have too few remaining channels, leading to poor fine-tuning performance of the pruned pedestrian search model. Furthermore, pedestrian search tasks often use Online Instance Matching (OIM) loss, which constructs positive and negative sample pairs by dynamically maintaining proxy features to assist in model training. However, during pruning, as the number of parameters in the pedestrian search model decreases, the proxy features' ability to represent fine-grained semantics degrades, ultimately affecting the performance of the pruned pedestrian search model.

[0007] In summary, existing general-purpose pruning algorithms are not specifically designed for deep pedestrian search models, therefore, there is significant room for performance optimization. Summary of the Invention

[0008] The technical problem to be solved:

[0009] To address the shortcomings of existing pedestrian search model pruning methods, this invention aims to propose a structured pruning method for deep pedestrian search models. Addressing the issues of existing importance-based pruning methods failing to adequately reflect the importance of channels to multiple sub-tasks in pedestrian search, over-pruning of some network layers, and fine-grained semantic degradation of features, this invention considers the importance of channels to individual sub-tasks. Utilizing channel quantity constraints and discriminative semantic constraints from pre-training convergence, it iterative pruning of channels improves the performance of the compressed pedestrian search model and reduces accuracy loss.

[0010] The technical solution adopted is as follows:

[0011] This invention proposes a structured pruning method for deep pedestrian search models, characterized by the following specific steps:

[0012] S1: Prepare image dataset and construct pedestrian search model to be pruned: Select and preprocess the image dataset, divide the image dataset, and construct the pedestrian search model to be pruned based on convolutional neural network;

[0013] S2: Pre-trained pedestrian search model to be pruned: Maintain the original OIM loss function and hyperparameter settings of the pedestrian search model, iteratively update the network parameters of the pedestrian search model until the pedestrian search model converges;

[0014] S3: Initialize the pruning module: The pruning module includes a subtask-aware channel importance estimation module, a channel number balance constraint module, and a variable OIM module; set the target pruning scale, and group the coupled layers of the convolutional neural network in combination with the computation graph structure automatically constructed in the deep learning framework, grouping convolutional layers with the same feature map input into a group of coupled layers; load the network parameters of the pre-trained pedestrian search model, and replace the OIM loss function in step S2 with the variable OIM module, and initialize the hyperparameters of each pruning module;

[0015] S4: Prune the pedestrian search model to obtain a pruned pedestrian search model: During the pruning stage, the pruning process is iterative, pruning a set of coupled channels each time. The coupled channels refer to channels at the same position in the coupled layer. To ensure the continuity of activation transmission, the coupled channels are pruned until a predetermined pruning rate is reached by sharing a pruning mask on the coupled layer. Specifically, it also includes forwarding the input data, calculating and updating the channel importance estimation module of the subtask-aware channel importance estimation module, and the channel importance measurement combined with the channel number balancing constraint module jointly determines the channels that can be pruned; at the same time, the network parameters of the pedestrian search model are updated using gradients, and the hyperparameters of the pruning module are updated to obtain the pruned pedestrian search model.

[0016] S5: Fine-tuning the pruned pedestrian search model: In the fine-tuning stage, only the pruned pedestrian search model and the variable OIM module are retained. The network parameters of the pruned pedestrian search model are loaded, the hyperparameters of the unpruned pedestrian search model are adjusted, and the pruned pedestrian search model is retrained until convergence.

[0017] S6: Use the fine-tuned and pruned pedestrian search model for inference: Input the query image and candidate images into the fine-tuned and pruned pedestrian search model respectively, compare the output results of the two images after inputting them into the fine-tuned and pruned pedestrian search model, and select the pedestrian whose identity features are most similar to those of the pedestrian in the query image among all the candidate pedestrians detected in the candidate images. This is the inference result of the pedestrian search model.

[0018] Furthermore, in step S1, the selected image dataset contains real images as well as pedestrian location and identity labeling information; the image preprocessing includes proportionally adjusting the image, flipping the image, and image normalization operations; the image dataset is divided according to the standard given by the selected image dataset itself.

[0019] The pedestrian search model to be pruned is an end-to-end two-stage pedestrian search model based on a convolutional neural network, specifically the SeqNet network model; the pedestrian search model to be pruned includes the following three parts:

[0020] Backbone network: Extracts image features from the input image data;

[0021] Network transition layer: Combines or processes the image features extracted through the backbone network;

[0022] Network Header: Includes two task heads: pedestrian detection subtask and pedestrian re-identification subtask, which are used for pedestrian localization prediction and pedestrian identity re-identification.

[0023] Furthermore, in step S2, a stochastic gradient descent optimizer is used to optimize the network parameters.

[0024] Further, in step S3, the target compression scale is expressed as the ratio of the floating-point computation FLOPs of the pruned pedestrian search model to that of the original pedestrian search model; wherein, the floating-point computation FLOPs for each convolutional layer is calculated as follows: for a convolutional layer, c i h is the number of channels in the input feature map. i and w i These are the height and width of the input feature map, respectively; meanwhile, c k h is the number of channels in the convolution kernel. k and w k These are the height and width of the convolution kernel, respectively, n k c is the number of convolution kernels; o h is the number of channels in the output feature map. o and w o These represent the height and width of the output feature map, respectively. When the input batch of image data is N, the floating-point computation cost (FLOPs) generated by this convolutional layer can be approximated by the following formula:

[0025] FLOPs = N × c i ×h k ×w k ×h o ×w o ×c o .

[0026] Furthermore, in step S4, the pruning stage also includes deleting input feature map channels; specifically, for a certain convolutional layer, the input feature map is A∈R N×C×H×W Where N represents the size of the input batch of image data, C is the number of feature map channels, and H and W are the height and width of the feature map, respectively; m∈R C R represents the mask for the input channels of the convolution. c Representing a C-dimensional vector space, initially all components of the mask m are set to 1. When the j-th channel of this convolutional layer is pruned, the mask m of the j-th channel... j Set to 0; after pruning, the input feature matrix is ​​transformed into... Here, ⊙ represents the dot product in the second channel dimension, meaning that in the input feature map, all elements of the pruned channels become 0.

[0027] Further, in step S3, the sub-task-aware channel importance estimation module: the sub-task-aware channel importance estimation module calculates the channel importance value to the sub-task based on the loss criterion and performs weighted fusion; for the nth input training image data, n = 1, 2, 3....N, during the forward propagation of the convolutional neural network, the sub-task-aware channel importance estimation module records the input feature map matrix A of each network layer. n For the nth training image data input, during the backpropagation of the convolutional neural network, the subtask-aware channel importance estimation module records the subtask loss function with respect to the input feature map matrix A of each network layer. n gradient According to the transformation formula Calculate the gradient of the subtask loss function with respect to the mask m. L n The pedestrian search model corresponds to the subtask loss function of the input nth training image data;

[0028] s j The change in the subtask loss function of the pedestrian search model before and after pruning channel j can be approximated by Taylor's formula as follows:

[0029]

[0030] Where L is the subtask loss function of the pedestrian search model, and T represents the matrix transpose operation. Represents the gradient; e j ∈R C It is in one-hot vector form, meaning that its j-th component has a value of 1, and all other components have a value of 0; denoted by g. The components corresponding to each channel j m j Let represent the component corresponding to channel j in the mask vector, i.e., the j-th component. Assuming the pedestrian search model is in a convergent state during pruning, then we have: H is the Hessian matrix of the loss function of the pedestrian search model with respect to the mask m, which can be obtained as follows: According to the Fisher information formula, for channel j, the (j,j) component H of the Hessian matrix is... jj It can be approximated as:

[0031]

[0032] Where E represents the mathematical expectation, x represents the input sample, y represents the category obtained by the model inference, and P(y|x) represents the probability that sample x is classified as category y;

[0033] The change in the subtask loss function of the pedestrian search model can be approximated as:

[0034]

[0035] Combining the coupling layer shared pruning mask mechanism, the change s of the subtask loss function of the pedestrian search model is used with respect to the gradients of all convolutional layers within the coupling layer group. j Further expansion yields:

[0036]

[0037] Where CG represents the set of all convolutional layers within the coupling group. This represents the copy of the pruning mask of the j-th channel on the convolutional layer l.

[0038] Further, in step S3, the channel number balancing constraint module: the channel number balancing constraint module dynamically sets the minimum remaining channel number threshold based on the position of the convolutional layer in the convolutional neural network structure and the pruning scale to alleviate the over-pruning phenomenon; in the initialization phase, the network layers are divided into backbone network layers and other network layers; as long as the coupling layer group contains a layer in the backbone network, all layers in the coupling layer group are divided into the backbone network part; the ratio of the remaining channel number of each layer to the original number of channels of the pedestrian search model is used as the channel number constraint quantity, and the thresholds of the backbone network part and the other part are represented by TB and TO, respectively, as hyperparameters of the channel number balancing constraint module; when the remaining channel number is lower than this threshold, pruning of the corresponding convolutional layer is stopped; PR∈[0,1] represents the current compression scale of the pedestrian search model; for other network layers, TO decreases dynamically with PR; for the backbone network layer, in order to protect the representational ability of the network in the early stage of pruning, the initial value of TB is set to a large value and remains unchanged. When all layers in the network reach the threshold and cannot be pruned, the threshold of TB is gradually reduced; the above process is specifically expressed as:

[0039]

[0040] In step S3, the variable OIM module: The variable OIM module is used to replace the original OIM loss function in the pedestrian search model, and optimizes the feature distribution of the compressed pedestrian search model using discriminative identity features from the pre-training stage, so as to alleviate the feature degradation problem of the compressed pedestrian search model; specifically, after pre-training, while storing the pedestrian search model parameter file, it saves the proxy features v of each identity in the feature library of the OIM loss function of the pedestrian search model. pt During the pruning or fine-tuning stage of the pedestrian search model, the proxy features are loaded and pre-stored.

[0041] During the pruning or fine-tuning phase, the proxy feature v is... pt The proxy features are introduced into the OIM loss function of the pedestrian search model at different proportions and then fixed. The fixed proxy features are defined as v. fix LB represents the number of labeled identity features, Q represents the number of unlabeled identity features, and v lb u represents the proxy feature of the lb-th identity in the set of labeled identities. q Let f represent the q-th identity feature in the unlabeled queue. Based on this, for the feature f extracted by the network, when the agent features are not fixed, the probability p that the feature f extracted by the network is identified as identity a is... a It can be represented as:

[0042]

[0043] When the agent characteristics are fixed, the probability p that the feature f extracted by the network is identified as identity a is... a It can be represented as:

[0044]

[0045] Where T represents the matrix transpose operation, and τ is a hyperparameter defined by the pedestrian search model described above;

[0046] Based on the above, the OIM loss function of the pedestrian search model can be expressed as L = E x [logp a This is used for channel importance calculation in the pedestrian search model network parameter update and importance estimation module.

[0047] Furthermore, in step S4, the entire pruning stage includes the following steps:

[0048] S41. The input data is propagated forward three times in the convolutional neural network: after the first forward propagation, the input feature maps of each convolutional layer are saved; after the second forward propagation, the loss function of the pedestrian detection subtask is calculated and gradient backpropagation is performed, while the change in the loss function of the detection subtask for each channel is calculated; after the third forward propagation, the loss function of the pedestrian re-identification subtask is calculated and gradient backpropagation is performed, while the change in the loss function of the re-identification subtask for each channel is calculated.

[0049] S42. Calculate the channel importance metric S for each channel of the subtask using the subtask-aware channel importance estimation module. j After pruning based on the importance metric, in order to maintain the convergence assumption of the pedestrian search model, the network parameters of the module are updated using the gradients accumulated from the last two forward passes.

[0050] Specifically, based on the calculation of the importance of sub-tasks, the importance metric S of each channel is obtained.j The steps are as follows:

[0051] First, backpropagation is performed using the loss functions of the pedestrian detection and pedestrian re-identification subtasks respectively, and the change s of the loss function of all channels of the pedestrian search model with respect to the pedestrian detection subtask is calculated. det The change s of the loss function of all channels of the pedestrian search model for the pedestrian re-identification subtask reid For channel ch, the computational cost of the convolution directly related to it is F. ch =h k ×w k ×h o ×w o ×c o For channels in a subtask that do not affect another subtask, the gradient of the subtask loss function is 0, and it is corrected using the maximum change in the subtask loss function per unit of computation; the change s of the channel loss function for the pedestrian detection subtask in the pedestrian re-identification subtask is used as the basis for this correction. det Taking (ch) as an example, the specific formula is as follows:

[0052]

[0053] Where CH represents the set containing all channels in the network model, F CH This refers to the amount of convolution computation related to the channel CH;

[0054] Then, the changes in the loss function of the two sub-tasks are normalized to the same scale. To eliminate the influence of long-tailed numerical distributions, the normalization standard is the top α% values ​​in descending order of each value. Specifically, the changes in the loss function s of the pedestrian re-identification sub-task are normalized. reid Scaling is performed, and the change s′ after scaling is calculated. reid The formula is as follows:

[0055]

[0056] Based on this, α and β are hyperparameters of the channel importance estimation module for subtask perception. The hyperparameter β reflects the proportion of subtasks, and the changes in the subtask loss functions are re-weighted and fused. The fused changes are used as the channel importance index S for the entire pedestrian search model. fuse The calculation formula is as follows:

[0057]

[0058] Finally, the memory reduction resulting from channel pruning was used to evaluate the importance metric S. fuse Normalization is performed to obtain the importance metric S for each channel. j The specific formula is as follows:

[0059]

[0060] S43. The pruning operation is performed once for every d sets of image data, where d is a preset value. For the first d-1 times, only the importance value is accumulated. After the dth accumulation, the channel number balancing constraint module is used to select the convolutional layers that can still be pruned, and the channel with the lowest importance metric is selected for pruning. After each pruning, the accumulated importance metric is cleared, the floating-point computation of the remaining pedestrian search model is calculated, and it is determined whether to terminate the pruning. The hyperparameters TB,TO of the channel number balancing constraint module are updated.

[0061] Furthermore, in the pruning stage of step S4, the pruning operation is completed simply by setting the corresponding channel position of the convolutional layer mask to zero; while in the fine-tuning stage of step S5, the network structure corresponding to the channel is deleted according to the mask.

[0062] Furthermore, in the pruning stage of step S4, the proxy features are dynamically updated in the early stage of the pruning stage, and pre-trained features are introduced in the later stage of the pruning stage. Specifically, in the early stage of the pruning stage, PR∈[r,1], where r is a pre-set value. In order to satisfy the convergence assumption of the pedestrian search model as much as possible, the same dynamic update strategy of proxy features as the traditional OIM loss function is still adopted, and the rules are consistent with the OIM loss function learning strategy adopted by the original pedestrian search model. Assuming that when PR=r, the proxy features are changed from the initial feature v pt Updated to v cur In the stage of compressing the pedestrian search model size PR∈[0,r], a fixed proxy feature is used, and the proportion of pre-trained features is gradually increased until it is completely updated to v. pt The specific update formula is as follows:

[0063] v fix =γ·v pt +(1-γ)·v cur

[0064] Where γ is a hyperparameter of the variable OIM module, which changes dynamically to control the proportion of pre-trained features in the surrogate features;

[0065] In the fine-tuning phase of step S5, the rules for the proxy features are the opposite of those in the pruning phase. Initially, the proxy features are kept fixed, and then dynamically updated. Specifically, in the initial stage of the fine-tuning phase, the v value, which was fixed at the end of the pruning phase, is maintained. pt As surrogate features, to maintain the discriminativeness of surrogate features; in the later stage of fine-tuning, the update strategy of the traditional OIM loss function module is adopted to dynamically update the surrogate features, so that the pedestrian search model after pruning can be fully optimized.

[0066] The beneficial effects of this invention compared to the prior art are as follows:

[0067] 1. This invention proposes a pruning method for deep pedestrian search models, and puts forward a pruning algorithm for the pedestrian search model based on subtask perception and channel number balance constraints. This makes up for the shortcomings of general pruning algorithms that do not fully consider the multi-task characteristics of pedestrian search, and improves the inference accuracy of the compressed pedestrian search model.

[0068] 2. This invention proposes a channel importance estimation module for subtask awareness. This module estimates the importance index of each channel to the subtask based on a loss criterion and performs weighted fusion to retain channels that are important to multiple subtasks.

[0069] 3. This invention proposes a channel number balance constraint module. Based on the network layer position of the channel and the pruning scale, a minimum channel number constraint is dynamically set for each layer to maintain network structure balance, thereby preserving the representation performance of the pruned pedestrian search model.

[0070] 4. This invention proposes a variable online instance matching (OIM) module. In the proxy features during the pruning and fine-tuning stages, this module dynamically adjusts the weighting of identity features in the pre-trained pedestrian search model, reducing the negative impact of fine-grained semantic discriminative degradation on the pruning and fine-tuning of the pedestrian search model. Attached Figure Description

[0071] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0072] Figure 1 This is a flowchart illustrating the structured pruning method for a deep pedestrian search model according to the present invention.

[0073] Figure 2 This is a flowchart illustrating the pruning or fine-tuning stages of each pruning module in this invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] The specific steps of an embodiment of the pruning method for the pedestrian search model of the present invention are as follows: Figure 1 As shown:

[0076] S1: Dataset preparation and construction of the pedestrian search model to be pruned. This includes selecting the image dataset, preprocessing the images, and dividing the dataset into training and testing sets.

[0077] The selected image dataset contains real images along with pedestrian location and identification annotations. To verify the performance and robustness of the pedestrian search model, this embodiment selects commonly used benchmark datasets for pedestrian search, namely the CUHK-SYSU dataset and the PRW dataset. The image and annotation organization of the datasets is consistent with the open-source object detection framework MMDetection.

[0078] The images are preprocessed. On the CUHK-SYSU and PRW datasets, this embodiment proportionally resizes the images to [900, 1500], randomly flips the images horizontally with a 50% probability, and finally performs image normalization.

[0079] The datasets were split according to the criteria given for each dataset. For the CUHK-SYSU dataset, 11,206 images of 5,532 pedestrians were selected for training, 2,900 images of 2,900 pedestrians were selected as query images, and 6,978 images were used as candidate test images. For the PRW dataset, 5,704 images of 482 pedestrians were selected for training, 2,057 pedestrians were selected as query images, and 6,112 images were used as candidate test images.

[0080] A pedestrian search model to be pruned is constructed. In this embodiment, an end-to-end two-stage SeqNet pedestrian search model is selected for pruning and compression. The pedestrian search model first extracts image features using the first four layers of ResNet50, then obtains candidate boxes through the RPN network, and performs cropping and pooling on the feature map according to the candidate boxes. To prioritize improving pedestrian detection accuracy, the pedestrian search model inputs the cropped feature map into the Faster-RCNN detection head and performs cropping and pooling again. Finally, the fifth layer of ResNet50 is used to extract feature vectors from the detection boxes for identity re-identification and prediction of pedestrian localization boxes.

[0081] S2: Pre-train the pedestrian search model to be pruned. Maintain the original pedestrian search model training settings, using a ResNet50 network pre-trained on ImageNet as the backbone network. The batch size is set to 4, the initial learning rate to 0.0024, and the total training time to 20. The learning rate uses a linear warmup strategy in the first training round and decreases by 10% in the 16th training round. A momentum of 0.9 and a weight decay of 5 × 10⁻⁶ are used. -4 The stochastic gradient descent optimizer is used to optimize the network parameters, and the network parameters are iteratively optimized until the pedestrian search model converges.

[0082] S3: Initialize the pruning module, completing the grouping of coupling layers and the initialization settings of hyperparameters for each module. In this embodiment, the hyperparameters α and β in the subtask-aware channel importance estimation module are set to 0.1 and 0.5, respectively. In the channel number balancing constraint module, the hyperparameter TB is initialized to 0.9; in the variable online instance matching module, i.e., the variable OIM module, the hyperparameter r is set to 0.8, meaning that when the pedestrian search model size is compressed to 80%, a fixed proxy feature is used, and γ increases from 0.2 to 1, with γ increasing by 0.1 for every 0.05 decrease in PR.

[0083] S4: Prune the pedestrian search model. Figure 2 The pruning process shown in this embodiment aims to compress the pedestrian search model to 25% and 10% of its original size, with a batch size of 1 and a fixed learning rate of 0.0001. A momentum of 0.9 and a weight decay of 5 × 10⁻⁶ are used. -4 The parameters are optimized using a stochastic gradient descent method. Furthermore, in this embodiment, pruning is performed every 10 importance metrics accumulated; this process is iterative until the computational cost of the pruned pedestrian search model reaches a threshold.

[0084] S5: The pedestrian search model after fine-tuning the pruning. According to... Figure 2 As shown, the process also includes fine-tuning. In this embodiment, the batch size is set to 4, the initial learning rate is 0.00024, and the training run is conducted for 20 rounds. The learning rate uses a linear warmup strategy in the first training round and decreases by 10% in the 16th training round. A momentum of 0.9 and a weight decay of 5 × 10⁻⁶ are used. -4 The network parameters were optimized using a stochastic gradient descent optimizer. In the variable OIM module, for the pedestrian search model compressed to 25% FLOPs, fixed surrogate features were used for the first 15 training epochs. For the pedestrian search model compressed to 10% FLOPs, fixed surrogate features were used for the first 5 training epochs.

[0085] S6: Inference is performed using the finely pruned pedestrian search model. Candidate images are input into the finely pruned pedestrian search model, which outputs the detected candidate locations and identity features. The image of the pedestrian to be queried is input into the finely pruned pedestrian search model, which outputs the identity features of the person to be queried. The output identity features of the person to be queried are compared with those of the candidate pedestrians to obtain the inference result. On the CUHK-SYSU dataset, the pedestrian search model compressed to 25% FLOPs achieves an average precision (mAP) of 91.90%, with Top-1 and Top-5 accuracies of 92.66% and 97.45%, respectively. The pedestrian search model compressed to 10% FLOPs achieves an average precision (mAP) of 90.84%, with Top-1 and Top-5 accuracies of 91.24% and 96.90%, respectively. The technique proposed in this invention achieves higher accuracy at the same compression scale, effectively reducing the number of parameters in the pedestrian search model and improving inference speed with less accuracy loss.

[0086] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A structured pruning method for deep pedestrian search models, characterized in that, The specific steps include: S1: Prepare image dataset and construct pedestrian search model to be pruned: Select and preprocess the image dataset, divide the image dataset, and construct the pedestrian search model to be pruned based on convolutional neural network; S2: Pre-trained pedestrian search model to be pruned: Maintain the original OIM loss function and hyperparameter settings of the pedestrian search model, iteratively update the network parameters of the pedestrian search model until the pedestrian search model converges; S3: Initialize the pruning module: The pruning module includes a subtask-aware channel importance estimation module, a channel number balance constraint module, and a variable OIM module; set the target pruning scale, and group the coupled layers of the convolutional neural network in combination with the computation graph structure automatically constructed in the deep learning framework, grouping convolutional layers with the same feature map input into a group of coupled layers; load the network parameters of the pre-trained pedestrian search model, and replace the OIM loss function in step S2 with the variable OIM module, and initialize the hyperparameters of each pruning module; S4: Prune the pedestrian search model to obtain a pruned pedestrian search model: During the pruning stage, the pruning process is iterative, pruning a set of coupled channels each time. The coupled channels refer to channels at the same position in the coupled layer. To ensure the continuity of activation transmission, the coupled channels are pruned until a predetermined pruning rate is reached by sharing a pruning mask on the coupled layer. Specifically, it also includes forwarding the input data, calculating and updating the channel importance estimation module of the subtask-aware channel importance estimation module, and the channel importance measurement combined with the channel number balancing constraint module jointly determines the channels that can be pruned; at the same time, the network parameters of the pedestrian search model are updated using gradients, and the hyperparameters of the pruning module are updated to obtain the pruned pedestrian search model. S5: Fine-tuning the pruned pedestrian search model: In the fine-tuning stage, only the pruned pedestrian search model and the variable OIM module are retained. The network parameters of the pruned pedestrian search model are loaded, the hyperparameters of the unpruned pedestrian search model are adjusted, and the pruned pedestrian search model is retrained until convergence. S6: Use the fine-tuned and pruned pedestrian search model for inference: Input the query image and candidate images into the fine-tuned and pruned pedestrian search model respectively, compare the output results of the two images after inputting them into the fine-tuned and pruned pedestrian search model, and select the pedestrian whose identity features are most similar to those of the pedestrian in the query image among all the candidate pedestrians detected in the candidate images. This is the inference result of the pedestrian search model.

2. The structured pruning method for a deep pedestrian search model according to claim 1, characterized in that, In step S1, the selected image dataset contains real images as well as pedestrian location and identity labeling information; the image preprocessing includes proportionally adjusting the image, flipping the image, and image normalization; the image dataset is divided according to the standard given by the selected image dataset itself. The pedestrian search model to be pruned is an end-to-end two-stage pedestrian search model based on a convolutional neural network, specifically the SeqNet network model; the pedestrian search model to be pruned includes the following three parts: Backbone network: Extracts image features from the input image data; Network transition layer: Combines or processes the image features extracted through the backbone network; Network Header: Includes two task heads: pedestrian detection subtask and pedestrian re-identification subtask, which are used for pedestrian localization prediction and pedestrian identity re-identification.

3. The structured pruning method for a deep pedestrian search model according to claim 2, characterized in that, In step S2, the network parameters are optimized using a stochastic gradient descent optimizer.

4. The structured pruning method for a deep pedestrian search model according to claim 3, characterized in that, In step S3, the target compression scale is expressed as the ratio of the floating-point computation FLOPs of the pruned pedestrian search model to that of the original pedestrian search model; wherein, the floating-point computation FLOPs for each convolutional layer is calculated as follows: for a convolutional layer, c i h is the number of channels in the input feature map. i and w i These are the height and width of the input feature map, respectively; meanwhile, c k h is the number of channels in the convolution kernel. k and w k These are the height and width of the convolution kernel, respectively, n k c is the number of convolution kernels; o h is the number of channels in the output feature map. o and w o These represent the height and width of the output feature map, respectively. When the input batch image data size is N, the floating-point computation cost (FLOPs) generated by this convolutional layer can be approximated by the following formula: 。 5. The structured pruning method for a deep pedestrian search model according to claim 4, characterized in that, In step S4, the pruning stage also includes deleting channels of the input feature map; specifically, for a certain convolutional layer, the input feature map is... ,in, Indicates the size of the input batch of image data. For the number of feature map channels, These represent the height and width of the feature map, respectively. The mask representing the input channel of the convolution. Representing a C-dimensional vector space, initially all components of the mask m are set to 1. When the j-th channel of this convolutional layer is pruned, the mask of the j-th channel... Set to 0; after pruning, the input feature matrix is ​​transformed into... , here This indicates a dot product in the second channel dimension, meaning that in the input feature map, all elements of the pruned channels become 0.

6. The structured pruning method for a deep pedestrian search model according to claim 5, characterized in that, In step S3, the sub-task-aware channel importance estimation module: This module calculates the channel importance value to the sub-task based on the loss criterion and performs weighted fusion; for the nth input training image data, n=1,2,3....N, during the forward propagation of the convolutional neural network, the sub-task-aware channel importance estimation module records the input feature map matrix of each network layer. For the nth training image data input, during the backpropagation of the convolutional neural network, the subtask-aware channel importance estimation module records the subtask loss function with respect to the input feature map matrix of each network layer. gradient According to the transformation formula Calculate the gradient of the subtask loss function with respect to the mask m. , The pedestrian search model corresponds to the subtask loss function of the input nth training image data; The change in the subtask loss function of the pedestrian search model before and after pruning channel j can be approximated by Taylor's formula as follows: ， Where L is the subtask loss function of the pedestrian search model, and T represents the matrix transpose operation. Represents the gradient; It is in one-hot vector form, meaning that its j-th component has a value of 1, and all other components have a value of 0; The components corresponding to each channel j , Let represent the component corresponding to channel j in the mask vector, i.e., the j-th component. Assuming the pedestrian search model is in a convergent state during pruning, then we have: H is the Hessian matrix of the loss function of the pedestrian search model with respect to the mask m, and we get... According to the Fisher information formula, for channel j, the Hessian matrix is... Quantity Approximately expressed as: Where E represents the expected value, x represents the input sample, and y represents the category obtained by the model inference. This represents the probability that sample x is classified into category y; The change in the subtask loss function of the pedestrian search model can be approximately expressed as: By combining the coupling layer shared pruning mask mechanism, the changes in the subtask loss function of the pedestrian search model are used with respect to the gradients of all convolutional layers within the coupling layer group. Expanding upon this, we get: Where CG represents the set of all convolutional layers within the coupling group. This represents the copy of the pruning mask of the j-th channel on the convolutional layer l.

7. The structured pruning method for a deep pedestrian search model according to claim 6, characterized in that, In step S3, the channel number balancing constraint module dynamically sets the minimum remaining channel number threshold based on the position of the convolutional layer in the convolutional neural network structure and the pruning scale, in order to alleviate the over-pruning phenomenon. During the initialization phase, the network layers are divided into backbone network layers and other network layers. If a coupling layer group contains a layer from the backbone network, then all layers in the coupling layer group are classified as the backbone network part. The ratio of the remaining number of channels in each layer to the original number of channels in the pedestrian search model is used as the channel number constraint. The thresholds for the backbone network part and the other part are represented by TB and TO, respectively, and are used as hyperparameters of the channel number balancing constraint module. When the remaining number of channels is lower than this threshold, pruning of the corresponding convolutional layer is stopped. This indicates the current compression scale of the pedestrian search model; for other network layers, TO decreases dynamically with PR; for the backbone network layer, to protect the network's representational capability in the early stages of pruning, the initial value of TB is set to a large value and remains unchanged. When all layers in the network reach the threshold and pruning is no longer possible, the threshold of TB is gradually reduced; the above process is specifically represented as follows: In step S3, the variable OIM module is used to replace the original OIM loss function in the pedestrian search model. It uses the discriminative identity features from the pre-training stage to optimize the feature distribution of the compressed pedestrian search model, thereby alleviating the feature degradation problem of the compressed pedestrian search model. Specifically, after pre-training, while storing the pedestrian search model parameter file, the proxy features of each identity in the OIM loss function feature library of the pedestrian search model are also saved. During the pruning or fine-tuning stage of the pedestrian search model, the proxy features are loaded and pre-stored. During the pruning or fine-tuning phase, the proxy features are... The proxy features are introduced into the OIM loss function of the pedestrian search model at different proportions and then fixed. The fixed proxy features are defined as follows: LB represents the number of labeled identity features, and Q represents the number of unlabeled identity features. This represents the proxy features of the lb-th identity in the set of labeled identities. Let f represent the q-th identity feature in the unlabeled queue. Based on this, for the feature f extracted by the network, when the agent feature is not fixed, the probability that the feature f extracted by the network is identified as identity a is... Represented as: When the agent characteristics are fixed, the probability that the feature f extracted by the network is identified as identity a is... Represented as: Where T represents the matrix transpose operation. These are hyperparameters, defined by the pedestrian search model described earlier; Based on the above, the OIM loss function of the pedestrian search model is expressed as follows: Channel importance calculation is used in the pedestrian search model network parameter update and importance estimation module.

8. The structured pruning method for a deep pedestrian search model according to claim 7, characterized in that, In the pruning stage of step S4, the pruning operation is completed simply by setting the corresponding channel position of the convolutional layer mask to zero; while in the fine-tuning stage of step S5, the network structure corresponding to the channel is deleted according to the mask.

Images