A long-time pedestrian re-identification method based on double-path cooperation and key frame guided reconstruction

Abstract

The application discloses a long-time pedestrian re-identification method based on double-path cooperation and key frame guidance reconstruction. The method first collects a to-be-identified pedestrian video and extracts a video feature sequence; then introduces spatial and time position coding to the video feature sequence; then captures local fine-grained dynamic features through local dynamic feature capture path and models long-range time sequence correlation through cross-frame global feature modeling path; then realizes double-path feature complementation through bidirectional gate interaction; further screens out key frames, realizes feature reconstruction through a full-frame attention propagation mechanism; finally, fuses double-path fusion features, key frame guidance reconstruction features and refined features to generate pedestrian identity features; processes the pedestrian identity features to obtain standardized feature vectors, compares the standardized feature vectors with pedestrian features in an image library, and returns a matching list. The application fully utilizes video time sequence information and effectively solves the problem of insufficient robustness caused by appearance changes in long-time pedestrian re-identification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction. Background Technology

[0002] With the increasing urgency of smart city construction and public safety assurance for comprehensive, all-weather intelligent sensing, pedestrian re-identification, a key technology for continuous pedestrian tracking across cameras, has encountered significant bottlenecks in its practical application, making truly reliable large-scale deployment difficult. The core root of this predicament lies in the fundamental contradiction between the basic assumption of "short-term appearance consistency" that is widely relied upon by existing mainstream technology systems and the complex requirements of long time spans and appearance changes that are common in real-world application scenarios.

[0003] To address these challenges, the research field has proposed the concept of long-term pedestrian re-identification. This technology aims to achieve continuous and accurate identification and association of pedestrians across cameras and time spans. Its core task is to overcome changes in appearance and obtain stable identity clues. Typical applications include criminal investigation tracking, counter-terrorism deployment, and the search for missing persons.

[0004] Existing research mainly falls into two categories: image-based and video-based long-term person re-identification methods. Image-based methods typically rely on extracting identity features from single-frame static images. Their inherent limitation is that the information in a single frame is limited, making it difficult to capture stable and highly discriminative identity representations.

[0005] To overcome the limitations of single-frame information and compensate for the shortcomings of static images in terms of temporal correlation and information richness, video-based methods have emerged. These methods can capture unique dynamic features of pedestrians and enhance feature reliability by filtering noise through information overlay. In existing technologies, a common approach is to extract features from each frame of the video individually and then concatenate or perform simple pooling to obtain a video-level representation. However, this method only accumulates static information from each frame, lacking effective modeling of temporal features and failing to filter out key information, making it susceptible to redundant or noisy frames. Furthermore, most methods only model local consecutive frames; while they can capture some short-term dynamic features, they lack full utilization of global temporal correlations across frames.

[0006] In summary, existing long-term pedestrian re-identification technologies generally suffer from the following technical bottlenecks: on the one hand, the models rely excessively on local features of single frames or local continuous frames, and lack effective modeling of long-term temporal correlations; on the other hand, redundant and noise information in video sequences lacks effective screening and control, making it difficult to maintain robust identity representation. Summary of the Invention

[0007] To address the problems existing in existing methods, the present invention aims to provide a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction, comprising the following steps:

[0008] 1. A long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction, characterized by comprising the following steps:

[0009] S1. Video Feature Extraction: Acquire pedestrian videos to be identified and process them into video sequences. For the acquired pedestrian video sequences, extract video sequence features through the backbone network to obtain video feature sequences.

[0010] S2. Spatiotemporal location coding enhancement: Spatial location coding and temporal location coding are introduced into the video feature sequence respectively, and the video feature sequence is superimposed with the two location codes to obtain an enhanced feature sequence;

[0011] Furthermore, the spatial location encoding is a learnable parameter vector; the temporal location encoding is also a learnable parameter vector; when the time step of the input video feature sequence is inconsistent with the preset length of the temporal location encoding, if the input time step is small, the corresponding part of the temporal location encoding is truncated; if the input time step is large, the temporal location encoding is matched with the input time step by a cyclic expansion method.

[0012] S3. Dual-path parallel feature extraction: The enhanced feature sequence is processed by depthwise separable convolution through the local dynamic feature capture path, and then output by nonlinear activation function and residual connection operation; the enhanced feature sequence is dilated temporally sampled through the cross-frame global feature modeling path, and then processed by multi-head attention mechanism to output cross-frame global features.

[0013] Furthermore, the depthwise separable convolution used in the local dynamic feature capture path includes temporal depthwise convolution and spatial depthwise convolution, wherein the number of groups for both types of convolution is consistent with the number of channels of the input feature; the convolution window of the temporal depthwise convolution covers features of multiple consecutive frames, and the convolution window of the spatial depthwise convolution covers the spatial region of features of a single frame; after the depthwise separable convolution processing, point convolution, nonlinear activation, point convolution and batch normalization processing are performed sequentially; the residual connection is implemented by adding the enhanced feature sequence to the obtained local dynamic features element by element.

[0014] Furthermore, the multi-head attention mechanism generates query vectors, key vectors, and value vectors through three independent linear projections; the query vectors, key vectors, and value vectors are divided into dimensions according to the number of attention heads; the dilation time sampling uses a fixed interval to sample the key vectors and value vectors; within each attention head, the multi-head attention mechanism calculates attention weights using the scaled dot product of the query vector and the key vector, and performs normalization processing using the Softmax function; the obtained attention weights are used to perform a weighted summation of the value vectors, the attention outputs of each head are concatenated, and after restoring the spatial dimension through broadcast expansion, the cross-frame global features are obtained through linear projection layer processing.

[0015] S4. Cross-path feature complementarity enhancement: The local dynamic features and cross-frame global features are processed by channel projection, and the corresponding bidirectional gating weights are calculated. The bidirectional gating weights and the projected features are weighted and fused to achieve mutual enhancement of the two types of features and output the enhanced dual-path features.

[0016] Furthermore, the channel projection is achieved through point convolution, which maintains the time, height, and width dimensions of the features unchanged. The calculation process of the bidirectional gating weights includes: performing average pooling on the local dynamic features and cross-frame global features respectively, and inputting the pooling results into a multilayer perceptron. The multilayer perceptron includes a dimension reduction convolutional layer, a nonlinear activation layer, and a dimension increase convolutional layer connected in sequence. Finally, the gating weights are generated through the Sigmoid function, and the values ​​of the gating weights are in the range of [0,1]. The enhanced dual-path features are achieved by adding the element-wise products of the local dynamic features or cross-frame global features, the corresponding gating weights, and the corresponding projection features.

[0017] S5. Adaptive Feature Fusion: Based on the enhanced feature sequence, calculate the motion difference index, feature energy index, and occlusion index, input the index into the gating module to generate fusion weights; use the fusion weights to perform weighted fusion of the enhanced dual-path features to obtain dual-path fused features;

[0018] Furthermore, the motion difference index is obtained by calculating the squared mean of the feature differences between adjacent time steps in the enhanced feature sequence; the feature energy index is obtained by calculating the channel and spatial mean of the single-frame features in the enhanced feature sequence; the occlusion index is obtained by calculating the absolute difference between the single-frame feature energy index and the global feature energy index of the enhanced feature sequence; the gating module first concatenates the motion difference index, feature energy index, and occlusion index, and after processing by a feature transformation network containing convolutional layers and nonlinear activation layers, it uses the Softmax function to generate fusion weights, and the sum of the fusion weights of the two types of features is 1; the dual-path fusion feature is achieved by adding the enhanced dual-path features and the obtained fusion weights after element-wise multiplication.

[0019] S6. Keyframe-guided reconstruction: Calculate the comprehensive score vector for each frame based on the dual-path fusion features, and select keyframes according to the score results; reconstruct the enhanced feature sequence through a full-frame attention propagation mechanism, and obtain keyframe-guided reconstruction features using a hybrid gating fusion method;

[0020] Furthermore, the comprehensive scoring vector is obtained by weighted summation of the standardized base scoring vector, motion difference index, feature energy index, and occlusion index; the base scoring vector is output by the scoring network after processing the dual-path fusion features; the number of keyframes is adaptively determined according to the total number of video frames; the full-frame attention propagation mechanism first projects the dual-path fusion features into query vector, key vector, and value vector through point convolution, extracts the key vector subset and value vector subset corresponding to the keyframe, and performs spatial pooling on the query vector, key vector subset, and value vector subset; calculates the similarity matrix between the query vector and the key vector subset, superimposes the keyframe bias and adjusts the scale, and obtains the attention weights through Softmax normalization; finally, the attention weights are used to weighted summation of the keyframe value vector subsets to obtain the full-frame reconstruction features; the hybrid gating weights are generated by a gating function through a gating matrix formed by concatenating the feature energy index, motion difference index, and occlusion index, and the gating function includes a convolutional layer and a sigmoid activation function.

[0021] S7. Multi-feature weighted fusion: The enhanced feature sequence is processed by a multilayer perceptron to obtain refined features; the dual-path fusion features, keyframe-guided reconstruction features and refined features are weighted and fused, and after convolution processing and activation function operation, pedestrian identity features are output;

[0022] Furthermore, the multilayer perceptron consists of multiple point convolutional layers, with nonlinear activation layers and Dropout layers sequentially arranged between adjacent convolutional layers; the weighted fusion adopts learnable fusion weight parameters; after the fused features are subjected to dimensionality adjustment, batch normalization, and activation function operation by convolutional layers, they are then fused with the enhanced feature sequence through residual connection to output pedestrian identity features.

[0023] S8. Feature Matching and Result Return: The pedestrian identity features are input into the backbone network for further processing to obtain a standardized feature vector; the standardized feature vector is compared with the pedestrian features in the image database, and a matching list is generated and returned based on the comparison results.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] First, this invention strengthens the spatiotemporal correlation of features by introducing spatial location coding and temporal location coding. It combines a dual-path structure consisting of a local dynamic feature capture path and a cross-frame global feature modeling path to capture fine-grained temporal changes and long-range temporal correlations, thereby effectively improving the integrity and stability of identity representation.

[0026] Second, this invention calculates motion difference index, feature energy index and occlusion index based on enhanced feature sequence, and dynamically generates fusion weights through gating module to effectively suppress redundancy and noise interference; at the same time, it selects key frames based on the comprehensive score of basic score vector, motion difference index, feature energy index and occlusion index, and realizes feature reconstruction by combining full frame attention propagation, which significantly enhances the discriminativeness and robustness of features.

[0027] Third, in the output stage, the present invention integrates the dual-path fusion features, keyframe-guided reconstruction features and machine-refined features with weights, and then optimizes the dimensions, batch normalization and residual connections through convolutional layers to generate an identity representation with strong discriminative ability.

[0028] Fourth, the present invention was experimentally verified on two publicly available video-based long-term person re-identification datasets, VCCR and CCVID. The experimental results, under the evaluation metrics of mAP and Rank-1, fully demonstrate the effectiveness and superiority of the method of the present invention. Attached Figure Description

[0029] The following are accompanying drawings provided by the applicant to more intuitively and clearly illustrate the technical solution and related structures of the present invention, which helps those skilled in the art to understand the technical features and overall technical solution of the invention:

[0030] Figure 1 A flowchart illustrating the steps of a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention;

[0031] Figure 2 The overall framework diagram of a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention is shown below.

[0032] Figure 3 A schematic diagram of the local dynamic feature capture path structure for a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention;

[0033] Figure 4 This is a schematic diagram of the cross-frame global feature modeling path structure for a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0035] like Figure 1 The diagram shows a flowchart of a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention, including:

[0036] S1. Video Feature Extraction: Acquire pedestrian videos to be identified and process them into video sequences. For the acquired pedestrian video sequences, extract video sequence features through the backbone network to obtain video feature sequences.

[0037] S2. Spatiotemporal location coding enhancement: Spatial location coding and temporal location coding are introduced into the video feature sequence respectively, and the video feature sequence is superimposed with the two location codes to obtain an enhanced feature sequence;

[0038] S3. Dual-path parallel feature extraction: The enhanced feature sequence is processed by depthwise separable convolution through the local dynamic feature capture path, and then output by nonlinear activation function and residual connection operation; the enhanced feature sequence is dilated temporally sampled through the cross-frame global feature modeling path, and then processed by multi-head attention mechanism to output cross-frame global features.

[0039] S4. Cross-path feature complementarity enhancement: Channel projection processing is performed on the local dynamic features and cross-frame global features respectively, and the corresponding bidirectional gating weights are calculated; based on the bidirectional gating weights and the projected features, weighted fusion is performed to achieve mutual enhancement of the two types of features and output enhanced dual-path features;

[0040] S5. Adaptive Feature Fusion: Based on the enhanced feature sequence, calculate the motion difference index, feature energy index, and occlusion index, input the index into the gating module to generate fusion weights; use the fusion weights to perform weighted fusion of the enhanced dual-path features to obtain dual-path fused features;

[0041] S6. Keyframe-guided reconstruction: Calculate the comprehensive score vector for each frame based on the dual-path fusion features, and select keyframes according to the score results; reconstruct the enhanced feature sequence through a full-frame attention propagation mechanism, and obtain keyframe-guided reconstruction features using a hybrid gating fusion method;

[0042] S7. Multi-feature weighted fusion: The enhanced feature sequence is processed by a multilayer perceptron to obtain refined features; the dual-path fusion features, keyframe-guided reconstruction features and refined features are weighted and fused, and after convolution processing and activation function operation, the final identity features are output;

[0043] S8. Feature matching and result return: Input the final identity feature into the backbone network for further processing to obtain a normalized feature vector; compare the similarity of this normalized feature vector with the pedestrian features in the image library, and generate and return a matching list according to the comparison result.

[0044] As Figure 2 shown, it is the overall framework diagram of a long-term pedestrian re-identification method based on dual-path collaboration and key-frame-guided reconstruction provided by the present invention.

[0045] The present invention provides a preferred embodiment to execute S2. The purpose of this embodiment is to explicitly embed temporal and spatial information into the input video feature sequence, construct the correlation relationship between frames and regions, and provide high-quality input features for subsequent feature modeling.

[0046] S21. Perform frame-level feature extraction processing on the original video sequence to obtain a video feature sequence X = {x1, x2, …, x s}. Among them, x i represents the single-frame feature corresponding to the i-th frame in the temporal feature sequence; s represents the actual time step of the input video feature sequence.

[0047] S22. Construct a learnable spatial position encoding vector P s , which is used to characterize the relative position relationship of different spatial regions in the single-frame feature map; construct a learnable time position encoding vector P t0 , where t0 represents the preset maximum time step.

[0048] Specifically, the time position encoding is adapted according to the actual time step of the input feature sequence, and its adaptation rule is: when the actual time step s of the input feature sequence < t0, intercept the first s time steps of P t0 to obtain the adapted time position encoding; when the actual time step s of the input feature sequence > t0, adopt a cyclic extension strategy, that is, map the time index i to a certain position in P t0 according to the modulo t0. Its calculation formula is:

[0049] P t [i] = P i0 [(i - 1) mod t0 + 1], 1 ≤ i ≤ s

[0050] Among them, mod represents the modulo operation.

[0051] S23. The spatial position encoding vector P s , the adapted time position encoding vector P tThe input video feature sequence X is fused element-wise to generate an enhanced feature sequence Z = {z1, z2, ..., z...} that simultaneously contains semantic and spatiotemporal location information. s The calculation formula is as follows:

[0052] Z = X + P s +P t

[0053] Specifically, the P s With P t All parameters are learnable and are adaptively updated during model training iterations to gradually optimize the correlation between location information and feature semantic information.

[0054] This invention provides a preferred embodiment for executing S3. The purpose of this embodiment is to capture local dynamic features and cross-frame global features through a dual-path parallel design, so as to take into account both fine-grained dynamic changes and long-range temporal correlations, thereby providing multi-level representations for subsequent feature fusion.

[0055] like Figure 3 The diagram shown is a schematic representation of the local dynamic feature capture path structure of a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention.

[0056] S31. Enhance the feature sequence Z by grouping it by channel and performing time-dimension deep convolution. The kernel size for each group of channels is (k t ,1,1), the time dimension padding is Convolution slides along the time dimension, acting on neighboring k. t The features at each time step are padded with zeros to ensure the time step size remains unchanged, resulting in the temporal convolutional features Z. t The calculation formula is as follows:

[0057]

[0058] Among them, Conv3d dw-temp This represents a time-dimension deep convolution.

[0059] S32. Temporal convolutional features Z t Grouping by channel and performing depthwise convolutions in the spatial dimension, each group of channels uses a convolution kernel of size (1, k). s ,k s The spatial dimension padding is p. s Convolution slides along a single frame spatial dimension, acting on k s ×k s Features of local spatial regions are padded with zeros to ensure that the spatial dimension remains unchanged, resulting in spatial convolutional features Z. s The calculation formula is as follows:

[0060] Z s =Conv3d dw-spat (Z t ,k=(1,k s ,k s ),padding=(0,p s ,p s ))

[0061] Among them, Conv3d dw-spat This represents spatial dimension depth convolution.

[0062] S33. Spatial convolution features Z s First, dimensionality reduction is achieved through point convolution, followed by GELU nonlinear activation. Then, point convolution is used to restore the number of channels. Subsequently, batch normalization is performed, and finally, residual connections are made with the enhanced feature sequence Z to obtain the local dynamic feature F. l The calculation formula is as follows:

[0063] F l =Z+BN(Conv3d pw (GELU(Conv3d pw (Z s ))))

[0064] Among them, Conv3d pw represents point convolution; BN represents batch normalization.

[0065] like Figure 4 The diagram shown is a cross-frame global feature modeling path structure for a long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction provided by the present invention.

[0066] S34. The enhanced feature sequence Z is first generated into a query vector Q, a key vector K, and a value vector V through three independent linear projections; then, it is divided into h heads along the channel dimension, i.e., Q = [Q (1) ,…,Q (h) ],K=[K (1) ,…,K (j) ],V=[V (1) ,…,V (h) Then, generate a set of sampling indexes along the time dimension according to the expansion interval d. Only the key vector K and value vector V are sampled according to the set of indices. Extract the features at the corresponding time step to obtain the sampled key vector. Sum vector V s =[V s (1) ,…,V s (h)The query vector Q retains the complete time step s. The calculation formulas are as follows:

[0067] Q = Conv3d Q (Z)

[0068]

[0069] in, Represents the set of sampling indices Perform time-step feature sampling; Conv3d Q Conv3d K Conv3d V These represent three independent convolution operations used to generate the query vector Q, the key vector K, and the value vector V, respectively.

[0070] S35. Regarding query vector Q and key vector K s Value vector V s First, split the dimensions by the head and then perform average pooling on the spatial dimension to obtain... Then With transposed Perform matrix multiplication and divide by the scaling factor. The attention weights A are obtained by Softmax normalization. (j) Finally, the attention weight A is obtained. (j) and Perform matrix multiplication to obtain the attention output O for each head. (j) The calculation formulas are as follows:

[0071]

[0072] Here, Pool represents pooling operations.

[0073] S36. Concatenate the attention outputs of all heads along the channel dimension, restore the spatial dimension by broadcast expansion, and then restore the number of channels by point convolution to obtain the final cross-frame global feature F. g The calculation formula is as follows:

[0074] F g =Conv3d pw (Expand(Concat(O (1) ,…,O (h) )))

[0075] Concat represents the concatenation operation; Expand represents the broadcast expansion operation.

[0076] This invention provides a preferred embodiment for executing S4. The purpose of this embodiment is to achieve mutual enhancement of local dynamic features and cross-frame global features through a bidirectional gating mechanism, allowing the two types of features to exchange information and output enhanced dual-path features.

[0077] S41. Regarding local dynamic features F l With cross-frame global features F g Local projection features F are obtained by performing channel projection through independent point convolution. lg With global projection features F gl The calculation formulas are as follows:

[0078] F lg =Conv3d pw (F l )

[0079] F gl =Conv3d pw (F g )

[0080] S42. The local dynamic features F l The supplementary weights of global features to local features are calculated using a gating network: for F l Adaptive global average pooling is used to extract global statistical information of each channel. After dedicated point convolution transformation, GELU activation, dedicated point convolution to compress the channels to 1 and Sigmoid normalization, the global feature supplementary weights g with values ​​in the range [0,1] are obtained. g The calculation formula is as follows:

[0081] g g =Sigmoid(Conv3d pw (GELU(Conv3d pw (AdaptiveAvgPool(F l )))))

[0082] AdaptiveAvgPool represents adaptive global average pooling.

[0083] The cross-frame global feature F g The supplementary weights of local features to global features are calculated using a gating network: for F g Adaptive global average pooling is used to extract global statistical information of each channel. After specific point convolution transformation, GELU activation, specific point convolution compression of the channels to 1, and Sigmoid normalization, local feature supplementary weights g with values ​​in the range [0,1] are obtained. l The calculation formula is as follows:

[0084] g l =Sigmoid(Conv3dpw (GELU(Conv3d pw (AdaptiveAvgPool(F g )))))

[0085] S43. Based on projection features and gating weights, achieve bidirectional information interaction: in the local enhancement stage, global features are used to supplement weights g. g Introducing global projection feature f gl , with the original local dynamic features F l The residuals are summed to obtain the enhanced local dynamic feature F′. l In the global enhancement stage, local features are used to supplement the weights g. l Introducing local projection feature F lg , and the original cross-frame global feature F g The residuals are summed to obtain the enhanced cross-frame global feature F′. g The calculation formulas are as follows:

[0086]

[0087] in, This indicates an element-wise multiplication operation.

[0088] This invention provides a preferred embodiment for executing S5. This embodiment generates fusion weights by calculating motion difference indices, feature energy indices, and occlusion indices, and then adaptively fuses enhanced local dynamic features and enhanced cross-frame global features.

[0089] S51. Using the enhanced feature sequence Z as input, firstly, the feature differences between adjacent time steps are squared, and then the average is calculated in both the channel and spatial dimensions to obtain the motion difference index D for each frame. Since the difference between adjacent frames is only s-1 time steps, zeros need to be padded to the beginning of the time dimension to ensure alignment with the enhanced feature sequence Z, guaranteeing that the final motion difference index matches the time step size of Z. Next, the feature energy index E is calculated by directly averaging the enhanced feature sequence Z in both the channel and spatial dimensions to obtain the information richness of each frame's features. Finally, the occlusion index M is calculated by averaging the single-frame feature energy indices E across all time steps to obtain the global feature energy index, and then calculating the absolute value of the difference between the single-frame feature energy index and the global feature energy index. The calculation formulas are as follows:

[0090] D = pad t (mean c,H,W ((z i+1 -z i ) 2 )), 1≤i≤s

[0091] E = mean C,H,W (Z)

[0092] M = |E - mean t (E)|

[0093] Among them, pad t This indicates padding with zeros to align the time dimension; mean C,H,W Mean represents the average across the channel and spatial dimensions; t This indicates that the average is taken over the time dimension.

[0094] S52. The three types of indicators D, E, and M are concatenated along the channel dimension to obtain a three-channel indicator matrix U, which is then input into a two-layer convolutional network to generate fusion weights. The first convolutional layer expands the indicator matrix from 3 channels to 32 channels and activates it with GELU to enhance the indicator's expressive power; the second convolutional layer compresses the 32 channels into 2 channels, obtaining the original weight values ​​L, which correspond to the weights for enhancing local dynamic features and the weights for enhancing cross-frame global features, respectively. Then, the weight values ​​are adjusted using a temperature coefficient τ and normalized using Softmax to ensure that the two types of weights satisfy W1 + W2 = 1, forming the final fusion weight W. The calculation formulas are as follows:

[0095] U = [D, E, M]

[0096]

[0097] W = Softmax(L)

[0098] W = [W1, W2]

[0099] Here, Conv represents the convolution operation.

[0100] S53. Enhance local dynamic features F′ l With enhanced cross-frame global features F′ g The two paths are multiplied element-wise by the fusion weight W and then summed to obtain the dual-path fusion feature F. fuse The calculation formula is as follows:

[0101]

[0102] This invention provides a preferred embodiment for executing S6. The purpose of this embodiment is to reconstruct the enhanced feature sequence guided by keyframes through keyframe screening and full-frame attention propagation mechanism, and to balance the full-frame reconstructed features and dual-path fusion features by combining a hybrid gating strategy, and finally output keyframe-guided reconstructed features.

[0103] S61. First, the dual-path fusion feature F fuseGlobal average pooling is performed on the spatial dimension, and the pooling result is then input into the scoring network S(·) to obtain the basic scoring vector. Next, motion difference index D, feature energy index E, and occlusion index M are introduced, and standardization is performed on the basic scoring vector and the three indices respectively. The three indices are then fused according to preset weighting coefficients α, β, and γ to obtain the comprehensive scoring vector r. Finally, r is sorted in descending order, and the first k frame indices are used to form the keyframe index set Ω. The calculation formulas are as follows:

[0104] r = Norm(S(GAP(F) fuse ))+α·Norm(D)+β·Norm(E)+γ·Norm(M))

[0105] Ω = TopK(r,k)

[0106] Norm represents the normalization operation; GAP represents the global average pooling operation.

[0107] S62. Fusing dual-path features F through three sets of independent point convolutions. fuse The projection consists of a query vector q, a key vector k, and a value vector v. Based on the keyframe index set Ω, a subset k corresponding to the keyframes is extracted from k and v. Ω and v Ω For q,k Ω ,v Ω Average pooling is performed on the spatial dimension. Then q and... The similarity matrix, divided by the single-head dimension And overlay keyframe offset b Ω The similarity matrix sim is obtained. Then, sim is divided by the temperature coefficient η and normalized using Softmax to obtain the attention weights a. Finally, a and v are compared... Ω Perform matrix multiplication to obtain the full-frame reconstruction features F a The calculation formulas are as follows:

[0108]

[0109] F a =a·v Ω

[0110] S63. Input the index matrix U into the hybrid gating network G(·) to obtain the hybrid gating weights g, with values ​​in [0,1]. Then, connect g with F... a Performing element-wise multiplication yields the reconstructed feature weighting term, (1-g) and F fuse Element-wise multiplication is performed to obtain the fused feature weighted terms, and element-wise addition is performed on the two weighted terms to obtain the final keyframe-guided reconstruction feature F. rec The calculation formulas are as follows:

[0111] g = Sigmoid(G(U))

[0112]

[0113] This invention provides a preferred embodiment for executing S7. The purpose of this embodiment is to refine and enhance the feature sequence through a multilayer perceptron, and then perform weighted fusion of dual-path fusion features, keyframe-guided reconstruction features, and refined features to achieve complementary integration of multi-source features.

[0114] S71. Input the enhanced feature sequence Z into a multilayer perceptron (MLP). This MLP consists of three pointwise convolutional layers, a GELU activation function, and a dropout layer. It refines the semantic information of the enhanced feature sequence through nonlinear transformation and outputs refined features F. refine The calculation formula is as follows:

[0115] F refine =MLP(Z)

[0116] S72. Using dual-path fusion feature F fuse Keyframe-guided reconstruction features F rec , Refine features F refine As input, learnable fusion weight parameters w1, w2, and w3 are introduced. Element-wise weighting is then performed on the three types of features, and the weighted features are summed element-wise to obtain the fused feature F. merge The learnable fusion weights are normalized using Softmax, satisfying w1 + w2 + w3 = 1. The calculation formula is as follows:

[0117] F merge =w1·F fuse +w2·F rec +w3·F refine

[0118] S73. Using fusion feature F merge The input is fed into the final fusion module. This module consists of a single point convolutional layer and batch normalization, followed by a ReLU activation function to introduce a non-linear transformation to obtain the final features. Finally, a residual connection is performed with the enhanced feature sequence Z to output fixed-dimensional pedestrian identity features F. id The calculation formula is as follows:

[0119] F id =Z+ReLU(BN(Conv3d) pw (F merge )))

[0120] In this embodiment, a ResNet50 network based on an improved AP3D convolutional block is used as the backbone network for feature extraction from the input video sequence. The temperature coefficient of the AP3D convolutional block is set to 4, and a shrinking attention mechanism is enabled. In the experiment, the input video frame sequences are uniformly adjusted to a resolution of 256×128 pixels. During training, 16 identities are randomly selected in each batch, and each identity contains 4 video sequences. The Adam optimizer is used, with an initial learning rate of 0.0002 and a weight decay of 5×10⁻⁶. -4 The learning rate is scheduled using a segmented decay strategy, decreasing every 20 epochs at a rate of 0.1, for a total of 80 training epochs. The loss function is jointly optimized using cross-entropy classification loss and triple pairing loss.

[0121] This embodiment is experimentally validated on two publicly available long-term person re-identification datasets, VCCR and CCVID. Both datasets include pedestrian appearance variations, particularly covering the typical challenging scenario of pedestrians changing clothes, and also include regular samples of pedestrians without changing clothes, which can support performance validation for clothing-changing (CC) and standard scenarios, respectively. The model performance is evaluated using common metrics in the field of person re-identification: mean average precision (mAP) and Rank-1 of cumulative matching features (CMC).

[0122] Table 1 compares the experimental results of the proposed method on the VCCR dataset.

[0123] Table 2 compares the experimental results of the proposed method on the CCVID dataset.

[0124]

[0125] As shown in Tables 1 and 2, the long-term person re-identification method based on dual-path collaboration and keyframe-guided reconstruction proposed in this embodiment outperforms most comparative methods in both the mean accuracy and the Rank-1 index of the cumulative matching features in the clothing-changing and standard scenarios of the VCCR and CCVID datasets. This fully demonstrates the effectiveness and practicality of this method in long-term person re-identification tasks, and it can meet the identity matching requirements in scenarios with complex appearance changes.

Claims

1. A long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction, characterized in that, Includes the following steps: S1. Video Feature Extraction: Acquire pedestrian videos to be identified and process them into video sequences. For the acquired pedestrian video sequences, extract video sequence features through the backbone network to obtain video feature sequences. S2. Spatiotemporal location coding enhancement: Spatial location coding and temporal location coding are introduced into the video feature sequence respectively, and the video feature sequence is superimposed with the two location codes to obtain an enhanced feature sequence; S3. Dual-path parallel feature extraction: The enhanced feature sequence is processed by depthwise separable convolution through the local dynamic feature capture path, and then output by nonlinear activation function and residual connection operation; the enhanced feature sequence is dilated temporally sampled through the cross-frame global feature modeling path, and then processed by multi-head attention mechanism to output cross-frame global features. S4. Cross-path feature complementarity enhancement: Channel projection processing is performed on the local dynamic features and cross-frame global features respectively, and the corresponding bidirectional gating weights are calculated; based on the bidirectional gating weights and the projected features, weighted fusion is performed to achieve mutual enhancement of the two types of features and output enhanced dual-path features; S5. Adaptive Feature Fusion: Based on the enhanced feature sequence, calculate the motion difference index, feature energy index, and occlusion index, input the index into the gating module to generate fusion weights; use the fusion weights to perform weighted fusion of the enhanced dual-path features to obtain dual-path fused features; S6. Keyframe-guided reconstruction: Calculate the comprehensive score vector for each frame based on the dual-path fusion features, and select keyframes according to the score results; reconstruct the enhanced feature sequence through a full-frame attention propagation mechanism, and obtain keyframe-guided reconstruction features using a hybrid gating fusion method; S7. Multi-feature weighted fusion: The enhanced feature sequence is processed by a multilayer perceptron to obtain refined features; the dual-path fusion features, keyframe-guided reconstruction features and refined features are weighted and fused, and after convolution processing and activation function operation, pedestrian identity features are output; S8. Feature Matching and Result Return: The pedestrian identity features are input into the backbone network for further processing to obtain a standardized feature vector; the standardized feature vector is compared with the pedestrian features in the image database, and a matching list is generated and returned based on the comparison results.

2. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The spatial location encoding is a learnable parameter vector; the temporal location encoding is also a learnable parameter vector; when the time step of the input video feature sequence is inconsistent with the preset length of the temporal location encoding, if the input time step is small, the corresponding part of the temporal location encoding is truncated; if the input time step is large, the temporal location encoding is matched with the input time step by a cyclic expansion method.

3. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The local dynamic feature capture path employs depthwise separable convolution, including temporal and spatial depthwise convolutions, where the number of groups for both types of convolutions is consistent with the number of channels in the input features. The convolution window of the temporal depthwise convolution covers features across multiple consecutive frames, while the convolution window of the spatial depthwise convolution covers the spatial region of a single frame's features. After depthwise separable convolution processing, pointwise convolution, nonlinear activation, pointwise convolution, and batch normalization are performed sequentially. The residual connection is implemented by element-wise addition of the enhanced feature sequence to the obtained local dynamic features.

4. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The multi-head attention mechanism generates query vector, key vector, and value vector through three independent linear projections; the query vector, key vector, and value vector are divided into dimensions according to the number of attention heads; the dilation time sampling uses a fixed interval to sample the key vector and value vector; The multi-head attention mechanism calculates attention weights within each attention head using the scaled dot product of the query vector and the key vector, and normalizes them using the Softmax function. The obtained attention weights are used to perform a weighted summation of the value vectors, and the attention outputs of each head are concatenated. After restoring the spatial dimension through broadcast expansion, the signal is then processed through a linear projection layer to obtain cross-frame global features.

5. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The channel projection is achieved through point convolution, which preserves the time, height, and width dimensions of the features. The calculation process of the bidirectional gating weights includes: performing average pooling on local dynamic features and cross-frame global features respectively, and inputting the pooling results into a multilayer perceptron. The multilayer perceptron includes a dimensionality reduction convolutional layer, a nonlinear activation layer, and an dimensionality increase convolutional layer connected in sequence. Finally, gating weights are generated through the Sigmoid function, and the values ​​of the gating weights are in the range of [0,1]. The enhanced dual-path features are achieved by adding the element-wise products of the local dynamic features or cross-frame global features, the corresponding gating weights, and the corresponding projection features.

6. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The motion difference index is obtained by calculating the squared mean of the feature differences between adjacent time steps in the enhanced feature sequence; the feature energy index is obtained by calculating the channel and spatial mean of the single-frame features in the enhanced feature sequence; the occlusion index is obtained by calculating the absolute difference between the single-frame feature energy index and the global feature energy index of the enhanced feature sequence; the gating module first concatenates the motion difference index, feature energy index and occlusion index, and after processing by a feature transformation network containing convolutional layers and nonlinear activation layers, it uses the Softmax function to generate fusion weights, and the sum of the fusion weights of the two types of features is 1; the dual-path fusion feature is achieved by adding the enhanced dual-path features and the obtained fusion weights after element-wise multiplication.

7. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The comprehensive scoring vector is obtained by weighted summation of the standardized base scoring vector, motion difference index, feature energy index, and occlusion index; the base scoring vector is output by the dual-path fusion feature processed by the scoring network; the number of keyframes is adaptively determined according to the total number of video frames; the full-frame attention propagation mechanism first projects the dual-path fusion feature into query vector, key vector, and value vector through point convolution, extracts the key vector subset and value vector subset corresponding to the keyframe, and performs spatial pooling on the query vector, key vector subset, and value vector subset; The similarity matrix between the query vector and the subset of key vectors is calculated. After superimposing the keyframe bias and adjusting the scale, the attention weights are obtained by Softmax normalization. Finally, the attention weights are used to weight and sum the subsets of keyframe value vectors to obtain the full-frame reconstruction features. The hybrid gating weights are generated by a gating function that combines the feature energy index, motion difference index, and occlusion index into an index matrix. The gating function includes a convolutional layer and a sigmoid activation function.

8. The long-term pedestrian re-identification method based on dual-path collaboration and keyframe-guided reconstruction according to claim 1, characterized in that, The multilayer perceptron consists of multiple point convolutional layers, with nonlinear activation layers and Dropout layers sequentially placed between adjacent convolutional layers; the weighted fusion adopts learnable fusion weight parameters; after the fused features are subjected to dimensionality adjustment, batch normalization, and activation function operation by the convolutional layers, they are then fused with the enhanced feature sequence through residual connection to output pedestrian identity features.

Images