Night mirror-crossing pedestrian tracking method based on adaptive pruning technology

Abstract

The invention discloses a night cross-mirror pedestrian tracking method based on an adaptive pruning technology, and the method comprises the steps: obtaining original images of night pedestrians in various environments, carrying out the unsupervised adaptive light enhancement, and obtaining an enhanced image; a double-branch model comprising an original image branch and an enhanced image branch is constructed based on a ViT network model, the original image is input into the original image branch, the enhanced image is input into the enhanced image branch, classification tokens in the original image and the enhanced image are extracted respectively, and classification tokens in the original image and the enhanced image are extracted respectively. And the classification tokens obtained by the original image branch and the enhanced image branch are spliced and fused to obtain complementary pedestrian feature representation so as to carry out pedestrian classification and identification. By adopting the method, complementary information of two images with different qualities can be fully utilized, so that the model can learn night pedestrian features from multiple angles, the problem of information loss of single image input under the condition of insufficient illumination at night is effectively solved, and the accuracy of night cross-mirror pedestrian tracking is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a nighttime cross-camera pedestrian tracking method based on adaptive pruning technology. Background Technology

[0002] In the fields of intelligent security and public safety, achieving continuous identification and tracking of the same pedestrian across multiple camera systems is a key technology for improving the intelligence level of surveillance systems, especially challenging in complex nighttime environments. Cross-camera pedestrian tracking aims to solve the problem of matching the identity of the same pedestrian from different perspectives and cameras. This task is particularly difficult in nighttime environments due to insufficient lighting, increased image noise, weakened color features, and significant loss of pedestrian appearance information, which severely impact the model's consistent modeling of target features. This technology has broad application prospects in intelligent surveillance, smart cities, and public safety. However, in practical applications, cross-camera pedestrian tracking still faces many challenges due to factors such as varying lighting, diverse poses, occlusion, and complex backgrounds. Currently, considerable progress has been made in accuracy under sufficient daylight conditions. However, in nighttime environments, due to insufficient lighting, traditional RGB images often lose a significant amount of texture and color information, leading to a severe decline in image quality, making nighttime cross-camera pedestrian tracking an extremely challenging task. Some researchers have addressed this issue by combining infrared and RGB images, and cross-camera pedestrian tracking based on infrared images is currently a hot research topic. However, infrared images require a significant increase in cost, cannot fully utilize existing surveillance networks, and have inherent problems such as loss of color information, blurred details, and susceptibility to temperature differences, especially at night when temperature differences are small. Therefore, improving the accuracy of nighttime cross-camera pedestrian tracking using pure RGB images is crucial.

[0003] In recent years, ViT models have demonstrated powerful performance in image recognition. Their global receptive field and strong feature learning capabilities have enabled them to outperform traditional convolutional neural networks (CNNs) in many visual tasks. However, ViT models typically require significant computational resources and data, and their performance can be affected when processing low-quality images. Furthermore, the self-attention mechanism in ViT models may suffer from inefficiency when handling redundant information, leading to substantial computational overhead, especially with high feature dimensions.

[0004] Therefore, how to effectively extract discriminative features from pure RGB nighttime images while optimizing model efficiency is a key technical challenge that urgently needs to be addressed in the field of nighttime cross-camera pedestrian tracking. Summary of the Invention

[0005] This invention proposes a nighttime cross-camera pedestrian tracking method based on adaptive pruning technology, aiming to solve the technical problem of pedestrian cross-camera tracking under conditions such as insufficient light at night and difficulty in distinguishing physical features.

[0006] This invention can be achieved through the following technical solutions: A nighttime cross-camera pedestrian tracking method based on adaptive pruning technology includes the following steps: Step 1: Obtain original images of pedestrians at night in various environments; Step 2: Use the Zero-DEC network model to perform unsupervised adaptive ray enhancement on the original image to obtain the enhanced image; Step 3: Construct a dual-branch model, which includes an original image branch and an enhanced image branch, both of which are built based on the ViT network model. The original image is input into the original image branch and the enhanced image is input into the enhanced image branch, and classification tokens are extracted from the original image and the enhanced image, respectively. The original image branch and the enhanced image branch both include multi-layer Transformer Block modules. Each Transformer Block module uses a learnable query space containing a fixed number of query tokens to guide each Transformer Block module to perform adaptive pruning on the original image or the enhanced image, remove unimportant image tokens, and pass them together with the retained image tokens and classification tokens to the next Transformer Block module. Finally, the classification token of the last Transformer Block module is extracted and used as the classification token extracted from the original image branch or the enhanced image branch; Step 4: Concatenate and fuse the classification tokens obtained from the original image branch and the enhanced image branch to obtain complementary pedestrian feature representations, which are then used for pedestrian classification and recognition.

[0007] Furthermore, an input feature representation X is constructed using image tokens, classification tokens, and spatial location encoding. For each Transformer Block module, adaptive pruning is performed according to the following steps: S1. Input the image token in the input feature representation X into the learnable query space, calculate the attention weight of the query token, and simultaneously input the input feature representation X into the current layer Transformer Block module to calculate the attention weight of the image token and the attention weight of the classification token. Then, average the attention weights of the classification token, the query token, and the image token to obtain the importance score of the image token. S2. Perform matrix multiplication between the importance score of the image token and the attention weight of the query token, and find the minimum value as the pruning threshold for the original image branch or the enhanced image branch. S3. Based on the obtained pruning threshold, remove image tokens that are not important in the original or enhanced image, and pass them together with the retained image tokens and classification tokens to the next layer Transformer Block module.

[0008] Furthermore, let the input features of the first-layer Transformer Block be represented as: , in, E represents the embedding feature of the classification token, P represents the embedding feature of the image token, and P represents the spatial location encoding. In step S1, firstly, the image token extracted from the input feature representation X is... Input the query tokens into a learnable query space and compute the attention weights of the query tokens. ; Simultaneously, the input feature representation X is input into the Transformer Block module. With the help of the multi-head attention module, the query matrix Q, key matrix K, and value matrix V are calculated. Then, by performing matrix dot product operations on the query matrix Q and key matrix K, and then normalizing the results, the attention distribution matrices between image tokens, between classification tokens, and between image tokens are obtained. Where B is the batch size, h is the number of attention heads, and N+1 represents N image tokens plus one classification token, then from the attention distribution matrix. Extract the attention distribution submatrix of the image tokens. Then, normalization is performed to obtain the attention weights of the image tokens. ; Simultaneously from the attention distribution matrix Extract the attention distribution submatrix corresponding to the classification tokens Then, by using the attention distribution matrix Matrix multiplication is performed on the value matrix V to obtain the feature representation of each attention head output, and then normalization is performed to obtain the density value, which is used as the attention distribution submatrix of the weights for the classification tokens. The attention weights of the classification tokens are obtained by weighting the tokens. ; Finally, the importance score of the image token is obtained by averaging the attention weights of the query token, image token, and classification token.

[0009] Furthermore, the attention weight of the query token is calculated using the following formula. ; in, Image token The transpose of N, where N represents the number of image tokens. This represents the query token; the SoftMax activation function is used to normalize the attention. The attention weights of the image tokens are calculated using the following formula. ; The attention weights of the classification tokens are calculated using the following formula. ; Where h represents the number of attention heads in the multi-head attention module.

[0010] Further, in step S3, the importance score of the image token is compared with the pruning threshold. Image tokens with an importance score greater than the pruning threshold are retained, while those with an importance score less than the pruning threshold are removed. The importance score of the removed image token is then multiplied by the corresponding image token to form a new token, i.e., the fused token. The fused token, the retained image token, and the classification token of the current Transformer Block module are then concatenated in the embedding dimension to form a new token sequence, which is then passed to the input of the next Transformer Block module via a feedforward neural network.

[0011] Furthermore, firstly, a convolution operation is performed on the input original image or enhanced image. Then, a linear projection layer is used to divide the convolutional image data into blocks and map them to a high-dimensional feature space to obtain the image's embedding features. Each embedded feature corresponds to an image token, B is the batch size, N is the number of image tokens, D is the embedding dimension of the model, and a classification token is also added. And add a spatial location code to each embedded feature. The final input feature representation X is: in, Let E represent the embedding feature of the classification token, E represent the embedding feature of the image token, and P represent the corresponding spatial location information encoding.

[0012] Furthermore, the total loss function of the dual-branch model Defined as follows, in, The query features represent the branches of the original image. This represents the query characteristics of the enhanced branch, where K represents the number of query tokens. Represents classification loss, Indicates the loss of the triplet. This represents the hyperparameters of the model.

[0013] The beneficial technical effects of this invention are as follows: 1. This invention introduces a dual-branch model based on the ViT network model, and processes the original image and the enhanced image after Zero-DCE enhancement separately. Since the original image retains the most realistic scene information, while the enhanced image provides clearer visual details, this dual-branch design can make full use of the complementary information of two images of different quality, enabling the model to learn nighttime pedestrian features from multiple angles. This effectively overcomes the problem of information loss when a single image input is used in low-light conditions at night, thus significantly improving the accuracy of nighttime cross-camera pedestrian tracking.

[0014] 2. This invention calculates the image token importance score by comprehensively considering the attention weights of the weighted classification token, the image token, and the query token, and then performs pruning based on this score to ensure that the model retains features crucial for pedestrian identification. Simultaneously, the use of a cosine loss function brings the learnable space features of the two branches closer together, ensuring semantic consistency of the retained image tokens and avoiding the erroneous pruning of critical image tokens. The final fused classification token integrates discriminative information from both the original and enhanced images, forming a more comprehensive pedestrian feature representation and enhancing the model's adaptability to complex nighttime scenes.

[0015] 3. This invention guides an adaptive pruning mechanism through a learnable query space, dynamically removing unimportant image tokens based on their importance. This avoids the enormous computational overhead of the ViT model when processing large amounts of redundant information, significantly reducing computational complexity and memory usage while maintaining high performance. This advantage is particularly pronounced in resource-constrained real-world deployment environments, contributing to the realization of an efficient nighttime cross-camera pedestrian tracking system. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the overall process of the present invention.

[0017] Figure 2 This is a visualization comparing the first 10 matching results of the present invention with those of the TransReid-SSL[1] method. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0019] To address the shortcomings of existing technologies, such as Figure 1 As shown, this invention provides a nighttime cross-camera pedestrian tracking method based on adaptive pruning technology. It acquires nighttime pedestrian images from multiple camera perspectives within an existing camera surveillance network; utilizes a Zero-DEC network for unsupervised adaptive light enhancement of the nighttime images; constructs a dual-branch model based on the VisionTransformer (ViT) model, with one branch processing the original image and the other processing the enhanced version; the model is initialized using weights trained on a large-scale unlabeled training set LUperson; a learnable query space guides each Transformer Block of the ViT model for adaptive pruning; and a cosine loss function is used to guide the semantic similarity of the query token features between the two branches, ensuring semantic consistency of the retained image tokens and avoiding the erroneous pruning of key image tokens. During the inference phase, the classification tokens of the two branches are directly concatenated, and their feature similarity is calculated to complete sorting and matching. Compared with existing technologies, this invention not only significantly improves the accuracy of cross-camera pedestrian tracking in pure RGB nighttime images but also reduces the computational overhead of the model and possesses good edge deployment capabilities. This method requires no additional hardware support and can be directly integrated into existing surveillance systems, making it suitable for key scenarios such as public safety monitoring and nighttime evidence collection.

[0020] The steps of a specific embodiment are as follows: Step 1: Obtain original images of pedestrians at night under various environments. Randomly select four pedestrian images from the existing large-scale nighttime cross-camera pedestrian tracking dataset (Night600) and name them pedestrian 1, 2, 3, and 4. This dataset was collected by eight cameras in a common surveillance system, mainly capturing images of pedestrians on campus roads at night, covering various complex lighting conditions, such as streetlights, car headlights, reflective lights from billboards, and starlight. Step 2: Use the Zero-DEC network model to perform unsupervised adaptive lighting enhancement on the image data of these four people to obtain enhanced images.

[0021] Step 3: Construct a dual-branch model, which includes an original image branch and an augmented image branch, both based on the ViT network model. The original image is input into the original image branch, and the augmented image is input into the augmented image branch. The parameters between the two branches are not shared. The classification tokens are extracted from the original image branch and the augmented image branch respectively.

[0022] Step S31: First, preprocess the original image, mainly by converting it into a tensor while fixing the image size to 128*256; then perform some common data augmentation techniques, such as horizontal flipping, random cropping, and erasing, resulting in the final image data. Where B represents the batch size, H represents the image height, and W represents the image width, the image data I is then input into the Zero_DEC network model to obtain the enhanced image data. .

[0023] The following describes image data I and The same operation will be performed, and the following explanations will all use image data I as the input data; To obtain more stable image feature embeddings, the original image branches are used. First, the input data is convolved before feature embedding. This feature embedding mainly includes a linear projection layer and positional encoding: the linear projection layer divides the image data into blocks and maps them to a high-dimensional feature space to obtain the image's embedded features. Each embedded feature corresponds to an image token, representing a local region feature of the image. B is the batch size, N is the number of image tokens, and D is the embedding dimension of the model. To further aggregate the global semantic information of the image and participate in the final classification decision, a classification token is added. To preserve spatial location information, a spatial location code is added to each image token and classification token. The final input feature representation X is: , in, Let E represent the embedding feature of the classification token, E represent the embedding feature of the image token, and P represent the corresponding spatial location information encoding.

[0024] Step S32: Typically, the ViT models corresponding to the original image branch and the enhanced image branch both include multiple Transformer Block modules. Each Transformer Block module includes the following operation steps: Layer Normalization, Multi-Head Attention, and Feed-Forward Network (FFN). This invention introduces an adaptive image token pruning module based on the traditional Transformer Block. This adaptive image token pruning module is inserted between the Multi-Head Attention module and the Feed-Forward Network to dynamically prune unimportant image tokens, thereby accelerating the overall model's computation process. One Transformer Block module represents one layer in the ViT model. The same adaptive pruning operation is performed on each layer. The following detailed explanation uses the first layer as an example: To optimize the computational efficiency of the ViT model and enhance its feature representation capabilities, a learnable query space is introduced into the ViT model. This query space consists of a fixed number of query tokens, and all query tokens in the learnable space are initialized. Where K is the number of query tokens and D is the embedding dimension of the model. These query tokens are learnable during model training, and their core role is to guide each TransformerBlock module in the dual-branch ViT model to perform adaptive pruning to reduce interfering features and accelerate the model's inference speed.

[0025] Image tokens in input feature representation X Input the query tokens into a learnable query space and compute the attention weights of the query tokens. The calculation formula is as follows: , in, for The transpose of , where N is the number of image tokens, and the SoftMax activation function is used to normalize the attention; Simultaneously, the input feature representation X is input into the current layer's Transformer Block module. First, the input feature representation X undergoes layer normalization processing, and then it is sent to the multi-head attention module for feature modeling. In this module, the input feature representation X generates the query matrix Q, the key matrix K, and the value matrix V through linear transformations, respectively. The calculation expression is as follows: ; in, These are the linear transformation weight matrices for the query, key, and value, respectively.

[0026] Subsequently, using the adaptive image token pruning module, based on the standard calculation method of the self-attention mechanism, a dot product operation is first performed on the query matrix Q and the key matrix K, and then the attention distribution matrix between each token is obtained by normalization using the softmax function. Where B is the batch size, h is the number of attention heads, and N+1 represents N image tokens plus one classification token, where 'token' refers to both image tokens and classification tokens. Attention distribution matrix. This represents the attention weight of each token to other tokens, such as image token to image token, image token to classification token; based on this, from the attention distribution matrix... Extract the attention distribution submatrix of the image tokens. Then, normalization is performed to obtain the attention weights of the image tokens. The calculation process can be represented as follows: ; ; Then from the attention distribution matrix Extract the attention distribution submatrix corresponding to the classification tokens By using the attention distribution matrix Matrix multiplication is performed on the value matrix V to obtain the feature representation of each attention head's output, which is used as the information density of that attention head. Then, the calculated information density is normalized to obtain the density value, which is used as the attention distribution submatrix for the weights of the classification tokens. The attention weights of the classification tokens are obtained by weighting the tokens. The calculation formula can be expressed as: , Where h is the number of attention heads in the multi-head attention module.

[0027] Finally, the importance score of the image token is obtained by averaging the attention weights of the query token, image token, and classification token. The formula for calculating the importance score of the image token is as follows: .

[0028] Step S33: Assign importance scores to image tokens Attention weights with query tokens Perform matrix multiplication and find the minimum value as the pruning threshold.

[0029] Based on the pruning threshold, a mask is generated for each image token. Image tokens larger than the pruning threshold are retained and passed to the next layer, while those smaller than or equal to the pruning threshold are removed. The formula is as follows: , in, Represents the i-th image token.

[0030] To prevent potentially important but inactive image tokens from being completely discarded, the importance scores of all removed image tokens are further multiplied with the image tokens to form a new token, the fused token. This fused token, along with the retained image tokens and the classification tokens of this layer, is concatenated along the embedding dimension to form a new token sequence, which is then passed to the input of the next Transformer Block module via a feedforward neural network. This process continues until the last Transformer Block module. At this point, the number of concatenated tokens increases by two compared to the retained image tokens, with one being the fused token and the other being the classification token, while the feature dimension of each token remains unchanged. Finally, the classification token of the last Transformer Block module is extracted and used as the classification token obtained from the original image branch.

[0031] Throughout the adaptive pruning process, the model continuously removes redundant image tokens in each Transformer Block module; at the same time, it retains the classification tokens and continuously interacts with the retained image tokens (including fusion tokens) through the self-attention mechanism of each layer to aggregate global semantic information.

[0032] During model training, a cosine loss function is introduced. As the total loss function Part of it, used to constrain the semantic consistency of query tokens in a two-branch model; in pedestrian re-identification tasks, classification loss is commonly used. and triplet loss Following the design of classic pedestrian re-identification models, the classification tokens are fed into two loss functions: on the one hand, the classification tokens are processed through a batch normalization bottleneck structure before being input into the classification loss function; on the other hand, classification tokens that have not undergone this structure are directly input into the triplet loss. This invention further introduces semantically aligned cosine loss to construct a complete total loss, where... The model's hyperparameters are set to 0.1, and the total loss function is... Defined as follows: Regarding the cosine loss function In this study, the cosine similarity between the query tokens of the original image branch and the augmented branch is calculated, and their difference is minimized to ensure the semantic consistency of the retained image tokens and avoid erroneous pruning of key image tokens. The cosine loss function... Defined as: , in, The query features for the original image branch. To enhance the query characteristics of branches, K is the number of query tokens.

[0033] Step 4: Concatenate and fuse the classification tokens obtained from the original image branch and the enhanced image branch to obtain complementary pedestrian feature representations. Pedestrian classification and recognition are then performed using these representations. Specifically, the similarity between the fused feature vector and other pedestrian image feature vectors is calculated. All other pedestrian image feature vectors are calculated in the same way and sorted according to their similarity to obtain the recognition ranking.

[0034] Since the nighttime environment naturally becomes a protective camouflage for criminals to evade surveillance, in order to better improve the role of cross-camera pedestrian tracking in public safety, this invention mainly targets the problem of cross-camera pedestrian tracking in pure RGB images at night. Compared with the TransReid-SSL[1] method, the cross-camera pedestrian tracking learning model obtained by the above method has the following advantages: Figure 2 It can be clearly seen that the method of the present invention can effectively improve the efficiency and accuracy of pedestrian target re-identification at night. Green indicates a correct match, and red indicates an incorrect match. The top of the image shows the matching result of the method of the present invention, and the bottom of the image shows the matching result of the TransReid-SSL[1] method.

[0035] References [1] Luo H, Wang P, Xu Y, Ding F, Zhou Y, Wang F, Li H, Jin R (2021) "Self-supervised pre-training for transformer-based person re-identification". arXiv:211112084.

[0036] The method of the present invention can be specifically applied to: A. Applications in urban security and intelligent monitoring: In urban nighttime security monitoring, smart communities, subway stations, bus stations, scenic spots and other places, it can realize personnel tracking, identity association and abnormal behavior detection under low light conditions at night, and does not require the deployment of additional infrared hardware, making it suitable for large-scale deployment; B. Application in criminal investigation: Assists criminal investigators in quickly locating and identifying suspects at night or in poor lighting conditions, thereby enhancing the efficiency of solving cases; C. Application of nighttime management in industrial parks and key areas: nighttime personnel control in key areas such as construction sites, industrial plants, logistics parks, airports, and ports, achieving nighttime identity verification, illegal intrusion warnings, and nighttime patrol trajectory recording.

[0037] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A nighttime cross-camera pedestrian tracking method based on adaptive pruning technology, characterized in that... Includes the following steps: Step 1: Obtain original images of pedestrians at night in various environments; Step 2: Use the Zero-DEC network model to perform unsupervised adaptive ray enhancement on the original image to obtain the enhanced image; Step 3: Construct a dual-branch model, which includes an original image branch and an enhanced image branch, both of which are built based on the ViT network model. The original image is input into the original image branch and the enhanced image is input into the enhanced image branch, and classification tokens are extracted from the original image and the enhanced image, respectively. The original image branch and the enhanced image branch both include multi-layer Transformer Block modules. Each Transformer Block module uses a learnable query space containing a fixed number of query tokens to guide each Transformer Block module to perform adaptive pruning on the original image or the enhanced image, remove unimportant image tokens, and pass them together with the retained image tokens and classification tokens to the next Transformer Block module. Finally, the classification token of the last Transformer Block module is extracted and used as the classification token extracted from the original image branch or the enhanced image branch; Step 4: Concatenate and fuse the classification tokens obtained from the original image branch and the enhanced image branch to obtain complementary pedestrian feature representations, which are then used for pedestrian classification and recognition.

2. The nighttime cross-camera pedestrian tracking method based on adaptive pruning technology according to claim 1, characterized in that: The input feature representation X is constructed using image tokens, classification tokens, and spatial location codes. For each Transformer Block module, adaptive pruning is performed according to the following steps. S1. Input the image token in the input feature representation X into the learnable query space, calculate the attention weight of the query token, and simultaneously input the input feature representation X into the current layer Transformer Block module to calculate the attention weight of the image token and the attention weight of the classification token. Then, average the attention weights of the classification token, the query token, and the image token to obtain the importance score of the image token. S2. Perform matrix multiplication between the importance score of the image token and the attention weight of the query token, and find the minimum value as the pruning threshold for the original image branch or the enhanced image branch. S3. Based on the obtained pruning threshold, remove image tokens that are not important in the original or enhanced image, and pass them together with the retained image tokens and classification tokens to the next layer Transformer Block module.

3. The nighttime cross-camera pedestrian tracking method based on adaptive pruning technology according to claim 2, characterized in that: Let the input features of the first-layer Transformer Block be represented as: , in, E represents the embedding feature of the classification token, P represents the embedding feature of the image token, and P represents the spatial location encoding. In step S1, firstly, the image token extracted from the input feature representation X is... Input the query tokens into a learnable query space and compute the attention weights of the query tokens. ; Simultaneously, the input feature representation X is input into the Transformer Block module. With the help of the multi-head attention module, the query matrix Q, key matrix K, and value matrix V are calculated. Then, by performing matrix dot product operations on the query matrix Q and key matrix K, and then normalizing the results, the attention distribution matrices between image tokens, between classification tokens, and between image tokens are obtained. Where B is the batch size, h is the number of attention heads, and N+1 represents N image tokens plus one classification token, then from the attention distribution matrix. Extract the attention distribution submatrix of the image tokens. Then, normalization is performed to obtain the attention weights of the image tokens. ; Simultaneously from the attention distribution matrix Extract the attention distribution submatrix corresponding to the classification tokens Then, by using the attention distribution matrix Matrix multiplication is performed on the value matrix V to obtain the feature representation of each attention head output, and then normalization is performed to obtain the density value, which is used as the attention distribution submatrix of the weights for the classification tokens. The attention weights of the classification tokens are obtained by weighting the tokens. ; Finally, the importance score of the image token is obtained by averaging the attention weights of the query token, image token, and classification token.

4. The nighttime cross-mirror pedestrian tracking method based on adaptive pruning technology according to claim 3, characterized in that: The attention weight of the query token is calculated using the following formula. ; in, Image token The transpose of N, where N represents the number of image tokens. This represents the query token; the SoftMax activation function is used to normalize the attention. The attention weights of the image tokens are calculated using the following formula. ; The attention weights of the classification tokens are calculated using the following formula. ; Where h represents the number of attention heads in the multi-head attention module.

5. The nighttime cross-camera pedestrian tracking method based on adaptive pruning technology according to claim 2, characterized in that: In step S3, the importance score of the image token is compared with the pruning threshold. Image tokens with an importance score greater than the pruning threshold are retained, while those with an importance score less than the pruning threshold are removed. The importance score of the removed image token is then multiplied by the corresponding image token to form a new token, i.e., the fused token. The fused token, the retained image token, and the classification token of the current Transformer Block module are then concatenated in the embedding dimension to form a new token sequence, which is then passed to the input of the next Transformer Block module via a feedforward neural network.

6. The nighttime cross-camera pedestrian tracking method based on adaptive pruning technology according to claim 2, characterized in that: First, a convolution operation is performed on the input original image or enhanced image. Then, a linear projection layer is used to divide the convolutional image data into blocks and map them to a high-dimensional feature space to obtain the image's embedding features. Each embedded feature corresponds to an image token, B is the batch size, N is the number of image tokens, D is the embedding dimension of the model, and a classification token is also added. And add a spatial location code to each embedded feature. The final input feature representation X is: in, Let E represent the embedding feature of the classification token, E represent the embedding feature of the image token, and P represent the corresponding spatial location information encoding.

7. The nighttime cross-camera pedestrian tracking method based on adaptive pruning technology according to claim 1, characterized in that: The total loss function of the two-branch model Defined as follows, in, The query features represent the branches of the original image. This represents the query characteristics of the enhanced branch, where K represents the number of query tokens. Represents classification loss. Indicates the loss of the triplet. This represents the hyperparameters of the model.

Images