Pedestrian re-identification method based on modal perception interaction and adaptive triple alignment

Abstract

The present application relates to the technical field of computer vision, and in particular to a pedestrian re-identification method based on modal perception interaction and adaptive triple alignment. In the present application, a pedestrian re-identification model including a feature extractor and a modal perception interaction module is trained, and pedestrian re-identification is realized through the pedestrian re-identification model. The modal perception interaction module guides the model to dynamically model the internal structural differences in the modal at the initial stage of feature extraction through a space and channel joint attention mechanism, thereby enhancing the modal adaptability and mutual information modeling capability; the dynamic temperature weighted triple loss introduces a dynamic temperature mechanism and cooperates with the adaptive center loss to strengthen the structural modeling between samples and alleviate the instability of the traditional triple loss in cross-modal training.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a pedestrian re-identification method based on modal-aware interaction and adaptive triple alignment. Background Technology

[0002] Person re-identification (ReID) is an important research direction in computer vision, aiming to achieve pedestrian matching and retrieval across cameras and scenes. With the widespread adoption of surveillance networks and the growing demand for intelligent security, this technology has broad application prospects in public safety, smart cities, and other fields. Traditional ReID relies on visible light images, but its performance significantly degrades at night or in low-light environments. Therefore, visible light-infrared person re-identification (VI-ReID) technology has emerged. The core challenge of VI-ReID lies in modal differences: visible light images are rich in color and texture information, suitable for well-lit scenes; infrared images provide stable contour and thermal information in low-light / no-light environments. The key issue is how to establish a semantic alignment mechanism between the two modalities and extract modality-independent discriminative features.

[0003] Existing VI-ReID methods fall into three categories: 1) Modality-shared feature learning, which attempts to map features of different modalities to a shared space, but modality differences pose a significant challenge; 2) Modality-specific information compensation, which generates missing modality images or features through generative adversarial networks (GANs), but the generation quality may be unstable; 3) Metric learning strategies, which optimize the feature space to bring cross-modal samples of the same identity closer together and push away samples of different identities.

[0004] Despite numerous efforts in recent years to mitigate representational differences across modalities, VI-ReID still faces two major challenges: 1) Insufficient modal feature alignment: Due to differences in perception methods and image distribution, there is a significant gap in the underlying features of infrared and visible light images, making it difficult for existing methods to simultaneously capture discriminative and modal semantic differences; 2) Unstable training optimization: Triple-based metric learning is effective in single-modal tasks, but hard sample mining strategies in cross-modal scenarios are susceptible to modal distribution biases, leading to noise introduction, training oscillations, or performance fluctuations. Summary of the Invention

[0005] Based on this, the present invention provides a pedestrian re-identification method based on modality-aware interaction and adaptive triplet alignment, which solves at least one problem in the prior art.

[0006] In a first aspect, the present invention provides a method for training a pedestrian re-identification model, which includes the following steps:

[0007] Construct a pedestrian re-identification model;

[0008] The pedestrian re-identification model was trained using a dataset that included visible light and infrared images, where the original feature maps were extracted through convolutional operations. Channel focus weights are obtained through a channel semantic focus mechanism. Spatial focusing weights are obtained through a spatial semantic focusing mechanism. And applied to the original feature map respectively. The following steps are taken: Low-dimensional statistical vectors of the visible light and infrared modes are extracted separately, concatenated, and processed by a multilayer perceptron to obtain cross-modal attention weights. Cross-modal attention weights Features that integrate visible light and infrared modes;

[0009] The parameters of the pedestrian re-identification model are updated based on the target loss.

[0010] In some optional embodiments, channel focus weights are obtained through a channel semantic focus mechanism. The operation is shown in the following formula:

[0011]

[0012] in, This indicates multilayer perceptron processing; Indicates global average pooling; Indicates global max pooling; This indicates the Sigmoid activation operation.

[0013] In some optional embodiments, spatial focusing weights are obtained through a spatial semantic focusing mechanism. The operation is shown in the following formula:

[0014]

[0015] in, This indicates that the average value is taken over the channel dimension; This indicates taking the maximum value in the channel dimension; This represents a 7×7 convolution operation.

[0016] In some optional embodiments, low-dimensional statistical vectors of the visible light mode and the infrared mode are extracted separately, concatenated, and then processed by a multilayer perceptron as shown in the following equation:

[0017]

[0018]

[0019]

[0020] in, and These represent the low-dimensional statistical vectors of the visible light mode and the infrared mode, respectively; Characteristic map representing visible light modes, Characteristic map representing infrared modes; This indicates a global average pooling operation; This indicates multilayer perceptron processing; Indicates the Sigmoid activation operation; This represents the cross-modal attention weights.

[0021] In some optional embodiments, cross-modal attention weights are used. The operation for fusing features from visible light and infrared modes is shown in the following equation:

[0022]

[0023] in, Indicates channel focus weight Applied to the original feature map The visible light modal feature map obtained above; Representation space focus weight Applied to the original feature map The infrared modal feature map obtained above; Represents cross-modal attention weights; This represents the fused feature map.

[0024] In some optional embodiments, the target loss is a joint loss consisting of identity classification cross-entropy loss, dynamic temperature-weighted triplet loss, KL loss, and adaptive center loss, as shown in the following equation:

[0025]

[0026] in, Indicates joint loss; This represents the cross-entropy loss for identity classification. This represents the dynamic temperature-weighted triplet loss; Represents the adaptive center loss; Indicates KL's loss; This indicates the relative weights for controlling various losses. The weights represent the weights that control the strength of the adaptive center loss. This represents the relative weights used to control KL loss.

[0027] In some optional embodiments, the dynamic temperature-weighted triplet loss is calculated by the following formula:

[0028]

[0029] in, This represents the dynamic temperature-weighted triplet loss; Indicates the number of training samples in a batch; This represents the preset interval hyperparameter; This indicates a larger value operation; Indicates the first The weighted positive sample distance of each sample is calculated by the following formula:

[0030]

[0031]

[0032] in, Indicates the first The set of positive samples of each sample; Indicates the first Individual samples and positive samples Euclidean distance; Indicates the first The sample and the first The weighting coefficients of each positive sample; This represents the distance to the mean of positive samples. Indicates the first Temperature control factors during round training; Indicates exponentiation;

[0033] Indicates the first The weighted negative sample distance of each sample is calculated by the following formula:

[0034]

[0035]

[0036] in, Indicates the first The set of negative samples of each sample; Indicates the first The sample and the first Euclidean distance of each negative sample; Indicates the first The sample and the first The weighting coefficients of each negative sample; This represents the distance to the mean of negative samples; Indicates the first Temperature control factors during round training; This indicates exponentiation.

[0037] Secondly, the present invention provides a pedestrian re-identification method based on modality-aware interaction and adaptive triple alignment, which includes the following steps:

[0038] The visible light image and infrared image to be identified are input into the pedestrian re-identification model, and the pedestrian re-identification model outputs the identification result.

[0039] The pedestrian re-identification model is obtained by the pedestrian re-identification model training method described above.

[0040] Thirdly, the present invention provides a pedestrian re-identification device, comprising:

[0041] At least one processor;

[0042] and memory that is communicatively connected to at least one processor;

[0043] The memory stores instructions that, when executed by at least one processor, implement the pedestrian re-identification model training method or the pedestrian re-identification method based on modal-aware interaction and adaptive triplet alignment as described above.

[0044] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the pedestrian re-identification model training method or the pedestrian re-identification method based on modal-aware interaction and adaptive triplet alignment as described above.

[0045] Due to the adoption of the above technical solutions, the embodiments of the present invention have at least the following beneficial effects: the modality-aware interaction module guides the model to dynamically model the internal structural differences of the modality in the early stage of feature extraction through the spatial and channel joint attention mechanism, thereby enhancing the modality adaptability and mutual information modeling ability; the introduction of dynamic temperature-weighted triplet loss and adaptive center loss with dynamic temperature mechanism strengthens the structural modeling between samples and alleviates the instability faced by traditional triplet loss in cross-modal training. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the network structure of a pedestrian re-identification model in one embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram of the network structure of the MAIN module in one embodiment of the present invention.

[0048] Figure 3 This is a scatter plot of the feature embedding space of the test set samples visualized using t-SNE in one embodiment of the present invention, where (a) is the result obtained from the baseline experiment and (b) is the result after introducing MAIN and DTW-TL. Detailed Implementation

[0049] The following will provide a clear and complete description of the concept and technical effects of the present invention, so as to fully explain the purpose, solution and effects of the present invention.

[0050] To address the issues of insufficient modal feature alignment and unstable training optimization in existing technologies, this invention provides a pedestrian re-identification method based on modal-aware interaction and adaptive triplet alignment. This method employs a novel framework of co-designed structure and optimization. Within this framework, a modal-aware interaction module is designed. This module dynamically models the internal structural differences of modalities in the early stages of feature extraction through a joint spatial and channel attention mechanism, enhancing modal adaptability and mutual information modeling capabilities. Furthermore, a dynamic temperature-weighted triplet loss function is designed, introducing a dynamic temperature mechanism and further employing an adaptive center loss function to strengthen structural modeling between samples and mitigate the instability of traditional triplet loss in cross-modal training.

[0051] Specifically, a pedestrian re-identification model is trained, and pedestrian re-identification is achieved through this model. For example... Figure 1 As shown, in one implementation example, the pedestrian re-identification model includes a backbone network, a feature aggregation and embedding mapping layer, and a recognition and retrieval module. The backbone network embeds a modality-aware interaction module (MAIN module). The backbone network can be a ResNet-50, used to extract initial features from visible light and infrared images. The MAIN module is embedded in the early stages of the backbone network (ResNet-50 includes convolutional layers and residual layers; the MAIN module is embedded between the convolutional and residual layers), enhancing intramodal saliency representation through channel and spatial attention mechanisms, and combining a multilayer perceptron (MLP) with a dynamic fusion mechanism to achieve adaptive interaction of cross-modal features. The feature aggregation and embedding layer maps features to a unified representation space. The recognition and retrieval module is optimized during training using a joint loss function, and performs cross-modal retrieval based on feature similarity during testing.

[0052] The pedestrian re-identification model can be trained using a cross-modal dataset containing visible light and infrared images, denoted as follows: , and These represent the visible light mode and the infrared mode, respectively. Each visible light image consists of three channels: R, G, and B, represented as follows: Each infrared image has a single oversaturated grayscale channel, represented as The goal is to achieve cross-modal visible-infrared matching through a feature extractor, with the learning objective being to optimize the relationship between the extracted features of the visible light image in the three-channel RGB space and the features of the infrared image in the single-channel space.

[0053] First, the input visible light image. and infrared images First, the image passes through a convolutional module to extract initial features. The feature representations of the visible light and infrared images are as follows: The convolutional module can consist of a 3x3 depthwise grouped convolution (stride of 1, padding of 1) and a 1x1 convolutional layer.

[0054] Then, the initial features are input into the MAIN module. For example... Figure 2 As shown, in the MAIN module, the features of the two modalities (visible light and infrared) enter the channel attention submodule (channel semantic focusing) and then the spatial attention submodule (spatial structure focusing) to enhance the salient regions and structural information within the modality. Then, in the guiding branch, the statistics (mean and variance) of the features are calculated, and the fusion weights are predicted using a multilayer perceptron (MLP) and input to the modality-aware fusion submodule, thereby achieving a weighted summation of visible light and infrared features to obtain adaptive interaction and collaborative modeling results across modal information. Considering that visible light images are more discriminative in terms of texture and color, while infrared images are more stable in terms of structure and shape, the MAIN module utilizes semantic focusing mechanisms in the spatial and channel dimensions to strengthen the expression of key information within the modality and weaken redundant or interfering regions for fine-grained modeling of modal differences.

[0055] Specifically, Channel-wise Semantic Highlighting extracts statistical information for each channel through global pooling, guiding the model to focus on feature dimensions with high semantic value, thus forming channel-wise weights. The operations in the channel attention submodule can be represented as:

[0056]

[0057] in, Represents the feature map; This indicates multilayer perceptron processing, used to further extract features or adjust dimensions; Indicates the feature map Perform global average pooling; Indicates the feature map Perform global max pooling; This represents the Sigmoid activation operation, used to map the generated channel weights to the [0,1] interval.

[0058] Spatial Semantic Highlighting models the spatial distribution of feature maps after channel compression and generates spatial attention weights through convolution. Highlighting key areas of the human body; the operations in the spatial attention submodule can be represented as:

[0059]

[0060] in, Indicates the feature map By averaging across the channel dimensions, a spatial attention map is obtained. Indicates the feature map Taking the maximum value in the channel dimension also yields a spatial attention map; This represents a 7×7 convolution operation.

[0061] The final output of channel semantic focusing and spatial semantic focusing is a fusion of the two attention weighting results. The two focusing weights are then applied to the original features to achieve semantic emphasis at the structural level, as shown in the following equation:

[0062]

[0063] in, This represents the enhanced feature map after joint weighting by channel attention and spatial attention. Indicates channel attention weights. represents the spatial attention weight, and ⊙ represents the feature map being multiplied at each spatial location.

[0064] Since the modal contribution levels may dynamically change in different scenarios, a modal fusion strategy (dynamic fusion controller) driven by statistical features is further designed to achieve information-guided interaction. Low-dimensional statistical vectors (such as channel mean) are extracted from each modality. and variance ), after inputting an MLP, generate fusion weights This is used to weight the features of the two modalities. Specifically, for each modality... and Perform global average pooling along the channel direction, as shown in the following formula:

[0065]

[0066]

[0067] in, Characteristic map representing visible light modes, Characteristic map representing infrared modes, This indicates a global average pooling operation. and These represent the channel mean vectors (i.e., global feature vectors). The two statistical vectors are concatenated and input into an MLP, which outputs the fusion coefficients. As shown in the following formula:

[0068]

[0069] in, Represents cross-modal attention weights. ; This indicates multilayer perceptron processing; This indicates the Sigmoid activation operation.

[0070] Modal information fusion is achieved through weighted summation to obtain the output features of the MAIN module. As shown in the following formula:

[0071]

[0072] in, and These represent the visible light and infrared modal feature maps after channel and spatial attention enhancement, respectively. ∈[0,1] represents the fusion weights generated by the multilayer perceptron (MLP), which are used to control the relative contributions of the two modalities in the fusion; This represents the weight corresponding to the infrared mode, ensuring that the sum of the weights of the two modes is 1; This represents the fused feature map output by the MAIN module, which serves as the input to the subsequent recognition network.

[0073] In the MAIN module, joint modeling using channel attention and spatial attention guides the network to focus on salient human regions during the initial feature extraction process, thereby enhancing the discriminative representation within modalities. The MLP performs nonlinear mapping and statistical modeling on the enhanced modal features to mitigate the distribution differences between visible and infrared modalities. The modal perception fusion submodule adaptively learns dynamic weights between modalities based on statistical information provided by the MLP, achieving bidirectional perception and structural collaborative modeling across modalities. Through this hierarchical design of "attention enhancement—feature mapping—dynamic fusion," the MAIN module guides the model to dynamically model internal structural differences within modalities in the early stages of feature extraction using a joint spatial and channel attention mechanism, effectively improving the modal adaptability and mutual information modeling capabilities of the features. The fusion process not only adapts to dynamic changes in the importance between modalities but also enhances the model's sensitivity to the quality of input modal features, improving the final matching performance. The MAIN module, through its dual mechanism of structural perception and weight guidance, enables the model to perceive structural differences between modalities in the early feature modeling stage, improving the flexibility and discriminative ability of the fused representation. This module does not require the introduction of significant parameters or computational overhead, and can be easily embedded into various backbone networks, exhibiting good adaptability and portability.

[0074] Next, the feature aggregation and embedding layer maps the modal features of the fused feature map to a unified representation space. This feature aggregation and embedding layer may include a global average pooling layer, BNNeck, a fully connected layer, and an L2 normalization layer.

[0075] Finally, the identification and retrieval module incorporates an ID classifier during the training phase and optimizes the model using identity classification cross-entropy loss, Dynamic Temperature-Weighted Triplet Loss (DTW-TL), KL loss, and adaptive center loss. During the testing phase, cross-modal pedestrian retrieval is achieved through feature similarity calculation. This implicitly aligns semantics under a unified objective without requiring additional modeling of modal differences.

[0076] Identity classification cross-entropy loss (identity loss) is a commonly used supervisory signal in multimodal person re-identification tasks, aiming to improve the semantic discriminativeness of samples. Specifically, the extracted features are mapped to a class probability distribution through a fully connected layer and optimized using cross-entropy loss, defined as follows:

[0077]

[0078] in, This represents the cross-entropy loss for identity classification. Indicates sample Embedded features; This indicates their identity label; Indicates input features Under these conditions, the network predicts the category to which the sample belongs through the Softmax layer. The probability of; This represents the number of samples contained in a training mini-batch. This loss effectively guides the network to learn semantically consistent discriminative features and forms the basis for subsequent metric learning modules.

[0079] Based on the boundary constraints, the metric learning part adopts two complementary mechanisms: First, to address the problem of treating all positive and negative samples equally in the traditional triplet loss, dynamic temperature-weighted triplet loss (DTW-TL) is introduced. Its dynamic temperature weighting strategy allocates weights according to the relative distance difference between samples and the progress of the training phase, making the model more focused on structurally sensitive difficult samples, while weakening simple samples, and avoiding treating all samples "equally".

[0080] For a dynamic temperature-weighted strategy, given the number of training rounds Temperature control factor Defined as:

[0081]

[0082] in, Indicates the first Temperature control factors during round training; Indicates the number of training rounds; Indicates the initial temperature; Indicates the attenuation coefficient (0 < <1); Indicates the lower limit of the minimum temperature; This indicates an operation to select the larger value.

[0083] Temperature decay allows training to focus on global feature structure in the early stages and gradually shift to fine-grained optimization in the later stages. For the samples... Its positive sample set negative sample set positive sample weighted distance for:

[0084]

[0085]

[0086] in, Indicates sample The weighted positive sample distance; Indicates sample The set of positive samples; Indicates sample Compared with positive samples Euclidean distance; Indicates sample Compared with positive samples Weighting coefficients; This represents the distance to the mean of positive samples. Indicates the first Temperature control factors during round training; This indicates exponentiation.

[0087] Negative sample weighted distance The definition is similar, as shown in the following formula:

[0088]

[0089]

[0090] Indicates sample The weighted negative sample distance; Indicates sample The set of negative samples; Indicates sample With negative samples Euclidean distance; Indicates sample With negative samples Weighting coefficients; This represents the distance to the mean of negative samples; Indicates the first Temperature control factors during round training; This indicates exponentiation.

[0091] The final dynamic temperature-weighted triplet loss is:

[0092]

[0093] in, This represents the dynamic temperature-weighted triplet loss; Indicates the number of training samples in a batch; Indicates sample The weighted positive sample distance; Indicates sample The weighted negative sample distance; This indicates the preset margin hyperparameter; This indicates the operation of taking the larger value, used to ensure that the loss is non-negative. It can be seen that the dynamic temperature-weighted strategy strengthens hard examples and mitigates the interference of outlier samples on training through Softmax weights.

[0094] To further compress the intra-class feature distribution and enhance discriminative power, an adaptive center loss is introduced to encourage sample features to cluster around the class center, specifically defined as:

[0095]

[0096] in, Represents the adaptive center loss; This represents the feature vector of the i-th sample; Indicates sample Belonging to the identity category The center vector. During training, this center vector is updated based on the mean of sample features, and a momentum mechanism is used to prevent it from being disturbed by noisy samples.

[0097] Furthermore, class variance is used as the "temperature state" to guide the center update frequency, improving the adaptability of the adaptive center loss to complex distributions. It's important to note that the adaptive center loss is not parallel to the dynamic temperature-weighted strategy, but rather added as a regularization constraint to the dynamic temperature-weighted triplet loss. In practice, the dynamic temperature-weighted triplet loss and the adaptive center loss are typically used together.

[0098] Finally, the cross-entropy loss based on identity classification is combined. Dynamic temperature-weighted triplet loss KL loss In a joint optimization model, the joint loss function can be defined as:

[0099]

[0100] in, This indicates the relative weights for controlling various losses. The weights represent the weights that control the strength of the adaptive center loss. This represents the relative weights controlling the KL loss. This joint loss design significantly enhances cross-modal feature alignment and training stability while ensuring feature discriminativeness.

[0101] To verify the effectiveness of the person re-identification model in this embodiment of the invention, experiments were conducted on the SYSU-MM01 dataset and the RegDB dataset, and a systematic comparison was made with existing mainstream models for person re-identification.

[0102] The SYSU-MM01 dataset, released by the Laboratory of Intelligent Science and Systems at Sun Yat-sen University, is the first RGB-infrared cross-modal pedestrian re-identification dataset. It contains 45,863 images of 491 pedestrians, covering various indoor and outdoor scenes, and was acquired by four visible light cameras and two infrared cameras. The dataset is divided into training, validation, and test sets, supporting both full search and indoor search test modes. Its multimodal nature, scene diversity, and large-scale data provide an important experimental platform for cross-modal pedestrian re-identification research, and it is widely used in fields such as security monitoring and intelligent transportation.

[0103] The RegDB dataset is a dedicated dataset for visible-infrared cross-modal person re-identification, containing 8240 images of 412 individual pedestrians, with 10 visible light images and 10 infrared images for each pedestrian. The dataset is divided into training and testing sets. Its bimodal characteristics, accurate annotation, and structured design make it an ideal choice for evaluating cross-modal person re-identification algorithms. It not only supports pedestrian re-identification tasks but can also be extended to research in cross-modal retrieval, multi-sensor fusion, and other fields.

[0104] A person re-identification model was trained and tested using PyTorch on an NVIDIA 3080Ti GPU. First, a ResNet-50 pre-trained on ImageNet was used as the base feature extractor, with the MAIN module inserted as an additional unit into the shallow stages of the network. This allows the model to perform differential modeling and semantic focusing on the visible light and infrared modalities before moving to higher-level semantic modeling. This maintains the integrity of the backbone network while enhancing the expressive power of cross-modal features. The input image size was uniformly adjusted to 3×288×144 pixels. During training, all images underwent random horizontal flipping and random erasure for data augmentation. The optimizer used stochastic gradient descent (momentum coefficient set to 0.9, weight decay coefficient set to 0.0005), and the training process consisted of 80 epochs. The learning rate was initially set at 0.1 and linearly increased to the target value in the first 10 rounds. It then decayed in stages: 0.1 was maintained from rounds 10 to 20, reduced to 0.01 from rounds 20 to 50, and further reduced to 0.001 after round 50. Each batch contained 8 identities, each provided with 4 pairs of visible-infrared images, for a total of 64 samples.

[0105] During the testing phase, two main evaluation metrics were used. First, the standard rank-k (rk) matching accuracy was used. This means that if any of the top k candidate results returned by the search database contains a sample that matches the identity of the query image, it is considered a successful match, and the overall matching accuracy is calculated. Second, the mean accuracy (mAP) was used. This metric calculates the average retrieval accuracy across all query samples, comprehensively reflecting the model's overall retrieval performance under different queries, and is the primary indicator for evaluating effectiveness.

[0106] Table 1 shows the evaluation results of different models on the SYSU-MM01 dataset, and Table 2 shows the evaluation results of different models on the RegDB dataset. It can be seen that the model proposed in this embodiment outperforms several existing methods in both recognition rate and retrieval accuracy on the SYSU-MM01 dataset, thus verifying its effectiveness and advancement in multi-scene cross-modal person re-identification tasks. In the single-sample all-search task of the SYSU-MM01 dataset, the Rank-1 recognition rate of the person re-identification model in this embodiment is 71.76%, and the mAP is 68.48%. In the indoor scene-specific search task, the Rank-1 recognition rate of the person re-identification model in this embodiment further increases to 75.54%, and the mAP increases to 78.60%. For cross-modal retrieval tasks on the RegDB dataset, the person re-identification model in this embodiment demonstrates superior performance in both visible light and infrared bidirectional searches. In the visible light image query to infrared image library (VIS→IR) mode, it achieves a Rank-1 of 86.02% and a mAP of 79.61%, while in the infrared image query to visible light image library (IR→VIS) mode, it achieves a Rank-1 of 85.61% and a mAP of 79.40%. Experimental data fully demonstrate that the person re-identification model in this embodiment achieves industry-leading levels in multi-scene, multi-modal retrieval tasks. Its performance advantage stems from its ability to collaboratively mine cross-modal shared features and modality-specific features, showing significant improvements across all metrics compared to similar models, thus validating the effectiveness of the proposed framework.

[0107] Table 1 Evaluation results of different models on the SYSU-MM01 dataset

[0108]

[0109] Table 2 Evaluation results of different models on the RegDB dataset

[0110]

[0111] In Table 2, V to I means querying the infrared image library through a visible light image, I to V means querying the visible light image library through an infrared image, and r=1 means rank-1 matching accuracy.

[0112] To verify the effectiveness of each module, detailed ablation experiments were conducted on the SYSU-MM01 dataset. The performance changes after gradually adding modules are shown below. Here, Base represents the basic model using only ResNet-50 as the backbone; Base+MAIN represents the introduction of the Modal Attention Interaction Network (MAIN) module into the basic model; Base+DTW-TL represents the addition of dynamic temperature-weighted triplet loss (with simultaneous introduction of adaptive center loss) to the basic model; and Base+MAIN+DTW-TL represents the complete model combining the MAIN module and dynamic temperature-weighted triplet loss. As shown in Tables 3 and 4, each component contributes to improving recognition performance. Adding the MAIN module significantly improves performance, verifying its effectiveness in modal structure modeling and attention guidance. Adding DTW-TL further improves performance, indicating that the dynamic temperature weighting strategy and the introduced center regularization loss effectively alleviate training instability. The combination of both achieves optimal performance, demonstrating a good synergistic effect between structural modeling and loss optimization. Ultimately, the synergistic combination of all components produces the best performance, thus verifying the effectiveness of the person re-identification model in this embodiment of the invention.

[0113] Table 3 Results of global search ablation experiments

[0114]

[0115] Table 4 Results of the Indoor Scene-Specific Search Ablation Experiment

[0116]

[0117] Furthermore, t-SNE is used to visualize the feature embedding space of the test set samples to demonstrate the advantages of the pedestrian re-identification model in feature distribution in this embodiment of the invention. For example... Figure 3 As shown, after adding the MAIN module and DTW-TL, the distribution of different identity features is more separated, the intra-class features are more compact, and the inter-modal fusion is significantly improved.

[0118] In summary, the pedestrian re-identification model in this embodiment achieves superior Rank-1 accuracy and mAP performance on both the SYSU-MM01 and RegDB datasets, while demonstrating stronger training robustness and consistency in feature visualization distribution. Ablation experiments further validate the effectiveness and synergy between the MAIN module and DTW-TL in structural modeling and feature learning.

[0119] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention by the same or equivalent means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A method for training a pedestrian re-identification model, characterized in that, Includes the following steps: Construct a pedestrian re-identification model; The pedestrian re-identification model was trained using a dataset that included visible light and infrared images, where the original feature maps were extracted through convolutional operations. Channel focus weights are obtained through a channel semantic focus mechanism. Spatial focusing weights are obtained through a spatial semantic focusing mechanism. And applied to the original feature map respectively. The following steps are taken: Low-dimensional statistical vectors of the visible light and infrared modes are extracted separately, concatenated, and processed by a multilayer perceptron to obtain cross-modal attention weights. Cross-modal attention weights Features that integrate visible light and infrared modes; Update the parameters of the pedestrian re-identification model based on the target loss; The process of extracting low-dimensional statistical vectors of the visible light mode and the infrared mode respectively, concatenating them, and then processing them through a multilayer perceptron is shown in the following equation: in, and These represent the low-dimensional statistical vectors of the visible light mode and the infrared mode, respectively; Characteristic map representing visible light modes, Characteristic map representing infrared modes; This indicates a global average pooling operation; This indicates multilayer perceptron processing; Indicates the Sigmoid activation operation; Represents cross-modal attention weights; The cross-modal attention weights The operation for fusing features from visible light and infrared modes is shown in the following equation: in, Indicates channel focus weight Applied to the original feature map The visible light modal feature map obtained above; Representation space focus weight Applied to the original feature map The infrared modal feature map obtained above; Represents cross-modal attention weights; Represents the fused feature map; The target loss is a joint loss consisting of identity classification cross-entropy loss, dynamic temperature-weighted triplet loss, KL loss, and adaptive center loss, as shown in the following equation: in, Indicates joint loss; This represents the cross-entropy loss for identity classification. This represents the dynamic temperature-weighted triplet loss; Represents the adaptive center loss; Indicates KL's loss; This indicates the relative weights for controlling various losses. The weights represent the weights that control the strength of the adaptive center loss. Represents the relative weights controlling the KL loss; The dynamic temperature-weighted triplet loss is calculated by the following formula: in, This represents the dynamic temperature-weighted triplet loss; Indicates the number of training samples in a batch; This represents the preset interval hyperparameter; This indicates a larger value operation; Indicates the first The weighted negative sample distance of each sample; Indicates the first The weighted positive sample distance of each sample is calculated by the following formula: in, Indicates the first The set of positive samples of each sample; Indicates the first Individual samples and positive samples Euclidean distance; Indicates the first The sample and the first The weighting coefficients of each positive sample; This represents the distance to the mean of positive samples. Indicates the first Temperature control factors during round training; Indicates exponentiation; Indicates the first The weighted negative sample distance of each sample is calculated by the following formula: in, Indicates the first The set of negative samples of each sample; Indicates the first The sample and the first Euclidean distance of each negative sample; Indicates the first The sample and the first The weighting coefficients of each negative sample; This represents the distance to the mean of negative samples; Indicates the first Temperature control factors during round training; This indicates exponentiation.

2. The method according to claim 1, characterized in that, Obtain channel focus weights through channel semantic focus mechanism The operation is shown in the following formula: in, This indicates multilayer perceptron processing; Indicates global average pooling; Indicates global max pooling; This indicates the Sigmoid activation operation.

3. The method according to claim 2, characterized in that, Obtain spatial focus weights through spatial semantic focus mechanism The operation is shown in the following formula: in, This indicates that the average value is taken over the channel dimension; This indicates taking the maximum value in the channel dimension; This represents a 7×7 convolution operation.

4. A pedestrian re-identification method based on modality-aware interaction and adaptive triple alignment, characterized in that, Includes the following steps: The visible light image and infrared image to be identified are input into the pedestrian re-identification model, and the pedestrian re-identification model outputs the identification result. The pedestrian re-identification model is obtained by the pedestrian re-identification model training method as described in any one of claims 1-3.

5. A pedestrian re-identification device, characterized in that, include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that, when executed by at least one processor, implement the pedestrian re-identification model training method as described in any one of claims 1-3 or the pedestrian re-identification method based on modal-aware interaction and adaptive triplet alignment as described in claim 4.

6. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed by a processor, implement the pedestrian re-identification model training method as described in any one of claims 1-3 or the pedestrian re-identification method based on modal-aware interaction and adaptive triplet alignment as described in claim 4.

Images