Single-modal and adaptive fusion re-identification method based on relationship distillation guidance

Abstract

The application discloses a single-modal and adaptive fusion re-identification method based on relationship distillation guidance, which comprises the following steps: step S1, constructing online multi-modal teacher features; step S2, respectively calculating a pair relationship distillation loss for aligning local metric relationships and a global structure distillation loss for aligning global metric topological structures; step S3, obtaining enhanced single-modal student features; step S4, taking the enhanced single-modal student features as a plurality of independent expert features; step S5, generating a set of adaptive modal fusion weights for a current sample; step S6, weighting and fusing corresponding expert features by using the adaptive modal fusion weights; and step S7, optimizing parameters of the modal perception router. The application enhances single-modal representation and adaptively fuses multi-modal features through relationship distillation, so as to relieve modal inertia and improve the robustness and retrieval precision of multi-modal re-identification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a single-modality and adaptive fusion re-identification method based on relation distillation guidance. Background Technology

[0002] The goal of object re-identification is to match images of the same object in non-overlapping camera views. Currently, most research focuses on re-identification tasks based on RGB images. However, RGB modalities are prone to failure under conditions such as low light and inclement weather. In contrast, near-infrared (NIR) and thermal infrared (TIR) ​​modalities maintain high reliability in complex environments. Therefore, utilizing complementary information from different modalities can significantly improve the robustness and generalization ability of re-identification, making multimodal object re-identification a promising research direction.

[0003] Existing multimodal object re-identification methods primarily focus on feature fusion to leverage complementary information between different modalities. However, these methods often overlook the problem of modality imbalance. Due to differences in information richness and quality among different modalities, models tend to over-rely on the dominant modality while neglecting weaker modalities. This ultimately leads to "modal inertia" and degrades the quality of the fused representation.

[0004] Effective multimodal fusion depends on two key factors: discriminative and complementary single-modal representations and a dynamically adaptive fusion mechanism. Specifically, if single-modal features are poorly represented, direct fusion not only fails to leverage the complementary advantages of different modalities but also amplifies modal imbalance, severely limiting the integration of complementary information and reducing overall performance. On the other hand, even if single-modal representations are sufficiently robust and discriminative, different modalities often exhibit varying degrees of reliability under constantly changing environmental conditions, and fixed fusion strategies cannot dynamically adapt to changes in the reliability of each modality. Therefore, an adaptive fusion mechanism is essential for effectively utilizing complementary cues from multimodal features.

[0005] To address the problem of unimodal representation enhancement, a common solution is to apply task-related losses independently to each modality. However, this paradigm utilizes only information from a single modality, resulting in poor representation performance and failing to capture complementary knowledge between different modalities. Multimodal representation integrates complementary information between different modalities, thus capturing richer identity semantics than unimodal representation and theoretically serving as a useful guide for unimodal learning. However, knowledge distillation in multimodal learning remains in the exploratory stage. Traditional feature distillation forces strict alignment between features, which is ineffective due to inherent modal heterogeneity.

[0006] In summary, existing technologies suffer from problems such as insufficient single-modal representation learning, modal imbalance, lack of adaptive fusion mechanisms, and the unsuitability of traditional distillation methods for multi-modal attitude learning. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the present invention aims to provide a single-modal and adaptive fusion re-identification method based on relation distillation, which enhances single-modal representation through relation distillation and adaptively fuses multimodal features to alleviate modal inertia and improve the robustness and retrieval accuracy of multimodal re-identification.

[0008] To achieve the objective of this invention, the following solution is adopted: The relation distillation-guided single-modality and adaptive fusion re-identification method includes the following steps: Step S1: Obtain a small batch of data containing multiple modal images, extract the features of each unimodal student using a shared-weight visual encoder, and concatenate the features of each unimodal student to construct online multimodal teacher features; Step S2: Based on the unimodal student features and the multimodal teacher features, calculate the pairwise distillation loss for aligning local metric relations and the global structural distillation loss for aligning global metric topology, respectively. Step S3: Combine the preset task loss, the pairwise distillation loss, and the global structure distillation loss to perform end-to-end training on the visual encoder to obtain enhanced single-modal student features; Step S4: Freeze the parameters of the trained visual encoder and treat the enhanced unimodal student features as multiple independent expert features; Step S5: Stack the expert features and input them into the modality-aware router. Model the intermodal relationships through a self-attention mechanism to generate a set of adaptive modality fusion weights for the current sample. Step S6: Use the adaptive modal fusion weights to perform weighted fusion on the corresponding expert features to obtain multimodal fusion features; Step S7: Calculate the task loss based on the multimodal fusion features, and introduce a modal balance loss to prevent modal collapse. Optimize the parameters of the modal-aware router to obtain the final multimodal fusion features for object re-identification retrieval.

[0009] Furthermore, in step S1, the construction of online multimodal teacher features specifically involves: concatenating the CLS token features of each modality corresponding to each sample within the same mini-batch of data along the feature dimension, and this concatenation operation is performed synchronously with the forward propagation of the visual encoder, without requiring a pre-trained independent teacher model.

[0010] Further, in step S2, the pairwise distillation loss is calculated by: calculating the cosine distance matrix between the multimodal teacher features and the unimodal student features in the mini-batch samples, and then minimizing the mean square error between the two distance matrices.

[0011] Further, in step S2, the global structural distillation loss is calculated as follows: the Gromov-Wasserstein distance is introduced to measure the internal structural difference between the first metric space composed of the multimodal teacher features and the second metric space composed of the unimodal student features, and the optimal transmission plan is solved by the Sinkhorn-Knopp algorithm to calculate the loss.

[0012] Furthermore, in step S3, the task loss includes: label smoothing cross-entropy loss for identity classification and triplet loss for metric learning.

[0013] Further, in step S5, the modality-aware router specifically includes: A multi-head self-attention layer is used to process the stacked expert features and output attention-enhanced features; Residual connections are used to add the stacked expert features to the attention-enhanced features; Flattening operation is used to flatten the output of residual connection into one dimension; Multilayer perceptron is used to map the results of flattening operations to unnormalized modal weights; The Softmax function is used to transform the unnormalized modal weights into normalized adaptive modal fusion weights.

[0014] Further, in step S6, the weighted fusion specifically involves multiplying the expert features of each modality by the adaptive modality fusion weight corresponding to that modality, and then concatenating them along the feature dimension to obtain the multimodal fusion features.

[0015] Further, in step S7, the modal balance loss specifically involves calculating the Kullback-Leibler divergence between the distribution of the adaptive modal fusion weights and the preset uniform distribution, which is used to constrain the weight distribution of each mode to tend towards uniformity and avoid the fusion process from relying solely on the dominant mode.

[0016] Furthermore, in step S1, the visual encoder is a pre-trained CLIP ViT backbone network, and the same shared weights are used for the three modes of RGB, near-infrared and thermal infrared.

[0017] Further, in step S3, the pairwise distillation loss and the global structure distillation loss are multiplied by the first hyperparameter and the second hyperparameter, respectively, and then added to the task loss for optimization; In step S7, the modal balance loss is multiplied by the third hyperparameter and then added to the task loss for optimization.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention effectively alleviates modal inertia and improves weak modality discrimination ability. Addressing the problem in existing multimodal fusion models that over-rely on the dominant modality and neglect weak modalities, this invention constructs online multimodal teacher features and simultaneously employs pairwise relation distillation loss and global structure distillation loss to guide single-modal student learning. This transfers complementary multimodal knowledge to each single-modal feature, significantly enhancing the representation quality of weak modalities, thereby alleviating modal inertia and improving the overall average performance of single modalities.

[0019] 2. This invention achieves dynamic adaptive fusion, improving fusion robustness. Addressing the problem that existing fixed fusion strategies cannot adapt to environmental changes leading to fluctuations in modal reliability, this invention treats enhanced single-modal features as independent experts and dynamically generates adaptive fusion weights for each sample using a modality-aware router with a self-attention mechanism. Simultaneously, by combining modality balance loss to constrain the weight distribution to tend towards uniformity, it can flexibly adjust the fusion contribution according to the modal reliability of each sample, preventing modal collapse and maintaining superior fusion representation under changing conditions.

[0020] 3. This invention significantly improves the retrieval accuracy of multimodal re-identification. The two-stage design of first enhancing the single-modal representation and then adaptively fusing it ensures that the final multimodal fusion feature contains both discriminative single-modal information and can dynamically combine complementary clues. Experiments show that this invention achieves better mAP and Rank-1 accuracy than state-of-the-art methods on multiple public datasets such as RGBNT201, RGBNT100, and MSVR310, realizing the best multimodal retrieval performance.

[0021] 4. This invention employs a lightweight design and boasts high computational efficiency. It utilizes a shared-weight visual encoder, and online teacher features do not require pre-training of independent teacher models; the modality-aware router introduces only a small number of learnable parameters. The overall model has a small number of parameters and low floating-point operations, achieving lightweight design while maintaining high performance, facilitating practical deployment. Attached Figure Description

[0022] Figure 1 This is a flowchart of the single-modal and adaptive fusion re-identification method based on relational distillation in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0024] like Figure 1 As shown, this embodiment of the invention provides a single-modality and adaptive fusion re-identification method based on relation distillation guidance, including the following steps: Step S1: Obtain a small batch of data containing multiple modal images, extract the features of each unimodal student using a shared-weight visual encoder, and concatenate the features of each unimodal student to construct online multimodal teacher features.

[0025] Step S2: Based on the unimodal student features and the multimodal teacher features, calculate the pairwise distillation loss for aligning local metric relations and the global structural distillation loss for aligning global metric topology.

[0026] Step S3: Combine the preset task loss, the pairwise distillation loss, and the global structure distillation loss to perform end-to-end training on the visual encoder to obtain enhanced single-modal student features.

[0027] Step S4: Freeze the parameters of the trained visual encoder and treat the enhanced unimodal student features as multiple independent expert features.

[0028] Step S5: Stack the expert features and input them into the modality-aware router. Model the intermodal relationships through a self-attention mechanism to generate a set of adaptive modality fusion weights for the current sample.

[0029] Step S6: Use the adaptive modal fusion weights to perform weighted fusion on the corresponding expert features to obtain multimodal fusion features.

[0030] Step S7: Calculate the task loss based on the multimodal fusion features, and introduce a modal balance loss to prevent modal collapse. Optimize the parameters of the modal-aware router to obtain the final multimodal fusion features for object re-identification retrieval.

[0031] The following is a more detailed description of the single-modality and adaptive fusion re-identification method based on relation distillation guided by the present invention.

[0032] This invention proposes a two-stage training framework called RDAF, which is a single-modal and adaptive fusion re-identification method based on relation distillation. This framework first improves the single-modal representation through multimodal relation distillation, and then performs adaptive multimodal fusion to dynamically utilize complementary information.

[0033] Phase 1: Enhancement of single-modal representation based on multimodal relation distillation.

[0034] In this embodiment, step S1 is the multimodal feature extraction and online teacher construction step, the specific content of which is as follows: Input: Mini-batch data containing RGB, NIR, and TIR modal images. ,in .

[0035] deal with: 1. Use a shared-weights visual encoder (such as a pre-trained CLIPViTbackbone) to extract features from each modality image. For the , From a sample, the CLStoken features of each modality are obtained. .

[0036] 2. Constructing online teachers: Concatenate the CLStoken features of the three unimodal students to form multimodal teacher features. The teacher features are dynamically generated and do not require pre-training of a separate teacher model.

[0037] Output: Unimodal student features and multimodal teacher characteristics .

[0038] In this embodiment, step S2 includes a pairwise relation distillation (PRD) step, the details of which are as follows: Principle: Utilizing the rich complementary information in the teacher's features, the pairwise distance between samples within a small batch is calculated, and the student's features are guided to imitate this distance relationship.

[0039] deal with: 1. Calculate the cosine distance matrix between teacher samples. The cosine distance matrix between the student samples and the sample .

[0040] 2. Define the PRD loss function Minimize the mean square error between the student distance matrix and the teacher distance matrix: .

[0041] 3. During the calculation process, the teacher's side distance is cut off. Gradient propagation.

[0042] Function: To convey fine-grained local metric relationships, enabling single-modal features to learn more discriminative and complementary representations.

[0043] In this embodiment, step S2 includes a global structure distillation (GSD) step, the details of which are as follows: Principle: For PRDs, when the gap between students and teachers is too large, students will overfit teacher pairwise relationships, or when there are noisy samples, students will fit incorrect pairwise relationships. These factors cause local errors to accumulate at the batch level, ultimately destroying the global structure of the student metric space. Therefore, it is necessary to align the student and teacher metric spaces at the global structural level.

[0044] deal with: 1. Introduce the Gromov-Wasserstein (GW) distance to measure the difference in the internal structure of two metric spaces (teacher space and student space).

[0045] 2. Define the GSD loss function : in To find the optimal transmission plan, the Sinkhorn-Knopp algorithm is used, and entropy regularization is introduced to accelerate the optimization.

[0046] 3. Similarly, cut off the propagation of the teacher-side gradient.

[0047] Function: To maintain a compact global topology within classes and a separation between classes, improve the robustness of the distillation process, and prevent noisy samples from destroying the feature space structure.

[0048] In this embodiment, step S3 is the stage one total loss optimization step, the specific content of which is as follows: Combining task loss (label smoothing cross-entropy) and triplet loss Using distillation loss, the visual encoder is trained end-to-end: .

[0049] Phase 2: Adaptive multimodal fusion based on modality-aware routers.

[0050] In this embodiment, step S4 is the freeze encoder and expert feature preparation step, the specific content of which is as follows: Freeze the parameters of the trained visual encoder in Phase 1. Then apply the enhanced unimodal features. It is considered an independent branch of "experts".

[0051] In this embodiment, step S5 is the weight generation step for the modality-aware router (MA-Router), and the specific content is as follows: Principle: Explicitly model the relationships between multiple modalities and dynamically allocate fusion weights based on the specific circumstances of the samples.

[0052] deal with: 1. Feature stacking: Stacking the features of the three modalities into a matrix. .

[0053] 2. Multimodal self-attention: Processed through a multi-head self-attention mechanism. Capture interaction information between modalities to obtain attention output. .

[0054] 3. Weight Mapping: The original stacked features are added to the attention output, then mapped through residual connections, flattening operations, and a multilayer perceptron (MLP). Finally, normalized modality weights are generated using the Softmax function. : , where F is the flattening operation and G is the multilayer perceptron.

[0055] Output: The modality fusion weights for each sample.

[0056] In this embodiment, step S6 is an adaptive weighted fusion step, the details of which are as follows: The generated weights are used to weight and combine the features of each modality to obtain the final multimodal fusion feature. : .

[0057] In this embodiment, step S7 is the modal balance constraint and stage two optimization step, the specific content of which is as follows: deal with: 1. To prevent the router from collapsing into a single dominant mode, a modal balance loss is introduced. That is, weighted distribution and uniform distribution KL divergence between: .

[0058] 2. Train only the parameters of MA-Router, and optimize using task loss and balancing loss: .

[0059] Output: The final multimodal fusion features used for retrieval and the trained routing network.

[0060] The relation distillation-guided single-modal and adaptive fusion re-identification method of this invention adopts a two-stage training framework, decoupling the multimodal re-identification task into two independent optimization stages: "single-modal relation distillation enhancement" and "adaptive multimodal fusion." The first stage transfers complementary multimodal knowledge to each single-modal branch through relation distillation, enhancing the discriminative power of the single-modal representation. The second stage, based on a frozen encoder, optimizes only the modality-aware router, dynamically fusing features from each modality. Each stage focuses on its core task, avoiding conflicts arising from joint multi-objective optimization.

[0061] The relation distillation-guided unimodal and adaptive fusion re-identification method of this invention employs an online multimodal teacher construction mechanism. It eliminates the need for pre-training independent teacher models, instead concatenating the CLS token features extracted from each unimodal student within the same batch along the feature dimension to dynamically construct multimodal teacher features. These teacher features integrate complementary information from different modalities, capturing richer identity semantics than unimodal representations, and evolve synchronously with student network updates, forming a positive feedback mechanism for student-teacher collaborative enhancement.

[0062] The single-modal and adaptive fusion re-identification method based on relation distillation in this invention employs a combined relation distillation loss function, using both pairwise relation distillation (PRD) and global structure distillation (GSD) to guide single-modal representation learning. PRD transmits fine-grained local metric relationships by minimizing the mean square error of the pairwise distance matrices between students and teachers in mini-batch samples; GSD introduces Gromov-Wasserstein distance to align the global metric space topology of students and teachers, and solves for the optimal transmission plan using the Sinkhorn-Knopp algorithm. These two methods complement each other, balancing local metric alignment and global topology preservation, improving the robustness of the distillation process to noisy samples, and preventing the global structure of the student's metric space from being destroyed.

[0063] The relation distillation-guided single-modality and adaptive fusion re-identification method of this invention employs a modality-aware router (MA-Router). Expert features are stacked and input into a multi-head self-attention layer to capture intermodal interaction information. Combined with residual connections, flattening operations, and multilayer perceptron mapping, sample-level adaptive modality fusion weights are generated via Softmax. Simultaneously, a modality balance loss, i.e., the KL divergence between the weight distribution and a uniform distribution, is introduced to constrain the weight distribution of each modality to tend towards uniformity, preventing the router from collapsing into a single dominant modality and ensuring that information from each modality is effectively utilized in the fusion representation.

[0064] Experimental example: 1. Data Acquisition and Preprocessing: This experiment uses three publicly available multimodal re-identification datasets: RGBNT201 is a multimodal pedestrian re-identification dataset containing 4787 aligned RGB, NIR, and TIR images for 201 identities; RGBNT100 is a large-scale multimodal vehicle re-identification dataset containing 17250 image triplets, covering a wide range of challenging visual conditions, suitable for evaluating the robustness of multimodal vehicle re-identification; MSVR310 contains 2087 high-quality multimodal vehicle re-identification images collected under different environmental conditions and time spans.

[0065] For data preprocessing, pedestrian images were uniformly scaled to 256×128 pixels, and vehicle images were uniformly scaled to 128×256 pixels. Data augmentation strategies included random horizontal flipping, random cropping, and random erasing. Training parameters were set as follows: for the RGBNT201 and MSVR310 datasets, the batch size was set to 64, with 8 images sampled per identity; for the RGBNT100 dataset, the batch size was set to 128, with 16 images sampled per identity.

[0066] 2. Model Building: Backbone: Uses a pre-trained CLIP ViT-B / 16 as a shared visual encoder.

[0067] Phase 1 configuration is as follows: Optimizer is Adam; learning rate is encoder. Classification Head Hyperparameters are (PRD weight) (GSD weights); the training epochs are 50 epochs long, and the Cosine Decay learning rate is used for scheduling.

[0068] Phase 2 configuration is as follows: Freeze the ViT encoder; learning rate is [missing information]. Hyperparameters are (Modal balance weights); Training epochs: 10.

[0069] 3. Analysis of experimental results: This invention uses mean precision (mAP) and cumulative matching features (CMC) at Rank-K (K=1, 5, 10) for evaluation.

[0070] Table 1. Comparison with existing state-of-the-art methods on the RGBNT201 dataset. The proposed RDAF outperforms the compared methods, achieving the best results in mAP and Rank-1. Specifically, RDAF has an mAP of 83.0% and a Rank-1 accuracy of 86.1%, which is 2.7% and 0.9% higher than the state-of-the-art method Signal

[24] , respectively. In addition, compared with DeMo, IDEA and 2-ReID, RDAF improves mAP by 4.0%, 2.8% and 2.7%, respectively, and increases Rank-1 accuracy by 3.8%, 4.0% and 1.3%, respectively. These results demonstrate the effectiveness of the proposed RDAF framework.

[0071] Table 2 Comparison with existing state-of-the-art methods on the RGBNT100 and MSVR301 datasets. This invention also compares the proposed method with existing methods on two multimodal vehicle re-identification datasets (RGBNT100 and MSVR310). RDAF achieves the highest mAP (88.5%) while maintaining a competitive Rank-1 accuracy on RGBNT100. Compared to DeMo, IDEA, SD2-ReID, and Signal, RDAF improves mAP by 2.3%, 1.3%, 4.1%, and 2.2%, respectively. For the highly challenging MSVR310 dataset, RDAF's advantage is even more pronounced. RDAF achieves 56.1% mAP and 74.6% Rank-1 accuracy. Compared to DeMo, IDEA, SD2-ReID, and Signal, RDAF improves mAP by 6.9%, 9.1%, 8.3%, and 2.5%, respectively, and Rank-1 accuracy by 14.8%, 12.2%, 10.5%, and 2.7%, respectively. These results demonstrate the effectiveness of the proposed RDAF framework.

[0072] Table 3. Comparison of single-modal performance with existing state-of-the-art methods on the RGBNT201 dataset. On the RGBNT201 dataset, NIR significantly underperforms compared to RGB and TIR, and is therefore considered a weaker modality. RDAF achieves the best average single-modality performance, with a 1.8% improvement in mAP and a 3.5% improvement in Rank-1 accuracy compared to DeMo. Notably, RDAF achieves a significant improvement in the NIR modality, exceeding DeMo by 6.5% in mAP and 9.5% in Rank-1 accuracy, significantly improving the performance of weaker modalities. DeMo and K2-ReID perform slightly better than RDAF on TIR and RGB, respectively, because they allocate more optimization resources to TIR and RGB during training, resulting in over-optimization of TIR and RGB and under-optimization of NIR.

[0073] Table 4. Comparison of unimodal performance with existing state-of-the-art methods on the RGBNT100 dataset. On the RGBNT100 dataset, TIR becomes a weaker modality. RDAF achieves the best average single-modal performance, with a 2.7% improvement in mAP compared to DeMo. More importantly, in the TIR modality, RDAF improves mAP by 4.2% compared to DeMo, significantly enhancing its discriminative ability. These experimental results demonstrate that the proposed RDAF framework can sufficiently enhance single-modal representations, thereby enabling more powerful multimodal learning.

[0074] Table 5 Efficiency Analysis RDAF requires only 88.1M parameters and 33.3G of floating-point operations, achieving relatively low model complexity and the lowest number of floating-point operations among all compared methods. Despite this lightweight design, RDAF still achieves the best performance with an mAP of 83.0% and a Rank-1 accuracy of 86.1%. This result demonstrates that RDAF achieves an excellent balance between performance and efficiency.

[0075] The relation distillation-guided single-modality and adaptive fusion re-identification method of this invention has the following advantages: 1. This invention effectively improves multimodal retrieval performance. Compared to existing methods, this invention achieves the best multimodal retrieval performance on three public datasets: RGBNT201, RGBNT100, and MSVR310, with mAP and Rank-1 accuracy superior to all comparable methods. 2. This invention effectively alleviates modal inertia and improves weak modality discrimination. Through first-stage relation distillation (including pairwise relation distillation and global structure distillation), this invention effectively transfers multimodal complementary knowledge to each single-modal branch, significantly enhancing the representation quality of weak modalities. Experiments show that compared to state-of-the-art methods, this invention significantly improves mAP and Rank-1 accuracy on weak modalities, achieving optimal average performance on single modalities. 3. This invention is lightweight and efficient. This invention requires only 88.1M parameters and 33.3G of floating-point operations, achieving the lowest number of floating-point operations among all comparable methods while maintaining relatively low model complexity. Despite its lightweight design, this invention still achieves optimal multimodal retrieval performance, achieving an excellent balance between performance and efficiency.

[0076] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A single-modal and adaptive fusion re-identification method based on relationship distillation guidance, characterized in that, Includes the following steps: Step S1: Obtain a small batch of data containing multiple modal images, extract the features of each unimodal student using a shared-weight visual encoder, and concatenate the features of each unimodal student to construct online multimodal teacher features; Step S2: Based on the unimodal student features and the multimodal teacher features, calculate the pairwise distillation loss for aligning local metric relations and the global structural distillation loss for aligning global metric topology, respectively. Step S3: Combine the preset task loss, the pairwise distillation loss, and the global structure distillation loss to perform end-to-end training on the visual encoder to obtain enhanced single-modal student features; Step S4: Freeze the parameters of the trained visual encoder and treat the enhanced unimodal student features as multiple independent expert features; Step S5: Stack the expert features and input them into the modality-aware router. Model the intermodal relationships through a self-attention mechanism to generate a set of adaptive modality fusion weights for the current sample. Step S6: Use the adaptive modal fusion weights to perform weighted fusion on the corresponding expert features to obtain multimodal fusion features; Step S7: Calculate the task loss based on the multimodal fusion features, and introduce a modal balance loss to prevent modal collapse. Optimize the parameters of the modal-aware router to obtain the final multimodal fusion features for object re-identification retrieval.

2. The single-modality and adaptive fusion re-identification method based on relationship distillation guidance according to claim 1, characterized in that, In step S1, the construction of online multimodal teacher features specifically involves concatenating the CLS token features of each modality corresponding to each sample within the same mini-batch of data along the feature dimension. This concatenation operation is performed synchronously with the forward propagation of the visual encoder, without requiring a pre-trained independent teacher model.

3. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S2, the pairwise distillation loss is calculated as follows: the cosine distance matrix between the multimodal teacher features and the unimodal student features in the mini-batch samples is calculated respectively, and then the mean square error between the two distance matrices is minimized.

4. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S2, the global structural distillation loss is calculated as follows: the Gromov-Wasserstein distance is introduced to measure the internal structural difference between the first metric space composed of the multimodal teacher features and the second metric space composed of the unimodal student features, and the optimal transmission plan is solved by the Sinkhorn-Knopp algorithm to calculate the loss.

5. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S3, the task loss includes: label smoothing cross-entropy loss for identity classification and triplet loss for metric learning.

6. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S5, the modality-aware router specifically includes: A multi-head self-attention layer is used to process the stacked expert features and output attention-enhanced features; Residual connections are used to add the stacked expert features to the attention-enhanced features; Flattening operation is used to flatten the output of residual connection into one dimension; Multilayer perceptron is used to map the results of flattening operations to unnormalized modal weights; The Softmax function is used to transform the unnormalized modal weights into normalized adaptive modal fusion weights.

7. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S6, the weighted fusion specifically involves multiplying the expert features of each modality by the adaptive modality fusion weight corresponding to that modality, and then concatenating them along the feature dimension to obtain the multimodal fusion features.

8. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S7, the modal balance loss specifically involves calculating the Kullback-Leibler divergence between the distribution of the adaptive modal fusion weights and the preset uniform distribution, which is used to constrain the weight distribution of each mode to tend towards uniformity and avoid the fusion process from relying solely on the dominant mode.

9. The single-modal and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S1, the visual encoder is a pre-trained CLIP ViT backbone network, and the same shared weights are used for the three modes of RGB, near-infrared and thermal infrared.

10. The single-modality and adaptive fusion re-identification method based on relation distillation guidance according to claim 1, characterized in that, In step S3, the pairwise distillation loss and the global structure distillation loss are multiplied by the first hyperparameter and the second hyperparameter, respectively, and then added to the task loss for optimization. In step S7, the modal balance loss is multiplied by the third hyperparameter and then added to the task loss for optimization.

Images