A texture recovery driven low-light cross-modal pedestrian re-identification model and method

Abstract

This invention discloses a texture restoration-driven low-light cross-modal person re-identification model and method, relating to the field of computer vision technology, and solving the technical problems of modal differences and semantic shifts under low-light conditions. The invention includes a spectral alignment module that performs pseudo-infrared processing and spectral compensation on the human image region in the input image to obtain a pseudo-infrared enhanced visible light image; a semantic feature mining module that extracts visual features from the pseudo-infrared enhanced visible light image and the real infrared image respectively, and cross-attention the enhanced visible light features and infrared features; a frequency-guided shallow enhancement module that uses the enhanced visible light features and infrared features as shallow features and performs controllable redistribution of the energy spectrum in the frequency domain; and a multi-layer feature fusion module that maps the fused features to a common embedding space for matching. This invention can significantly improve cross-modal recognition performance while maintaining structural interpretability and model compactness.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a texture restoration-driven low-light cross-modal pedestrian re-identification model and method. Background Technology

[0002] Pedestrian re-identification is a key technology that uses computer vision to identify specific pedestrians in images or video sequences, and it has significant value in modern intelligent security systems. In nighttime security scenarios, visible light cameras struggle to acquire effective identification features, easily leading to target loss and trajectory breaks, rendering continuous tracking and post-event retrieval ineffective, and posing serious public safety hazards. Meanwhile, although infrared cameras can provide stable imaging, modal differences make it difficult for traditional re-identification methods to identify target pedestrians in cross-modal scenarios.

[0003] With the increasing demand for 24 / 7 continuous security in city-level surveillance systems, constructing a cross-modal pedestrian re-identification framework suitable for continuous security has become a key problem to be solved. Although infrared-visible light pedestrian re-identification has made significant progress in recent years, its performance is still limited by problems such as modal differences and feature degradation. Existing methods can be broadly divided into two categories: generative methods and non-generative methods. The former usually uses generative adversarial networks (GANs) or diffusion models to transform infrared and visible light images into a unified modal space to bridge cross-modal differences at the image level.

[0004] However, generative adversarial networks (GANs) often struggle to overcome generation noise caused by artifacts, leading to unstable reconstruction results. While diffusion models offer stronger generation quality and modality consistency, their large model size and high computational cost limit their application in security scenarios. In contrast, non-generative methods primarily utilize images from both modalities directly at the feature level for cross-modal alignment by designing novel network structures and metric learning strategies. These methods rely on learning discriminative features, but because the training process is typically based only on image-level supervision and lacks high-level semantic constraints, modality alignment in the common feature space remains difficult, limiting recognition accuracy.

[0005] In recent years, with the development of visual language training models, they have demonstrated strong generalization capabilities in cross-modal tasks and achieved initial success in downstream tasks such as pedestrian re-identification. However, for infrared-visible light pedestrian re-identification tasks, although the consistency between prompts and images can be guaranteed during the training phase, in actual security monitoring, the language descriptions provided by witnesses are often vague or biased, thus affecting the reliability of the recognition results. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a texture restoration-driven low-light cross-modal pedestrian re-identification model and method. This method solves the technical problem of modal differences caused by the different imaging mechanisms of visible light and infrared light under low-light conditions. These differences result in the color, texture, edge, and local response of a target being different in the two modalities, leading to semantic instability in subsequent processing and preventing consistent semantic representation of the same target. A texture restoration-driven low-light cross-modal pedestrian re-identification model includes:

[0007] The spectral alignment module performs pseudo-infrared processing and spectral compensation on the human image region in the input image to obtain a pseudo-infrared enhanced visible light image.

[0008] The semantic feature mining module extracts visual features from the pseudo-infrared enhanced visible light image and the real infrared image respectively, and then obtains visible light enhancement features and infrared enhancement features through cross attention to achieve information interaction;

[0009] The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features. By controlling the redistribution of the energy spectrum in the frequency domain, it achieves detail enhancement and noise suppression while preserving shallow features.

[0010] The multi-layer feature fusion module performs hierarchical fusion of the frequency domain enhanced features, and then maps them to the common embedding space for matching to obtain the recognition result.

[0011] A texture restoration-driven method for training a low-light cross-modal person re-identification model includes:

[0012] A lightweight joint objective function is designed, which is composed of a weighted average batch contrast loss and a modality consistency alignment loss, to simultaneously constrain semantic consistency and distribution consistency. The average batch contrast loss constructs a stable contrast relationship by using the modality mean of the same identity within a batch to improve the cross-modal semantic alignment effect. The modality consistency alignment loss aligns modal statistical features at the distribution level and suppresses modality shift by minimizing the difference between the mean and covariance.

[0013] The overall optimization objective is to integrate spectral, semantic, structural, and identity constraints, while also considering cross-modal matching, semantic alignment, identity discrimination, and distribution consistency. This includes cross-modal matching loss to enhance the correspondence between pseudo-infrared and infrared features, batch-average contrast loss to constrain semantic layer consistency, identity classification loss to maintain discriminative ability, and modality consistency alignment loss to statistically align the two modal distributions. The weights of batch-average contrast loss and modality consistency alignment loss are also retained, and the strength of semantic alignment and distribution consistency constraints is adjusted.

[0014] A texture restoration-driven low-light cross-modal pedestrian re-identification model recognition method includes:

[0015] Step 1: Given an input image, the spectral alignment module calculates a pseudo-infrared brightness map within the human image area and performs spectral compensation to obtain a pseudo-infrared enhanced visible light image;

[0016] Step 2: The semantic feature mining module uses a feature extractor to extract visible visual representations and infrared visual representations from the pseudo-infrared enhanced visible light image and the real infrared image, respectively; then, through cross attention, the information interaction between the visible visual representations and the infrared visual representations is realized to obtain visible light enhancement features and infrared enhancement features that fuse heteromodal semantic information, respectively.

[0017] Step 3: The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features, generates channel adaptive gain through a gating mechanism, adjusts the shallow features using the channel adaptive gain, and then enhances the adjusted features in the frequency domain.

[0018] Step 4: Interact with the enhanced features to strengthen the features that are important for recognition, and then map them to the common embedding space for matching to obtain the recognition result.

[0019] Furthermore, the frequency domain enhancement includes:

[0020] The adjusted features are mapped to the frequency domain, and the amplitude and phase of the spectrum are calculated.

[0021] Based on the normalized distance from the spectral point to the center, a smooth high-pass or band-pass mask is constructed for weighted modulation of different frequency components in the amplitude spectrum.

[0022] The enhanced features are then recovered through inverse transformation, and the final enhanced features are obtained by fusing the shallow features and the enhanced features through residual fusion.

[0023] Furthermore, the hierarchical fusion of the frequency-domain enhanced features includes:

[0024] In the low-dimensional stage, the corresponding low-dimensional features of the visible light branch and the infrared branch are fused to obtain a low-dimensional fused representation with enhanced details. The low-dimensional features include edges, textures and local detail information.

[0025] In the mid-dimensional stage, the mid-dimensional features of the two modalities are fused to model the local regions of the human body, the contour structure, and the correspondence between regions, resulting in a mid-level fused representation with stronger structural consistency.

[0026] In the high-dimensional stage, the high-dimensional features of the two modalities are fused, and identity semantics and global discriminative information are aggregated to obtain a high-level fusion representation with stronger semantic consistency.

[0027] Finally, the fusion results of low-dimensional, medium-dimensional, and high-dimensional data are jointly aggregated to form a final shared representation that takes into account details, structure, and semantics.

[0028] Furthermore, the joint aggregation includes hierarchical feature compression, channel-level splicing, adaptive weight allocation, and residual normalization fusion.

[0029] The beneficial effects of this invention include:

[0030] To address the significant modal differences between visible and infrared light in low-light scenarios, a network framework driven by detail recovery for cross-modal alignment is proposed. The model first achieves a perceptual transition from the visible to the infrared domain through spectral consistency reconstruction at the input stage, mitigating spectral discrepancies. Then, SCM (Semantic Cohesion) is introduced at the semantic layer, utilizing cross-attention to enhance semantic alignment and complementarity between the two modalities. At the structural layer, frequency-guided shallow enhancement (FGSE) is employed to recover texture and smooth energy distribution. Finally, a multi-layer feature fusion module combined with a lightweight loss function design achieves globally consistent representation optimization. Experimental results on the LLCM low-light dataset and SYSU-MM01 demonstrate that this method significantly improves cross-modal recognition performance while maintaining structural interpretability and model compactness, validating the effectiveness of the design concept of using spectral reconstruction to promote semantic alignment and frequency domain enhancement to facilitate global fusion. Attached Figure Description

[0031] Figure 1 This is an architecture diagram of a texture restoration-driven low-light cross-modal pedestrian re-identification model according to an embodiment of this application.

[0032] Figure 2 This is a comparison diagram of attention effects involved in the embodiments of this application; Figure 2 (a) is the original low-light visible light input image. Figure 2 (b) The attention visualization results of the recognition method proposed in this embodiment. Figure 2 (c) shows the attention results for the baseline method.

[0033] Figure 3 This is a visualization of the feature distribution involved in the embodiments of this application.

[0034] Figure 4 This is a visualization result of cross-modal retrieval in an extreme low-light scene, as described in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0036] The following is in conjunction with the appendix Figures 1-4 Specific embodiments of the present invention will be described in detail;

[0037] In low-light visible-infrared pedestrian re-identification tasks, performance degradation does not stem from a single modality difference, but rather from the combined effects of inconsistent spectral distributions, semantic representation shifts, and structural detail degradation across different representation levels, resulting in a mismatch from the input domain to the embedding domain. Spectral shifts in the input layer affect early feature responses, thus weakening the stability of mid-level semantic alignment; simultaneously, low-light noise and texture loss further exacerbate cross-modal distribution drift. Existing methods often perform local compensation at a single level, lacking a consistent modeling path that spans multiple representations, making it difficult to achieve overall optimization between semantic consistency and distribution consistency.

[0038] To address this, this embodiment proposes a texture restoration-driven low-light visible-infrared re-identification network, constructing a hierarchical consistency framework encompassing spectral alignment, semantic complementarity, structural enhancement, and fusion optimization. First, local pseudo-infraredization is implemented at the input layer to alleviate the spectral shift at the lower layers. Then, a lightweight semantic interaction mechanism is introduced at the token layer to stabilize the alignment at the middle layers. Further, frequency-domain guided structural enhancement restores texture details and suppresses noise. Finally, joint loss constraints are applied in the common embedding space to achieve collaborative convergence of cross-modal features at both the semantic and statistical levels. Based on this hierarchical consistency approach, various implementation paths can be formed, but this embodiment primarily adopts a PIM–SCM–FGSE combination as a lightweight implementation that achieves a good balance between complexity and performance.

[0039] A texture restoration-driven low-light cross-modal person re-identification model, such as Figure 1 As shown, it includes:

[0040] The spectral alignment module performs pseudo-infrared processing and spectral compensation on the human image region in the input image to obtain a pseudo-infrared enhanced visible light image.

[0041] The semantic feature mining module uses a visual feature encoder to extract visual features from pseudo-infrared enhanced visible light images and real infrared images respectively, and then uses cross attention to achieve information interaction to obtain visible light enhancement features and infrared enhancement features.

[0042] The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features. By controlling the redistribution of the energy spectrum in the frequency domain, it achieves detail enhancement and noise suppression while preserving shallow features.

[0043] The multi-layer feature fusion module interacts with the features enhanced by the frequency-guided shallow enhancement module and maps them to the common embedding space for matching.

[0044] Specifically, such as Figure 1 As shown, the model input includes low-light visible light images and infrared images. The two types of images are first processed by the image preprocessing module PIM for uniform size adjustment, normalization and low-light feature enhancement preprocessing to reduce the impact of low-light degradation, noise interference and modal distribution differences on subsequent feature extraction.

[0045] Preprocessed visible light and infrared images are input into the visible light image encoder and infrared image encoder, respectively, to obtain corresponding modal feature representations. The output features of the visible light branch are denoted as Ev, and the output features of the infrared branch are denoted as Ei. The two encoders employ a parallel structure to extract local texture, human structure, and identity semantic features in the visible light and infrared modalities, respectively. Due to insufficient brightness, texture degradation, and enhanced background noise in low-light visible light images, relying solely on shared feature extraction can easily lead to cross-modal identity representation shifts. Therefore, a semantic feature mining module and a frequency-guided shallow enhancement module are further designed to enhance the consistency and discriminative power of cross-modal features. In the backbone network, Ev and Ei are first input into the semantic feature mining module (SCM). This module explicitly models identity-related regions in different modalities by constructing the correlation between visible light and infrared features, thereby mining shared semantic information between modalities. Specifically, SCM establishes cross-modal attention relationships by query vector Q, key vector K, and value vector V. It then uses the Softmax operation to obtain inter-modal semantic response weights and recalibrates features based on these weights, making the network pay more attention to regions with identity-discriminating significance, such as human contours, clothing structure, and local region relationships. After semantic feature mining, the high-level semantic representations of the visible light and infrared modalities are further aligned, thereby mitigating the semantic shift problem caused by differences in imaging mechanisms.

[0046] Simultaneously, the model introduces a Frequency-Guided Shallow Enhancement (FGSE) module to strengthen shallow detail features in low-light visible light images. This module first performs a frequency domain transformation on the input features to obtain their frequency response representation, and then constructs high-frequency and low-frequency masks respectively. The high-frequency mask highlights edges, textures, and local details, while the low-frequency mask preserves the subject's outline, brightness distribution, and global structural information. Through the synergistic enhancement of high-frequency and low-frequency components, FGSE effectively improves the weakened edge texture and human structure representation in low-light images. The enhanced shallow features are further fused with modal features extracted by the backbone network, enabling the model to simultaneously possess detail recovery capabilities and cross-modal semantic alignment capabilities.

[0047] Subsequently, the features processed by SCM and FGSE enter a multi-layer feature fusion module. This module fuses the features of the visible light branch and the infrared branch at shallow, medium, and deep layers, respectively. Shallow fusion is mainly used to compensate for the loss of edge, texture, and local response information under low-light conditions; medium-layer fusion is mainly used to model local human regions, structural contours, and cross-regional correspondences; deep fusion is mainly used to constrain the consistency of identity semantic features across different modalities. Through this multi-layer fusion approach, the model can progressively reduce the representational differences between the visible light and infrared modalities at three levels: local details, human structure, and identity semantics.

[0048] Finally, the fused features are fed into the identity discrimination and metric learning constraint part. The model optimizes the model through multiple loss functions, including identity classification loss, center constraint loss, cross-modal matching loss, and soft-center triplet loss, further reducing the feature distance of the same identity in different modalities and further increasing the feature distance between different identities. Therefore, this invention can achieve more stable cross-modal identity association in low-light security monitoring scenarios, improving the matching accuracy and robustness between low-light visible light images and infrared images.

[0049] In another embodiment, a texture restoration-driven low-light cross-modal pedestrian re-identification model training method is provided, comprising:

[0050] Design a lightweight joint objective function consisting of batch-average contrast loss and modality consistency alignment loss to simultaneously constrain semantic consistency and distribution consistency;

[0051] By constructing stable contrast relationships using the modal mean of the same identity within a batch, cross-modal semantic alignment can be improved.

[0052]

[0053] in and These represent the batch-average characteristics of the visible light and infrared modes, respectively. The normalization factor is taken as 0.1 in the experiment.

[0054] MCA aligns modal statistical features at the distribution level, suppressing modal shift by minimizing the difference between the mean and covariance.

[0055]

[0056] in Let represent the mean vectors of the two modal features, respectively. Here is the corresponding covariance matrix. and Let represent the Euclidean norm and the Frobenius norm, respectively. The joint optimization objective is defined as:

[0057]

[0058] in The main loss weights are used to maintain the stability of semantic alignment. The strength of modal consistency constraints is controlled. Both factors work together to balance the optimization relationship between cross-modal semantic consistency and distributional consistency.

[0059] The objective function has a simple structure, few parameters, and low computational cost, making it suitable for low-light visible-infrared re-identification tasks.

[0060] After completing spectral compensation, semantic complementarity, and structural enhancement, cross-modal features have gradually stabilized at the representation level. However, even with improved intermediate representations, cross-modal semantic boundary drift and statistical distribution shift may still occur without explicit constraints at the embedding space level. Therefore, in the final stage of the hierarchical consistency path, joint optimization constraints need to be imposed on the common embedding space to achieve coordinated convergence of semantic consistency and distribution consistency.

[0061] Overall optimization goal:

[0062] By integrating spectral, semantic, structural, and identity constraints, the network's training objective simultaneously considers cross-modal matching, semantic alignment, identity discrimination, and distribution consistency, with cross-modal matching loss being a key component. Batch-average contrast loss used to enhance the correspondence between pseudo-infrared and infrared features. Constraining semantic layer consistency, identity classification loss Maintaining discriminative ability, modal consistency alignment loss Aligning the two modal distributions at the statistical level and The semantic alignment and distribution consistency constraints are adjusted separately. The final total loss function is defined as:

[0063]

[0064] As the final stage of the hierarchical consistency path, the joint objective function imposes global constraints on cross-modal features at the embedding space level, so that the representational advantages obtained by the aforementioned spectral compensation, semantic complementarity and structural enhancement are stabilized and solidified during the training process, and finally achieves the collaborative convergence of cross-modal features.

[0065] In another embodiment, a texture restoration-driven low-light cross-modal pedestrian re-identification method is provided, comprising the following steps:

[0066] Step 1: Given the input image The spectral alignment module calculates the pseudo-infrared brightness map within the portrait mask M:

[0067]

[0068] The part in parentheses represents the perceptual brightness-weighted grayscale transformation, and CLAHE (·) enhances the local contrast. Subsequently, the converted infrared image replaces the RGB region within the mask M in the form of a three-channel grayscale image, while the background retains its original color.

[0069] In the hierarchical consistency framework, the spectral stability of the input layer is fundamental to subsequent semantic modeling and structural enhancement. If there is a significant spectral distribution shift between visible light and infrared images at the input stage, this difference will be amplified layer by layer during feature extraction, leading to inconsistent low-level responses across modalities and thus affecting the alignment of mid-to-high-level representations. Therefore, a local spectral compensation strategy is first implemented at the input layer to alleviate the low-level distribution mismatch problem across modalities.

[0070] Then, spectral compensation is performed on the visible light mode of the pseudo-infrared brightness map to obtain a pseudo-infrared enhanced visible light image, making its statistical distribution closer to that of the real infrared image, thereby reducing the response differences of cross-modal low-level features. This strategy is not a simple image enhancement, but rather the first constraint step in the hierarchical consistency path, providing a stable feature foundation for subsequent semantic complementarity and structural enhancement.

[0071] Step 2: The semantic feature mining module uses a feature extractor to extract features from pseudo-infrared enhanced visible light images. Compared with real infrared images Extracting visual token representations and ;

[0072]

[0073] In the formula, CLIP stands for CLIP semantic encoding function, which is used to extract high-level semantic features from input images or image features to obtain feature representations with category semantics and identity discrimination information; where CLIP is a contrastive language-image pre-trained model.

[0074] Subsequently, cross-attention is used to achieve information interaction, resulting in visible light enhancement features and infrared enhancement features that respectively fuse heteromodal semantic information, as expressed below:

[0075]

[0076] In the formula, Indicates visible light enhancement characteristics, The infrared enhancement features are represented by Q, K, and V, which are linear projection matrices.

[0077] After input layer spectral compensation, the two modalities have stabilized at the bottom layer distribution. However, due to differences in imaging mechanisms and the influence of low-light environments, structural shifts still exist between visible and infrared images at the semantic representation level. For example, weakened local textures in low-light visible images may lead to insufficient expression of target boundaries and identity-related features, while infrared images, although possessing relatively stable contour information, lack rich appearance details. Therefore, relying solely on a shared feature extractor is insufficient to achieve explicit semantic alignment; a cross-modal information interaction mechanism needs to be established at the mid-level representation stage.

[0078] Unlike cross-modal alignment methods that rely on text prompts, SCM is a text-free structure, avoiding the instability caused by semantic drift of text prompts in low-light scenes. By establishing explicit interaction relationships at the token layer, SCM achieves semantic alignment and complementarity with lower computational complexity, ensuring that the features of the two modalities have a more consistent semantic expression before entering the shallow structure enhancement stage. As the second stage in the hierarchical consistency path, SCM inherits the spectral compensation results from the input layer and stabilizes the semantic distribution in the middle-layer representation space, providing a more consistent semantic foundation for subsequent structure enhancement and embedding alignment.

[0079] Step 3: The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features. An SE-like gating mechanism is used to generate adaptive channel gain, which is used to dynamically control the enhancement magnitude of each channel.

[0080]

[0081] In the formula, ,in , , , These represent batch size, number of channels, and feature size, respectively. For global average pooling, The Sigmoid activation function is used. , for Convolution mapping.

[0082] Subsequently, channel adaptive gain adjustment of shallow features was used. Then, the adjusted features are mapped to the frequency domain:

[0083]

[0084] In the formula, This represents a two-dimensional Fast Fourier Transform. Used to move low-frequency components to the center position.

[0085] Calculate the amplitude and phase spectra: based on the normalized distance from the spectral points to the center. ,Will and These represent the amplitude and phase of the spectrum, respectively.

[0086] Constructing smooth high-pass or band-pass masks This is used to weight and modulate different frequency components, thereby enhancing the mid-to-high frequency texture of the amplitude spectrum and suppressing high frequency noise; specifically, it can be expressed as:

[0087]

[0088] Here, ⊙ represents element-wise multiplication. is the learnable global gain coefficient.

[0089] The above operations enable frequency-channel joint modeling, achieving interpretable energy redistribution in the frequency domain.

[0090] The enhanced features are then recovered using an inverse transform:

[0091]

[0092] and with learnable coefficients Controlling residual fusion yields the final enhanced features:

[0093]

[0094] In the formula, For learnable fusion weights.

[0095] By modulating only the amplitude spectrum, frequency-explicit structure preservation enhancement was achieved, enabling the model to have stronger texture recovery, modal consistency, and optimization stability under low-light conditions.

[0096] After spectral compensation and semantic complementation, the two modal features have become consistent in distribution and semantics. However, the degradation of structural details in low-light environments has not been completely eliminated. Specifically, low-light visible light images often exhibit texture blurring and reduced local contrast, while infrared images, although structurally stable, lack rich high-frequency appearance information. This imbalance in structural layers weakens the expression of identity-related details and amplifies cross-modal discrimination errors during the embedding stage. Therefore, in the third stage of the hierarchical consistency path, it is necessary to explicitly stabilize and enhance shallow structural features. To this end, this embodiment uses a frequency-guided shallow enhancement module, namely, a frequency-domain guided shallow enhancement module (FGSE), which achieves detail enhancement and noise suppression while preserving structure by controllably redistributing the energy spectrum in the frequency domain. Unlike traditional convolution, which directly superimposes features in the spatial domain, FGSE only modulates the amplitude spectrum in the frequency domain while maintaining the phase information, thus theoretically ensuring that the structural morphology is not destroyed.

[0097] As the third stage of the hierarchical consistency path, FGSE further stabilizes the structural representation based on semantic alignment, reduces the high-frequency energy imbalance caused by low-light degradation, and provides more consistent structural feature support for the convergence of cross-modal distribution in the final embedding space.

[0098] Step 4: After completing spectral alignment, semantic complementarity and structural enhancement, the frequency domain enhanced features are fused in layers and then mapped to the common embedding space for matching to obtain the recognition result.

[0099] The step of performing hierarchical fusion of the frequency-domain enhanced features includes:

[0100] In the low-dimensional stage, the corresponding low-dimensional features of the visible light branch and the infrared branch are fused to obtain a low-dimensional fused representation with enhanced details. The low-dimensional features include edges, textures and local detail information.

[0101] In the mid-dimensional stage, the mid-dimensional features of the two modalities are fused to model the local regions of the human body, the contour structure, and the correspondence between regions, resulting in a mid-level fused representation with stronger structural consistency.

[0102] In the high-dimensional stage, the high-dimensional features of the two modalities are fused, and identity semantics and global discriminative information are aggregated to obtain a high-level fusion representation with stronger semantic consistency.

[0103] Finally, the fusion results of low-dimensional, medium-dimensional, and high-dimensional data are jointly aggregated to form a final shared representation that takes into account details, structure, and semantics.

[0104] The joint aggregation includes hierarchical feature compression, channel-level concatenation, adaptive weight allocation, and residual normalization fusion. Specifically, low-dimensional, mid-dimensional, and high-dimensional fused features are adaptively pooled to obtain feature vectors at three scales. Then, they are concatenated along the channel dimension, and contribution coefficients at different levels are generated through fully connected mapping and attention weights. Finally, the three types of features are weighted and summed based on these contribution coefficients, and residual connections are made with the original high-level features to obtain the final shared representation. Through this joint aggregation method, the model can simultaneously utilize low-dimensional details, mid-dimensional structure, and high-dimensional semantic information, avoiding the dominance of single-level features in cross-modal matching results.

[0105] In another embodiment, experiments were conducted on two cross-modal person re-identification datasets: SYSU-MM01 and LLCM. SYSU-MM01 contains 34,167 images for 395 identities, and the evaluation uses both Indoor-Search and All-Search modes; LLCM was acquired by nine low-light cameras and contains 30,921 training images for 713 identities and 13,909 test images for 351 identities.

[0106] Implementation details

[0107] The DRM network proposed in this embodiment is implemented in PyTorch on an NVIDIA RTX 4090D GPU. The backbone network uses a pre-trained CLIP-ViT / B-16 fine-tuned structure, and the input images are uniformly adjusted to 3 × 384 × 144. Conventional data augmentation operations (random cropping, horizontal flipping, and color dithering, etc.) are applied during training. The batch size is set to 16, and the training lasts for 150 epochs. The optimizer used is AdamW, with an initial learning rate of... It decays in three stages at epochs 40, 80, and 120, eventually decreasing to 1 × 10⁻⁶. - 6. In the joint loss function, take... , Normalization factor .

[0108] 4.3 Main Results

[0109] Table 1 shows the performance comparison between the proposed method and mainstream VI-ReID methods on the LLCM and SYSU-MM01 datasets. In the more challenging low-light LLCM scene, the proposed network achieves R1 of 61.8% and 69.7% for IR-to-VIS and VIS-to-IR retrieval, respectively, and mAP of 70.4% and 69.4%, respectively, significantly outperforming traditional methods such as PDM and CN-GC that rely on color or contour enhancement. This demonstrates that pseudo-infrared transformation and FGSE frequency domain enhancement can effectively recover dark area textures and achieve robust cross-modal alignment. On the well-lit SYSU-MM01 dataset, the model still achieves high accuracy performance with R1 of 78.7% and 75.6% in All-search and Indoor-search modes, and mAP of 86.2% and 88.8%, respectively, demonstrating excellent feature extraction and recognition robustness under various lighting conditions.

[0110] Table 1 Performance comparison between the method in this embodiment and existing mainstream methods

[0111]

[0112] 4.4 Ablation Experiment

[0113] A. Module Contribution Analysis: To verify the effectiveness of each core module in low-light cross-modal recognition, stepwise ablation was performed on the most challenging IR-to-VIS retrieval task in the LLCM dataset. The results are shown in Table 2. Pseudo-infrared PIM significantly reduced the spectral difference, improving Rank-1 by more than 6%; the Frequency-Guided Shallow Enhancement (FGSE) module further recovered dark area texture details, increasing mAP to 65.9%; finally, after introducing the joint loss structure, the overall performance reached 61.8% / 69.7%, an improvement of 16.0% / 15.3% compared to the baseline model. The results demonstrate that the proposed "pseudo-infrared first, then frequency domain enhancement" strategy can effectively alleviate modal differences under low-light conditions and significantly improve recognition accuracy.

[0114] Table 2 Ablation contribution analysis of each module to the proposed network performance

[0115]

[0116] B. λ² weighted balance analysis

[0117] To further determine the optimal weight λ2 of the modality alignment term in the joint loss, a parameter scan was performed on the LLCM dataset (I To V), and the results are shown in Table 3. The model performance is optimal when λ2=0.3, with Rank-1 and mAP of 61.8% and 69.7%, respectively. When λ2 is too small (≤0.1), the modality constraint is insufficient, while when it is too large (≥0.5), it weakens the identity discrimination ability, indicating that it is crucial to appropriately balance identity consistency and modality alignment in low-light cross-modal tasks.

[0118] Table 3. Impact of different weight assignments of the objective function on the clustering accuracy of the model.

[0119]

[0120] C. Backbone Network Analysis

[0121] To verify the impact of the feature extraction backbone, ResNet-50 and the SCM proposed in this embodiment were compared as backbones under pseudo-infrared and frequency domain enhancement conditions, respectively. As shown in Table 4, the SCM model improved Rank-1 and mAP by 3.8% and 3.3% compared to ResNet, respectively. This indicates that VIT's cross-modal pre-trained features have stronger semantic extraction and modal alignment capabilities in low-light scenes, thus providing solid support for the overall performance improvement.

[0122] Table 4. Impact of backbone on network performance

[0123]

[0124] Experimental results show that the mapping enhancement, frequency domain enhancement, and semantic feature mining modules work together to effectively mitigate low-light modal differences. The proposed network improves the Rank-1 and mAP of the baseline model by 16.0% and 15.3%, respectively, in IR-to-VIS retrieval on the LLCM dataset, validating the significant advantages of the proposed strategy in low-light cross-modal recognition.

[0125] D. Robustness to Local Infrared Transformation. Robustness was evaluated in the most challenging LLCM (infrared to visible) setting by perturbing the binary mask M at inference time (without retraining) with boundary erosion / dilation (±3 pixels), spatial offset (5 pixels), and foreground dropout (30% and 50%). As shown in Table 5, performance remained stable under typical noise levels, only degrading significantly under severe mask damage, indicating robustness within the range of real-world perturbations.

[0126] Table 5. Robustness of local masks to noise

[0127]

[0128] 4.5 Visualization

[0129] Attention visualization. The results show that the method in this embodiment can more accurately focus on the main body area of ​​pedestrians in low-light environments, with attention responses concentrated on key areas (such as the head, shoulders, and body contours), and background responses significantly reduced. In contrast, the baseline method's attention distribution is more dispersed and easily affected by shadows, ground reflections, and noise, making it difficult for the model to capture effective identity-discriminating features.

[0130] This comparison shows that the frequency-domain guided shallow enhancement and cross-modal attention mechanism proposed in this embodiment plays a significant role in detail recovery and modal alignment, enabling the network to maintain clear and stable attention focus in low-light scenes, thereby improving the discriminability and robustness of cross-modal re-identification.

[0131] Figure 2 (a) is the original low-light visible light input image, (b) is the attention visualization result of the method proposed in this embodiment, and (c) is the attention result of the baseline method.

[0132] like Figure 3 The figure shows the feature distribution results for infrared retrieval of visible light under low-light conditions. The left figure shows the original feature distance, the middle figure shows the baseline method, and the right figure shows the complete model proposed in this embodiment.

[0133] As can be seen, the visible light and infrared samples in the original features are clearly separated in the embedding space, with severe modal shift, making it difficult to form effective clusters. The baseline method slightly improves the distribution structure through cross-modal alignment constraints, but cross-modal misalignment still exists for samples of the same type, resulting in insufficient aggregation. In contrast, the method in this embodiment, through the combined effect of pseudo-infrared completion and frequency domain guided enhancement, effectively restores the texture details and semantic consistency of low-light visible light, and achieves semantic layer alignment with the help of cross-modal attention, making samples of the same identity highly overlapping in different modalities, and the inter-class boundaries clearer. This result intuitively verifies the design idea of ​​"restore first, then align" in this embodiment, effectively alleviating the problem of excessive modal differences in low-light scenes.

[0134] Visualization of cross-modal retrieval in extremely low-light scenarios. Figure 4This paper demonstrates the cross-modal retrieval results of the proposed method on self-collected low-light scene samples. Infrared images are used to retrieve visible light pedestrian targets under low-light or even near-black conditions, and the Rank-10 results are displayed. Green boxes indicate successful matches, and red boxes indicate failed matches. It can be seen that even under extreme lighting conditions, complex background noise, and severe color distortion, the model can still accurately locate pedestrians of the same identity within the specified range. The feature responses are concentrated in the pedestrian's main body area rather than the distorted background. This is attributed to the detail recovery brought about by pseudo-infrared completion and frequency domain guided enhancement, which allows low-light visible light features to regain discriminative power at the semantic layer and achieves consistent embedding through cross-modal attention. This result further verifies the core idea of ​​this embodiment, "alignment driven by recovery": first recovering modal details, then achieving consistent cross-modal alignment, thereby maintaining stable cross-modal recognition performance in complex scenes.

[0135] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A texture restoration-driven low-light cross-modal pedestrian re-identification model, characterized in that, include: The spectral alignment module performs pseudo-infrared processing and spectral compensation on the human image region in the input image to obtain a pseudo-infrared enhanced visible light image. The semantic feature mining module extracts visual features from the pseudo-infrared enhanced visible light image and the real infrared image respectively, and then obtains visible light enhancement features and infrared enhancement features through cross attention to achieve information interaction; The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features. By controlling the redistribution of the energy spectrum in the frequency domain, it achieves detail enhancement and noise suppression while preserving shallow features. The multi-layer feature fusion module performs hierarchical fusion of the frequency domain enhanced features and then maps them to the common embedding space for matching.

2. A texture restoration-driven training method for a low-light cross-modal pedestrian re-identification model, characterized in that, Training a texture restoration-driven low-light cross-modal pedestrian re-identification model according to claim 1 includes: A lightweight joint objective function is designed, which is composed of a weighted average batch contrast loss and a modality consistency alignment loss, to simultaneously constrain semantic consistency and distribution consistency. The average batch contrast loss constructs a stable contrast relationship by using the modality mean of the same identity within a batch to improve the cross-modal semantic alignment effect. The modality consistency alignment loss aligns modal statistical features at the distribution level and suppresses modality shift by minimizing the difference between the mean and covariance. The overall optimization objective is to integrate spectral, semantic, structural, and identity constraints, while also considering cross-modal matching, semantic alignment, identity discrimination, and distribution consistency. This includes cross-modal matching loss to enhance the correspondence between pseudo-infrared and infrared features, batch-average contrast loss to constrain semantic layer consistency, identity classification loss to maintain discriminative ability, and modality consistency alignment loss to statistically align the two modal distributions. The weights of batch-average contrast loss and modality consistency alignment loss are also retained, and the strength of semantic alignment and distribution consistency constraints is adjusted.

3. A texture restoration-driven low-light cross-modal pedestrian re-identification model recognition method, characterized in that, include: Step 1: Given an input image, the spectral alignment module calculates a pseudo-infrared brightness map within the human image area and performs spectral compensation to obtain a pseudo-infrared enhanced visible light image; Step 2: The semantic feature mining module uses a feature extractor to extract visible visual representations and infrared visual representations from the pseudo-infrared enhanced visible light image and the real infrared image, respectively; then, through cross attention, the information interaction between the visible visual representations and the infrared visual representations is realized to obtain visible light enhancement features and infrared enhancement features that fuse heteromodal semantic information, respectively. Step 3: The frequency-guided shallow enhancement module uses visible light enhancement features and infrared enhancement features as shallow features, generates channel adaptive gain through a gating mechanism, adjusts the shallow features using the channel adaptive gain, and then enhances the adjusted features in the frequency domain. Step 4: Perform hierarchical fusion of the frequency domain enhanced features to strengthen the features that are important for recognition, and then map them to the common embedding space for matching to obtain the recognition result.

4. The texture restoration-driven low-light cross-modal pedestrian re-identification model recognition method according to claim 3, characterized in that, The frequency domain enhancement includes: The adjusted features are mapped to the frequency domain, and the amplitude and phase of the spectrum are calculated. Based on the normalized distance from the spectral point to the center, a smooth high-pass or band-pass mask is constructed for weighted modulation of different frequency components in the amplitude spectrum. The enhanced features are then recovered through inverse transformation, and the final enhanced features are obtained by fusing the shallow features and the enhanced features through residual fusion.

5. The texture restoration-driven low-light cross-modal pedestrian re-identification model recognition method according to claim 3, characterized in that, The step of performing hierarchical fusion of the frequency-domain enhanced features includes: In the low-dimensional stage, the corresponding low-dimensional features of the visible light branch and the infrared branch are fused to obtain a low-dimensional fused representation with enhanced details. The low-dimensional features include edges, textures and local detail information. In the mid-dimensional stage, the mid-dimensional features of the two modalities are fused to model the local regions of the human body, the contour structure, and the correspondence between regions, resulting in a mid-level fused representation with stronger structural consistency. In the high-dimensional stage, the high-dimensional features of the two modalities are fused, and identity semantics and global discriminative information are aggregated to obtain a high-level fusion representation with stronger semantic consistency. Finally, the fusion results of low-dimensional, medium-dimensional, and high-dimensional data are jointly aggregated to form a final shared representation that takes into account details, structure, and semantics.

6. The texture restoration-driven low-light cross-modal pedestrian re-identification model recognition method according to claim 5, characterized in that, The joint aggregation includes hierarchical feature compression, channel-level splicing, adaptive weight allocation, and residual normalization fusion.

Images