A cross-modal pedestrian re-identification method based on frequency-space decoupling enhancement

Abstract

This invention discloses a cross-modal pedestrian re-identification method and system based on frequency-space decoupling enhancement, belonging to the field of intelligent video surveillance technology. The method includes: acquiring visible light and infrared images; decomposing the input features into low-frequency sub-bands and multiple high-frequency sub-bands using discrete wavelet transform; utilizing the global structural stability of the low-frequency sub-bands, generating weight vectors through an adaptive channel attention mechanism to weight and enhance the high-frequency sub-bands, restoring degraded high-frequency detail information; extracting global structural features using a low-frequency filter, and performing cross-modal feature alignment using multi-scale convolution; fusing the enhanced frequency domain features and spatial domain features to generate a discriminative cross-modal pedestrian feature representation. This invention fully utilizes the structural stability of low-frequency information and the detail discrimination capability of high-frequency information, effectively solving the feature degradation problem caused by illumination changes, occlusion, and pose changes in cross-modal pedestrian re-identification, and significantly improving the pedestrian recognition accuracy in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of intelligent video surveillance technology, specifically relating to a cross-modal pedestrian re-identification method based on frequency-space decoupling enhancement. Background Technology

[0002] Person re-identification (ReID) is a core technology in intelligent video surveillance systems, aiming to identify the same pedestrian from images captured by different cameras at different times. Traditional single-modal person re-identification methods are mainly based on visible light cameras, achieving good results in well-lit scenes. However, in nighttime or low-light environments, visible light cameras struggle to capture clear pedestrian images, leading to a sharp decline in recognition performance.

[0003] To address the aforementioned issues, Visible-Infrared Person Re-Identification (VI-ReID) technology has emerged. This technology combines the complementary advantages of visible light cameras and infrared cameras to achieve all-weather, all-time pedestrian identification. Visible light cameras can capture rich color and texture information during the day, while infrared cameras can provide stable thermal imaging information at night or in low-light environments. However, due to the fundamental differences between visible light images and infrared images in terms of imaging principles and spectral characteristics, cross-modal matching faces significant challenges.

[0004] Existing cross-modal pedestrian re-identification methods mainly have the following problems: (1) Most existing methods rely only on spatial domain features, which are highly sensitive to changes in illumination and are difficult to recover degraded high-frequency details (such as texture and edges); (2) Existing methods ignore the structural stability of low-frequency components, while the global contour information encoded by low-frequency components is crucial for recovering discriminative details of identity; (3) In complex scenarios such as occlusion and pose changes, high-frequency responses are inconsistent between different modalities, making feature alignment difficult.

[0005] Frequency domain analysis has proven effective in image restoration and enhancement tasks, achieving more robust feature learning by separating structural and detail components. Frequency domain features can be decomposed into high-frequency and low-frequency components: high-frequency components correspond to fine-grained details such as texture and edge contours, which are prone to degradation under low light or high contrast conditions; low-frequency components represent global structural information and remain relatively stable across different modalities and conditions. Therefore, how to fully utilize the structural stability of low-frequency information to guide the restoration of high-frequency details and achieve effective cross-modal feature alignment is a key problem that current cross-modal person re-identification technologies urgently need to solve. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a cross-modal pedestrian re-identification method based on frequency-space decoupling enhancement. By jointly modeling frequency and spatial domain features, it fully utilizes the structural stability of low-frequency information to enhance high-frequency details, thereby achieving cross-modal pedestrian matching. The method includes the following steps:

[0007] S1. Acquire the visible light image and infrared image to be identified, wherein the visible light image and infrared image are images of the same pedestrian collected under different lighting conditions.

[0008] S2. The input features are decomposed into one low-frequency sub-band and three high-frequency sub-bands using Discrete Wavelet Transform (DWT). The low-frequency sub-band (approximation coefficient cA) mainly captures global structure and background information, while the high-frequency sub-bands (horizontal detail cH, vertical detail cV, and diagonal detail cD) represent finer details and texture information.

[0009] S3. Utilizing the structural stability of the low-frequency subband to guide the enhancement and recovery of high-frequency details. First, global average pooling (GAP) is performed on the low-frequency subband to generate channel weight vectors. Then, the channel weight vectors are used to adaptively weight the high-frequency subbands to enhance identity-discriminating high-frequency details while suppressing noise. Finally, the weighted high-frequency subbands are recombined with the original low-frequency subbands through inverse discrete wavelet transform (Inverse DWT) to obtain the enhanced frequency domain features.

[0010] S4. Low-frequency components are extracted from the original features using a low-frequency filter to capture cross-modal stable global structural features. Then, multi-scale convolution is used to process the low-frequency components and the original features separately. Three parallel 3×3 convolutional layers are used with dilatancy rates of 1, 2, and 3, respectively, to capture multi-level structural information from coarse to fine. Next, the multi-scale features are refined through channel attention and spatial attention mechanisms to dynamically suppress redundant signals and enhance informative components. Finally, a weighted fusion strategy is used to integrate the multi-branch feature outputs to optimize cross-modal information flow.

[0011] S5. The enhanced frequency domain features are fused with the low-frequency decoupled multi-scale aligned features to generate the final cross-modal pedestrian feature representation. Preferably, a residual connection method is used to add the enhanced features to the original input features element-wise, enriching feature diversity and promoting effective cross-modal alignment.

[0012] S6. Calculate the similarity between the query image and candidate images, and determine the pedestrian's identity based on the similarity ranking results. During the model training phase, a multi-loss joint optimization strategy is adopted to optimize model parameters and improve recognition performance.

[0013] The beneficial effects of this invention are:

[0014] 1. This invention innovatively proposes a low-frequency guided high-frequency enhancement mechanism, which makes full use of the structural stability of low-frequency information to guide the recovery of high-frequency details, effectively solving the problem of high-frequency information degradation under weak light or high contrast conditions, and significantly improving feature discrimination ability.

[0015] 2. This invention proposes a low-frequency decoupling multi-scale alignment mechanism, which achieves effective alignment of cross-modal features through low-frequency filtering and multi-scale convolution, and can still maintain a high recognition accuracy in complex scenarios such as occlusion and pose changes.

[0016] 3. This invention jointly models frequency domain and spatial domain features, achieving a balance between multi-level feature representation and rich shared semantics.

[0017] 4. The method of the present invention can be widely applied to scenarios such as intelligent security, smart transportation, and smart parks, realizing all-weather and all-time pedestrian identification and tracking, and has broad application prospects and important practical value. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the cross-modal pedestrian re-identification method.

[0019] Figure 2 This is a schematic diagram of the process of a cross-modal person re-identification method based on frequency-space decoupling enhancement according to the present invention;

[0020] Figure 3 This is a schematic diagram of the low-frequency guiding high-frequency enhancement module structure of the present invention;

[0021] Figure 4 This is a schematic diagram of the low-frequency decoupling multi-scale alignment module structure of the present invention; Specific implementation methods

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] A cross-modal person re-identification method based on frequency-space decoupling and enhancement networks, such as Figure 2As shown, the process includes: constructing a dual-stream ResNet-50 backbone network to process input images in both visible and infrared modalities; introducing a low-frequency guided high-frequency enhancement module (LF-GHFE) at a specific stage of the backbone network, which uses discrete wavelet transform to perform frequency decomposition on the feature map and enhances high-frequency details through a low-frequency information-guided attention mechanism; introducing a low-frequency decoupled multi-scale alignment module (LF-DMSA) in subsequent network stages, which extracts and enhances low-frequency global structural information shared between modalities through low-pass filtering and multi-scale convolution; and finally, fusing the enhanced frequency domain features with the original spatial domain features and optimizing them through a global average pooling layer and a loss function to achieve cross-modal pedestrian feature matching.

[0024] This embodiment proposes a cross-modal person re-identification method based on a Frequency-Spatial Decoupling and Enhancement Network (FSDE-Net), which improves upon traditional matching methods that rely solely on spatial domain features. The proposed low-frequency-guided high-frequency enhancement module aims to comprehensively utilize the global structure and detailed texture of the image, selecting the most discriminative high-frequency features through wavelet transform and adaptive masking mechanisms. Simultaneously, addressing the challenge of cross-modal feature alignment, the proposed low-frequency decoupling multi-scale alignment module extracts relatively stable low-frequency global information and utilizes multi-scale convolution for fine alignment, effectively narrowing the semantic gap between modalities. An innovative lightweight module integration method is designed, abandoning complex multi-stage feature post-processing, reducing computational overhead, and achieving efficient and robust person retrieval.

[0025] Specifically, such as Figure 3 As shown, the construction of the low-frequency guided high-frequency enhancement module specifically includes:

[0026] S1. Frequency Feature Decomposition. This invention introduces Discrete Wavelet Transform (DWT) for frequency decomposition. Based on DWT, it proposes to decompose the input spatial domain features. The image is decomposed into low-frequency approximate components and high-frequency detail components, which serve as the basis for subsequent enhancement processing. The logic behind this design is that by quantizing and decomposing the frequency components of the image, high-frequency details (such as edges and textures) that are easily damaged in cross-modal scenes can be enhanced in a targeted manner, while suppressing noise.

[0027] In this invention, Haar wavelets are used as wavelet basis functions for the input feature map. Perform single-layer decomposition. This process can be represented as:

[0028]

[0029] in These are low-frequency approximation components, mainly containing the contour and structural information of the image; These represent high-frequency detail components in the horizontal, vertical, and diagonal directions, respectively, and contain rich texture and edge information.

[0030] S2. Attention Weight Generation Based on Low-Frequency Context. To utilize stable low-frequency information to guide the enhancement of high-frequency features, this invention performs global average pooling on the low-frequency components to generate a weight vector that reflects the importance of the global context. This vector is then used to weight the high-frequency components to highlight key details and suppress redundant noise. The formula is as follows:

[0031]

[0032] in The sigmoid activation function is used to normalize the weights to the (0, 1) interval. The GAP operation aggregates global spatial information from low-frequency components, ensuring that the generated weights represent the importance of the entire image context.

[0033] S3. High-frequency component adaptive enhancement. This yields the weight vector. Then, element-wise multiplication is used to apply it to the three high-frequency components to achieve adaptive enhancement. This mechanism can dynamically improve the response of important high-frequency features while suppressing background noise unrelated to pedestrian identity.

[0034]

[0035] in Element-wise multiplication representing the channel dimension. It automatically expands to accommodate the spatial dimensions of high-frequency components according to the broadcast mechanism. Through this operation, high-frequency details that correspond to high-response regions in low-frequency components are preserved and amplified, while those that are suppressed are not.

[0036] S4. Feature Reconstruction and Fusion. Finally, the enhanced high-frequency components and the original low-frequency components are reconstructed using inverse discrete wavelet transform (IDWT) to obtain the final enhanced features for subsequent network processing.

[0037]

[0038] This reconstruction process seamlessly integrates the enhanced detail information back into the spatial domain, resulting in improved output features. It preserves the global structure while enriching key details, thus providing a more robust feature representation for subsequent cross-modal alignment.

[0039] Next, as Figure 4 As shown, the implementation of the low-frequency decoupling multi-scale alignment module specifically includes:

[0040] S1, Low-frequency feature decoupling. For input features First, a low-pass filter is used to extract its low-frequency components, which are insensitive to changes in illumination and noise and can reflect the global structure shared across modes. Simultaneously, the original features are preserved as another branch. This is to supplement mode-specific details. The low-pass filtering operation can be represented as:

[0041]

[0042] in It is the average filter kernel, and its size determines the smoothness. This invention preferably... The filter size is chosen to achieve the best balance between preserving necessary details and effectively smoothing noise.

[0043] S2. Multi-scale feature refinement. To capture spatial dependencies at different granularities, multi-scale convolution is performed on the decoupled low-frequency features and the original features to generate refined features. This step enhances the model's robustness to pedestrian pose and scale variations.

[0044] S3. Refinement and Reconstruction of Decoupled Features. To further enhance the discriminative power of the features, this invention introduces a channel attention mechanism. First, the refined low-frequency features are... and original features Global average pooling is performed to obtain feature descriptors. Then, attention weights are generated for each channel using two fully connected layers and non-linear activation functions ReLU and Sigmoid.

[0045]

[0046]

[0047] This mechanism enables the network to adaptively enhance important feature channels and suppress redundant information. Finally, the low-frequency features, further refined through spatial attention, are weighted and fused with the original features, and then combined with the original input features. Residual connections are performed to obtain the final output of LF-DMSA. This fusion strategy ensures the organic combination of modality-shared information and modality-specific information, providing a better feature representation for the final cross-modal similarity measurement.

[0048] In the FSDE-Net method, achieving accurate cross-modal person matching requires not only carefully designed network modules but also effective optimization objectives to guide feature learning. To comprehensively optimize network performance, this invention employs a strategy of joint optimization using multiple loss functions. In addition to the cross-entropy loss, triplet loss, and orthogonal constraint loss commonly used in person re-identification, it also introduces loss functions specifically designed for cross-modal scenarios: Center-Guided Pair Mining Loss (CPM Loss) and Dual-Center Separation Loss (DCSLoss). Details are as follows:

[0049] S1. Cross-entropy Loss. This invention introduces cross-entropy loss, which, by explicitly modeling pedestrian labels, enables the model to capture global discriminative information about pedestrians. The formula is as follows:

[0050]

[0051] S2. Triple Loss. This invention introduces triple loss to further compress the distance between samples with the same identity in the embedding space and widen the distance between samples with different identities. The formula is as follows:

[0052]

[0053] S3. Orthogonal Constraint Loss. This invention introduces orthogonal constraint loss, which achieves frequency decoupling in the feature space through the inner product of the i frequency components after constraint decoupling. For a training batch containing B samples, assume... and Let represent the feature representations of the b-th sample at the m-th and n-th frequencies, respectively. The formulas are as follows:

[0054]

[0055] S3. Center-Guided Mining Loss. A core challenge in cross-modal matching is reducing the feature distribution differences between visible and infrared modes. This invention introduces CPM loss, which enhances the modality invariance of features by explicitly constraining the distance from a feature to its corresponding modality center. This loss function aims to minimize intra-class distance across modalities while maintaining inter-class separability. For samples in a minibatch, let the visible light modality feature be... Its corresponding modal center is Infrared modal characteristics are Its corresponding modal center is For visible light and infrared samples belonging to the same identity, the calculation formula is as follows:

[0056]

[0057] in This represents the Euclidean distance, used to measure the similarity between feature vectors. This is the preset margin value. The first term of the formula... Ensure infrared signature To its own modal center The distance is smaller than the visible light characteristic. To its own modal center The distance, thus maintaining intramodal compactness. The second term of the formula This mandates that the visible light characteristics of the same identity be identical. To its modal center The distance between them should be less than the distance between the two modal centers. and The distance between them. This constraint brings features of the same identity in different modalities closer together through their modality centers, effectively reducing the differences between modalities and achieving cross-modal feature alignment.

[0058] S4. Dual-center Separation Loss. To further enhance the discriminative power of features and ensure clear separation of features from different identities in the embedding space, this invention introduces DCS loss. This loss function constrains the feature distribution from two dimensions: sample level and center level. A mini-batch is set to have... Each sample has the following corresponding features: Identity tag is The corresponding identity center is This batch contains a total of There are several different identity categories. The formula is as follows:

[0059]

[0060] The formula presents two constraints. The first constraint is the sample-center convergence condition, which encourages the distance between each sample feature and its corresponding identity center to be less than a preset threshold, thereby making samples of the same identity more clustered in the feature space. The second constraint is the center-center separation condition, which encourages the distance between centers of different identities to be greater than another preset threshold, thus ensuring that feature clusters of different identities are far apart in space. Through these two constraints, the DCS loss effectively compresses the intra-class variance and expands the inter-class margin, significantly improving the discriminative power of the features.

[0061] In summary, this invention weights and combines the above loss functions to form the overall objective function for training FSDE-Net. Among these constraints... It is 0.4. =1, It is 0.01.

[0062]

[0063] Minimize the total loss function end-to-end FSDE-Net can learn pedestrian features that are both modality invariant and highly discriminative, thus achieving optimal performance in cross-modal retrieval tasks.

[0064] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cross-modal person re-identification method based on frequency-space decoupling enhancement, characterized in that, include: The system acquires visible light and infrared images of the same pedestrian under different lighting conditions. It then performs discrete wavelet transform decomposition on the input features to obtain low-frequency and high-frequency sub-bands. The low-frequency sub-bands contain global structural information, while the high-frequency sub-bands contain detailed texture information. Global average pooling is applied to the low-frequency sub-bands to generate channel weight vectors. These channel weight vectors are then used to adaptively weight the high-frequency sub-bands to enhance discriminative details. Finally, inverse discrete wavelet transform is used to reconstruct the weighted high-frequency sub-bands with the original low-frequency sub-bands, resulting in enhanced frequency domain features. Low-frequency components are extracted from the original features using a low-frequency filter. The low-frequency components and the original features are then subjected to multi-scale convolution processing to obtain multi-scale structural information. The multi-scale features are then refined using channel attention and spatial attention mechanisms. Finally, a weighted fusion strategy is used to integrate the multi-branch feature output. The enhanced frequency domain features are fused with the low-frequency decoupled multi-scale aligned features to generate a cross-modal pedestrian feature representation. The similarity between the query image and the candidate image is calculated based on the cross-modal pedestrian feature representation, and the pedestrian identity is determined according to the similarity ranking result.

2. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 1, characterized in that, The discrete wavelet transform employs the Haar wavelet basis function, which is used to capture abrupt changes in intensity in the image to enhance the structural stability of the model under complex illumination variations.

3. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 2, characterized in that, The low-frequency guided high-frequency enhancement step includes: S101. Perform global average pooling on the low-frequency sub-band to generate channel weight vectors; S102. Perform element-wise adaptive weighting on each high-frequency sub-band according to the channel weight vector to obtain the weighted high-frequency sub-band. S103. The weighted high-frequency subband and the original low-frequency subband are recombined using inverse discrete wavelet transform to generate enhanced frequency domain features.

4. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 3, characterized in that, The low-frequency decoupling multi-scale alignment step includes: constructing a low-frequency filter using average pooling to extract low-frequency components; in, Indicates the position of the convolution kernel The weight of the position, This represents half the kernel size. Indicates the feature map at location The value at; A dual-branch structure is constructed to process low-frequency components and original features separately; multiple 3×3 convolutional layers with different hole ratios are set in each branch to capture multi-scale structural information; and the multi-scale features are weighted and fused for output.

5. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 4, characterized in that, The channel attention mechanism calculates channel weights by performing global pooling and linear transformation on multi-scale features, normalizes them, and then weights the features channel by channel to enhance the discriminative feature response. in, and These represent the channel attention weights for the low-frequency branch and the original feature branch, respectively. and The weight matrix represents the linear transformation. and These represent the feature vectors after global pooling. This represents the Sigmoid activation function. and This represents the features after multi-scale convolution. and This represents the refined features of the channel attention.

6. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 4, characterized in that, The spatial attention mechanism performs channel concatenation on the feature map by performing average pooling and max pooling operations, and generates a spatial weight map using 7×7 convolution. It then achieves adaptive enhancement of spatial dimensions through element-wise multiplication.

7. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 1, characterized in that, The feature fusion step employs a residual connection method, which adds the enhanced frequency domain features to the original input features element by element to improve cross-modal alignment stability and feature representation diversity.

8. The cross-modal person re-identification method based on frequency-space decoupling enhancement according to claim 1, characterized in that, The pedestrian recognition steps include: constructing a multi-loss joint optimization model, jointly training it using cross-entropy loss, triplet loss, orthogonal constraint loss, center-guided mining loss, and dual-center separation loss, and weighting and summing each loss term to obtain the total loss function, thereby improving the cross-modal feature discrimination capability. in, For cross-entropy loss, For triple loss, Centered on guiding the assessment of excavation losses, For the two-center separation loss, For orthogonal constraint loss, , and The balancing parameters are 0.4, 1, and 0.01, respectively.

Images