Cross-modal pedestrian re-identification method based on style-independent multi-axis feature collaborative embedding

Abstract

The application discloses a cross-modal pedestrian re-identification method based on style-independent multi-axis feature embedding and relates to the technical field of pedestrian re-identification. The application introduces an enhanced style-independent feature module (ESF) in view of the limitation of mode-specific feature extraction. The module enhances the style-independent discriminative features in the channel and spatial dimensions by using IN-guided style-independent weight maps, effectively alleviating the influence of style changes. In order to alleviate the problem of insufficient discriminative features in the feature embedding process, the application proposes a multi-axis collaborative embedding module, which can effectively aggregate discriminative feature information in multiple dimensions.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, specifically to a cross-modal pedestrian re-identification method based on style-independent multi-axis feature collaborative embedding. Background Technology

[0002] The core objective of Person Re-identification (ReID) technology is to identify and track the trajectory of a specific pedestrian using multiple non-overlapping cameras. In traditional scenarios, ReID primarily relies on visible light imaging for identification. However, when ambient lighting conditions are insufficient (e.g., at night), visible light cameras struggle to provide reliable identification clues, impacting accuracy. In recent years, due to the limitations of single-modal person re-identification technology in practical applications, Visible-infrared Person Re-identification (VI-ReID) technology has been proposed. VI-ReID achieves cross-modal person identification by combining the characteristics of visible light (VIS) and infrared (IR) images, aiming to retrieve a specific pedestrian from one modality of imagery within another.

[0003] The following references are provided:

[0004] [1] Wu, A., Zheng, W.-S., Yu, H.-X., Gong, S., Lai, J.: Rgb-infraredcross-modality person reidentification. In: Proceedings of the IEEEInternational Conference on Computer Vision, pp. 5380–5389 (2017)

[0005] [2] Ye, M., Shen, J., Shao, L.: Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEETransactions on Information Forensics and Security 16, 728739 (2020)

[0006] [3] Zhong, X., Lu, T., Huang, W., Ye, M., Jia, X., Lin, C.-W.:Grayscale enhancement colorization network for visible-infrared personreidentification. IEEE Transactions on Circuits and Systems for VideoTechnology 32(3), 14181430 (2021)

[0007] [4] Ye, M., Shen, J., J. Crandall, D., Shao, L., Luo, J.: Dynamicdual-attentive aggregation learning for visible-infrared person re-identification. In: Computer Vision–ECCV 2020: 16th European Conference,Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16, pp. 229–247(2020). Springer

[0008] [5] Zhang, Y., Yan, Y., Li, J., Wang, H.: Mrcn: A novel modalityrestitution and compensation network for visible-infrared personreidentification. In: Proceedings of the AAAI Conference on ArtificialIntelligence, vol. 37, pp. 3498–3506 (2023)

[0009] [6] Wu, Q., Dai, P., Chen, J., Lin, C.-W., Wu, Y., Huang, F., Zhong, B., Ji, R.: Discover crossmodality nuances for visible-infrared personreidentification. In: Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition, pp. 4330–4339 (2021)

[0010] The above literature summary states that Wu et al. [1] first defined the VI-ReID problem and open-sourced the SYSU-MM01 dataset, which played a crucial role in promoting subsequent research. However, the core challenge of VI-ReID lies in how to reduce the modal differences between two heterogeneous images.

[0011] The existing VI-ReID methods are mainly divided into two categories. Image-level methods: use image generation techniques to narrow the differences between different modalities. For example, Ye et al. [2] generated grayscale images to assist training, and GECNet [3] tried to establish a connection between VIS and IR modalities by generating intermediate grayscale images. The shortcomings of these methods include: 1) Noise problem: due to the lack of real cross-modal comparison images, the generated images may contain mismatched noise, resulting in suboptimal performance. 2) High computational cost: the image generation or translation process requires a lot of computational resources, which limits practical applications. Feature-level methods: extract modal shared features by designing network architecture, and at the same time use metric learning constraints to reduce modal differences. For example, the DDAG [4] method uses a modal weighting module, but may introduce interference information due to style differences between modalities. MRCN [5] recovers and compensates modal features through the instance normalization (IN) module, but generating multiple branches increases computational complexity. MPANet[6] also uses IN to design a modality mitigation module, but ignores the potential role of modality-specific features and focuses only on modality-shared features, resulting in the extracted features lacking discriminativeness.

[0012] Therefore, a new solution is needed to address the above problems. Summary of the Invention

[0013] The purpose of this invention is to provide a cross-modal pedestrian re-identification method based on style-independent multi-axis feature collaborative embedding, so as to solve the technical problems mentioned in the background art.

[0014] To achieve the above objectives, the present invention provides the following technical solution: a cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding, comprising at least the following steps:

[0015] S1: Image feature extraction, i.e., extracting style-independent modality-shared features;

[0016] S2: Enhancing Style-Independent Features. The backbone of the ResNet-50 convolutional neural network has four stages: Residual Block 1 to Residual Block 4. The feature maps extracted from Residual Blocks 1, 2, and 3 are input into the Enhance Style-Independent Features module (ESF) to enhance style-independent discriminative feature information in both channel and spatial dimensions. The Enhance Style-Independent Features module is the ESF module.

[0017] S3: To alleviate modal differences, the feature map extracted by the third ESF module is input into the multi-axis feature collaborative embedding (MCE) module, i.e., the MCE module. The MCE module embeds the feature map into a style-independent feature space to model the multi-axis information.

[0018] S4: Employing multi-loss joint optimization, under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss, and center clustering loss, the visible light-infrared light pedestrian re-identification performance of the entire network is improved.

[0019] Furthermore, S1 includes at least the following steps:

[0020] The SMCENet model first constructs two pattern-specific branches: convolutional layer Conv1 and convolutional layer Conv2;

[0021] Then, two special branches, Conv1 and Conv2, are used to extract visible light. and infrared Modal style-related pattern-specific features;

[0022] Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality-shared features.

[0023] Furthermore, the operation process of the ESF module in S2 includes at least the following steps:

[0024] For the feature map F obtained from the residual block Where C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0025] First, F is input into three independent convolutional layers to obtain three new feature maps. , and Then, the feature dimensions of the three new feature maps are deformed to make ,in ;

[0026] Then, in Q and The softmax function is used to calculate the channel attention map. ,for The calculation for each position is as follows:

[0027] (1)

[0028] in and express The row and column coordinates, Indicates the inner product. As the scaling factor, then and Perform matrix multiplication to obtain :

[0029] = (2)

[0030] right To deform, so that To further enhance the nonlinear expressive power, in A multilayer perceptron is applied; finally, matrix addition is used to... Incorporating this process into the output of channel attention, this process can be represented as:

[0031] (3)

[0032] Refer to the channel attention mentioned above The method for obtaining the attention map related to spatial attention is the same, used to obtain the attention map. This leads to the output of spatial attention. ;

[0033] This reinforces the previous point by using a style-independent weight map generated through instance normalization. and Two different attention mechanisms with style-independent features;

[0034] First, the feature instance normalization (IN) operation is applied. ;

[0035] Then, a 1x1 convolution Conv_1 with 1 output channel is used and The function transforms the normalized features of the instances to obtain a style-independent weight map. :

[0036] (4)

[0037] in To aggregate these two style-independent features, the resulting style-independent weight map is used to perform pixel-wise dot product operations with the outputs of channel attention and spatial attention, respectively.

[0038] Use parameters and The contributions of the two attention features are adaptively adjusted to obtain the output of the ESF module. , The Each channel can be described as:

[0039] (5)

[0040] In the formula Indicates the first One channel, This represents the matrix dot product.

[0041] Furthermore, the operation process of MCE in S3 includes at least the following steps:

[0042] The input to the MCE module is the features extracted by the third ESF module. ;

[0043] The MCE module first uses three separate 3×3 convolutional layers to... The embeddings are placed into three independent spaces, and then matrix transpose or identity mapping operations are used to generate embedding features in the three axial directions. , , This process can be described as follows:

[0044] (6)

[0045] Where Conv represents a 3×3 convolutional layer, and These represent a 90° counterclockwise transpose along the H and W axes, respectively, and IM represents the feature identity mapping operation.

[0046] The MCE module's feature embedding optimization process for different axes includes two operations: adaptive compression transformation and excitation transformation.

[0047] To clearly describe the problem, we will use the high axis as an example to illustrate the process;

[0048] For adaptive compression transform operations, firstly... Deformation makes ,in Then, global average pooling and global max pooling are used to aggregate feature information along the N-axis, generating two different abstract features. These two abstract features are then input into an adaptive aggregation mechanism to generate compressed information. This process is described as follows:

[0049] (7)

[0050] in G( ) and M( ) represent global average pooling and global max pooling, respectively. and These are two trainable floating parameters corresponding to the H-axis, which can assign different weights to features from average pooling and max pooling, thereby improving the discriminative power of the output feature descriptors. This indicates a feature compression transformation operation;

[0051] For feature activation operations, given a high-axis feature operator If we only consider the interaction between each channel and its k neighbors, then the high-axis feature weights... The calculation is as follows:

[0052] (8)

[0053] in This represents a convolution operation with a kernel size of 1×k, where k is equal to k. Vector length I m To establish a nonlinear mapping based on the quantitative relationship between them. :

[0054] (9)

[0055] This means rounding down to the nearest odd value, ensuring that the kernel size is always an odd value and closest to the calculated result. In all experiments, λ and γ were set to 1.5 and 1, respectively.

[0056] Immediately afterwards, pass The activation function generates attention weights about the high axis. ,

[0057] (10)

[0058] Then By multiplying row by row, we obtain the enhanced feature map. ,Right now The i-th row can be obtained through equation (11).

[0059] (11)

[0060] Then to To deform, so that .

[0061] Using to obtain This method yields enhanced features in two additional dimensions. and ;

[0062] Next, , , Perform a matrix transpose operation along the H, W, and C axes in the opposite direction to the previous operation to obtain the feature map. , , ;

[0063] Finally, the features enhanced along different axes are dynamically summed to obtain the final multi-axis collaborative embedding features. :

[0064] (12)

[0065] in, These represent three trainable hyperparameters used to dynamically adjust feature contributions along different axes.

[0066] In addition, to ensure that the MCE module can capture the specific feature representations of different axes in different axial directions, orthogonal separation loss is applied to ensure that non-repeating features can be learned in different axial directions;

[0067] The separation loss is defined as follows:

[0068] (13)

[0069] in and The first and second embedded features are generated respectively. One and 1 eigenvector The number of channels used to generate the embedded features.

[0070] Furthermore, S4 includes at least the following steps:

[0071] Once obtained after multi-axis collaborative embedding of the MCE module ,Will The input is fed into the last convolutional layer, and then after global average pooling and batch normalization (BN) layers, the classification features are obtained. ;

[0072] Then, two basic losses are calculated: the identity classification loss comes from the cross-entropy loss and the hard triplet loss;

[0073] The formula for calculating cross-entropy loss is as follows:

[0074] (14)

[0075] in , They represent the first The classification label and corresponding ground truth label of each sample are one-hot encoded, where N is the number of training identities, and Q is the number of training identities. express belong The probability of;

[0076] Hard triplet loss is used to optimize the distance between samples, as follows:

[0077] (15)

[0078] Where P represents the number of person identities included in a training batch, and K refers to the number of images randomly selected from each sample in each training batch. It is an interval parameter, [z] + Let max(z,0) represent the maximum value of the z-axis, and D( ) is used to calculate the Euclidean distance, maxD( () represents the most difficult positive sample, and () represents the anchor point. Compared with positive samples The maximum distance between them, minD( () represents the most difficult negative sample, and () represents the anchor point. With negative samples The minimum distance between them;

[0079] In addition, to enable the model to better learn modality-invariant features, a center clustering loss is introduced. The formula for calculating the center cluster loss is as follows:

[0080] + (16)

[0081] Where N is the number of training identities. It is marked as in the current batch The mean of the features, where P represents the number of person identities included in a training batch. It is the minimum interval between centers;

[0082] During the inference phase, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the images in the library.

[0083] Compared with the prior art, the beneficial effects of the present invention are:

[0084] 1. To address the limitations of pattern-specific feature extraction, an enhanced style-independent feature module (ESF) is introduced. This module enhances style-independent discriminative features in both channel and spatial dimensions by using an IN-guided style-independent weight map, effectively mitigating the impact of style changes.

[0085] 2. To alleviate the problem of insufficient discriminative features in the feature embedding process, a multi-axis collaborative embedding module is proposed, which can effectively aggregate discriminative feature information from multiple dimensions. Attached Figure Description

[0086] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0087] Figure 1 This is a schematic diagram of the overall structure of the present invention. Detailed Implementation

[0088] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0089] There is a pre-set requirement in the design process of this invention:

[0090] make and These represent visible light and infrared images from the cross-modal pedestrian Re-ID dataset, respectively. and These represent the number of samples in each of the two modalities. The dataset contains a total of... Each sample, and the corresponding set of truth labels. ,in For each person, the goal of cross-modal person re-identification is to find a sequence of images from another modality's image library based on similarity, where higher-ranked images indicate a higher match with that specific person.

[0091] Please see Figure 1 A cross-modal person re-identification method based on style-independent multi-axis feature embedding includes at least the following steps:

[0092] S1: Image feature extraction, i.e., extracting style-independent modality-shared features;

[0093] S2: Enhancing Style-Independent Features. The backbone of the ResNet-50 convolutional neural network has four stages: Residual Block 1 to Residual Block 4. The feature maps extracted from Residual Blocks 1, 2, and 3 are input into the Enhance Style-Independent Features module (ESF) to enhance style-independent discriminative feature information in both channel and spatial dimensions. The Enhance Style-Independent Features module is the ESF module.

[0094] S3: To alleviate modal differences, the feature map extracted by the third ESF module is input into the multi-axis feature collaborative embedding (MCE) module, i.e., the MCE module. The MCE module embeds the feature map into a style-independent feature space to model the multi-axis information.

[0095] S4: Employing multi-loss joint optimization, under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss, and center clustering loss, the visible light-infrared light pedestrian re-identification performance of the entire network is improved.

[0096] S1 includes at least the following steps:

[0097] The SMCENet model first constructs two pattern-specific branches: convolutional layer Conv1 and convolutional layer Conv2;

[0098] Then, two special branches, Conv1 and Conv2, are used to extract visible light. and infrared Modal style-related pattern-specific features;

[0099] Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality-shared features.

[0100] The operation process of the ESF module in S2 includes at least the following steps:

[0101] For the feature map F obtained from the residual block Where C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0102] First, F is input into three independent convolutional layers to obtain three new feature maps. , and Then, the feature dimensions of the three new feature maps are deformed to make ,in ;

[0103] Then, in Q and The softmax function is used to calculate the channel attention map. ,for The calculation for each position is as follows:

[0104] (1)

[0105] in and express The row and column coordinates, Indicates the inner product. As the scaling factor, then and Perform matrix multiplication to obtain :

[0106] = (2)

[0107] right To deform, so that To further enhance the nonlinear expressive power, in A multilayer perceptron is applied; finally, matrix addition is used to... Incorporating this process into the output of channel attention, this process can be represented as:

[0108] (3)

[0109] Refer to the channel attention mentioned above The method for obtaining the attention map related to spatial attention is the same, used to obtain the attention map. This leads to the output of spatial attention. ;

[0110] This reinforces the previous point by using a style-independent weight map generated through instance normalization. and Two different attention mechanisms with style-independent features;

[0111] First, the feature instance normalization (IN) operation is applied. ;

[0112] Then, a 1x1 convolution Conv_1 with 1 output channel is used and The function transforms the normalized features of the instances to obtain a style-independent weight map. :

[0113] (4)

[0114] in To aggregate these two style-independent features, the resulting style-independent weight map is used to perform pixel-wise dot product operations with the outputs of channel attention and spatial attention, respectively.

[0115] Use parameters and The contributions of the two attention features are adaptively adjusted to obtain the output of the ESF module. , The Each channel can be described as:

[0116] (5)

[0117] In the formula Indicates the first One channel, This represents the matrix dot product.

[0118] The operating procedure for MCE in S3 includes at least the following steps:

[0119] The input to the MCE module is the features extracted by the third ESF module. ;

[0120] The MCE module first uses three separate 3×3 convolutional layers to... The embeddings are placed into three independent spaces, and then matrix transpose or identity mapping operations are used to generate embedding features in the three axial directions. , , This process can be described as follows:

[0121] (6)

[0122] Where Conv represents a 3×3 convolutional layer, and These represent a 90° counterclockwise transpose along the H and W axes, respectively, and IM represents the feature identity mapping operation.

[0123] The MCE module's feature embedding optimization process for different axes includes two operations: adaptive compression transformation and excitation transformation.

[0124] To clearly describe the problem, we will use the high axis as an example to illustrate the process;

[0125] For adaptive compression transform operations, firstly... Deformation makes ,in Then, global average pooling and global max pooling are used to aggregate feature information along the N-axis, generating two different abstract features. These two abstract features are then input into an adaptive aggregation mechanism to generate compressed information. This process is described as follows:

[0126] (7)

[0127] in G( ) and M( ) represent global average pooling and global max pooling, respectively. and These are two trainable floating parameters corresponding to the H-axis, which can assign different weights to features from average pooling and max pooling, thereby improving the discriminative power of the output feature descriptors. This indicates a feature compression transformation operation;

[0128] For feature activation operations, given a high-axis feature operator If we only consider the interaction between each channel and its k neighbors, then the high-axis feature weights... The calculation is as follows:

[0129] (8)

[0130] in This represents a convolution operation with a kernel size of 1×k, where k is equal to k. Vector length I m To establish a nonlinear mapping based on the quantitative relationship between them. :

[0131] (9)

[0132] This means rounding down to the nearest odd value, ensuring that the kernel size is always an odd value and closest to the calculated result. In all experiments, λ and γ were set to 1.5 and 1, respectively.

[0133] Immediately afterwards, pass The activation function generates attention weights about the high axis. ,

[0134] (10)

[0135] Then By multiplying row by row, we obtain the enhanced feature map. ,Right now The i-th row can be obtained through equation (11).

[0136] (11)

[0137] Then to To deform, so that .

[0138] Using to obtain This method yields enhanced features in two additional dimensions. and ;

[0139] Next, , , Perform a matrix transpose operation along the H, W, and C axes in the opposite direction to the previous operation to obtain the feature map. , , ;

[0140] Finally, the features enhanced along different axes are dynamically summed to obtain the final multi-axis collaborative embedding features. :

[0141] (12)

[0142] in, These represent three trainable hyperparameters used to dynamically adjust feature contributions along different axes.

[0143] In addition, to ensure that the MCE module can capture the specific feature representations of different axes in different axial directions, orthogonal separation loss is applied to ensure that non-repeating features can be learned in different axial directions;

[0144] The separation loss is defined as follows:

[0145] (13)

[0146] in and The first and second embedded features are generated respectively. One and 1 eigenvector The number of channels used to generate the embedded features.

[0147] S4 includes at least the following steps:

[0148] Once obtained after multi-axis collaborative embedding of the MCE module ,Will The input is fed into the last convolutional layer, and then after global average pooling and batch normalization (BN) layers, the classification features are obtained. ;

[0149] Then, two basic losses are calculated: the identity classification loss comes from the cross-entropy loss and the hard triplet loss;

[0150] The formula for calculating cross-entropy loss is as follows:

[0151] (14)

[0152] in , They represent the first The classification label and corresponding ground truth label of each sample are one-hot encoded, where N is the number of training identities, and Q is the number of training identities. express belong The probability of;

[0153] Hard triplet loss is used to optimize the distance between samples, as follows:

[0154] (15)

[0155] Where P represents the number of person identities included in a training batch, and K refers to the number of images randomly selected from each sample in each training batch. It is an interval parameter, [z] + Let max(z,0) represent the maximum value of the z-axis, and D( ) is used to calculate the Euclidean distance, maxD( () represents the most difficult positive sample, and () represents the anchor point. Compared with positive samples The maximum distance between them, minD( () represents the most difficult negative sample, and () represents the anchor point. With negative samples The minimum distance between them;

[0156] In addition, to enable the model to better learn modality-invariant features, a center clustering loss is introduced. The formula for calculating the center cluster loss is as follows:

[0157] + (16)

[0158] Where N is the number of training identities. It is marked as in the current batch The mean of the features, where P represents the number of person identities included in a training batch. It is the minimum interval between centers;

[0159] During the inference phase, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the images in the library.

[0160] For hard triplet loss[7], identity loss[8], orthogonal separation loss[9] and center clustering loss

[10] , see the following literature:

[0161] [7] Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)

[0162] [8] Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W.: Bag of tricks and a strong baseline for deep person re-identification. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.0–0 (2019)

[0163] [9] Zhang, Y., Wang, H.: Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. In: Proceedings of the IEEE / CVF Conference on Computer Vision and PatternRecognition, pp. 2153–2162 (2023)

[0164]

[10] Wu, Q., Dai, P., Chen, J., Lin, C.-W., Wu, Y., Huang, F., Zhong, B., Ji, R.: Discover crossmodality nuances for visible-infrared personreidentification. In: Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition, pp. 4330–4339 (2021).

[0165] In conclusion:

[0166] The Enhanced Style-independent Feature Module (ESF) in this invention enhances style-independent discriminative features in both channel and spatial dimensions by using an IN-guided style-independent weight map, effectively mitigating the impact of style variations.

[0167] The Multi-axis Feature Collaborative Embedding (MCE) module in this invention performs multi-dimensional modeling of features along the height, width, and channel axes during the feature embedding stage. Then, it aggregates the feature information along the multiple axes and embeds it into a style-independent feature space, effectively reducing the impact of modality differences.

[0168] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding, characterized in that: At least the following steps are included: S1: Image feature extraction, i.e., extracting style-independent modality-shared features; S2: Enhance style-independent features. The backbone network of the ResNet-50 convolutional neural network has four stages, namely residual block 1 to residual block 4. The feature maps extracted from residual blocks 1, 2 and 3 are respectively input into the style-independent feature enhancement module to enhance the style-independent discriminative feature information in the channel and spatial dimensions. The style-independent feature enhancement module is also known as the ESF module. The operation process of the ESF module in S2 includes at least the following steps: The feature map obtained from the residual block ,in C, H, W These represent the number of channels, height, and width of the feature map, respectively. F Input the channel attention module and spatial attention module respectively, and obtain the output of the channel attention module. and spatial attention output ; Normalization by an instance IN The generated style-independent weight map is used to reinforce the previously described approach. and Two different attention mechanisms with style-independent features; First, the feature instance normalization operation is applied to ; Then, a 1x1 convolution with 1 output channel is used. Conv_ 1 and The function transforms the normalized features of the instances to obtain a style-independent weight map. : ; in To aggregate these two style-independent features, the resulting style-independent weight map is used to perform pixel-wise dot product operations with the outputs of channel attention and spatial attention, respectively. Use parameters and The contributions of the two attention features are adaptively adjusted to obtain the output of the ESF module. , The Each channel can be described as: ; In the formula Indicates the first One channel, Represents matrix dot product; S3: To alleviate modal differences, the feature map extracted by the third ESF module is input into the multi-axis feature collaborative embedding module, i.e., the MCE module. The MCE module embeds the feature map into a style-independent feature space to model the multi-axis information. S4: Employing multi-loss joint optimization, under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss, and center clustering loss, the visible light-infrared light pedestrian re-identification performance of the entire network is improved.

2. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 1, characterized in that: S1 includes at least the following steps: The SMCENet model first constructs two pattern-specific branches: convolutional layer Conv1 and convolutional layer Conv2; Then, two special branches, Conv1 and Conv2, are used to extract visible light. and infrared Modal style-related pattern-specific features; Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality-shared features.

3. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 2, characterized in that: The operation process of MCE in S3 includes at least the following steps: The input to the MCE module is the features extracted by the third ESF module. ; The MCE module first passes through 3 separate The convolutional layer will The embeddings are placed into three independent spaces, and then matrix transpose or identity mapping operations are used to generate embedding features in the three axial directions. This process can be described as follows: ; in, Conv express convolutional layers, and They represent along H and W Rotate the axis 90° counterclockwise. IM This represents the feature identity mapping operation; The MCE module's feature embedding optimization process for different axes includes two operations: adaptive compression transformation and excitation transformation. To clearly describe the problem, we will use the high axis as an example to illustrate the process; For adaptive compression transform operations, firstly... Deformation makes ,in Then, global average pooling and global max pooling are used respectively to... N The axis aggregates feature information to generate two different abstract features. These two abstract features are then input into an adaptive aggregation mechanism to generate compressed information. This process can be described as follows: ; in , and These represent global average pooling and global max pooling, respectively. and It corresponds H The two trainable floating parameters of the axis can assign different weights to features from average pooling and max pooling, thereby improving the discriminativeness of the output feature descriptors. This indicates a feature compression transformation operation; For feature activation operations, given a high-axis feature operator Consider only each channel and its k The interaction between neighbors, then the high-axis feature weights The calculation is as follows: ; in This indicates that the kernel size is 1× k The convolution operation, for k and Vector length I m To establish a nonlinear mapping based on the quantitative relationship between them. ; ; This indicates rounding down to the nearest odd value, ensuring the convolution kernel size is always an odd value and closest to the calculated result. In all experiments, this was set... λ and γ They are 1.5 and 1 respectively; Immediately afterwards, pass The activation function generates attention weights about the high axis. , ; Then By multiplying row by row, we obtain the enhanced feature map. ,Right now The i The row can be obtained through equation (11); ; Then to To deform, so that ; Using to obtain This method yields enhanced features in two additional dimensions. and ; Next, along H, W, C The axis is subjected to a matrix transpose operation in the opposite direction to the previous operation, resulting in the feature map F″. H , F″ W , F″ C ; Finally, the features enhanced along different axes are dynamically summed to obtain the final multi-axis collaborative embedding feature F″. F″=μ1F″ H +μ2F″ W +μ3F″ C (12); in, These represent three trainable hyperparameters used to dynamically adjust feature contributions along different axes. In addition, to ensure that the MCE module can capture the specific feature representations of different axes in different axial directions, orthogonal separation loss is applied to ensure that non-repeating features can be learned in different axial directions; The separation loss is defined as follows: ; in and The first and second embedded features are generated respectively. Individual and 1 eigenvector The number of channels used to generate the embedded features.

4. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 3, characterized in that: The S4 includes at least the following steps: Once F″ is obtained after multi-axis collaborative embedding by the MCE module, it is input into the last convolutional layer. After global average pooling and BN layers, the classification features are obtained. ; Then, two basic losses are calculated: the identity classification loss comes from the cross-entropy loss and the hard triplet loss; The formula for calculating cross-entropy loss is as follows: ; in They represent the first The classification label and the one-hot encoding of the corresponding true label for each sample. N To train the number of identities, express belong The probability of; Hard triplet loss is used to optimize the distance between samples, as follows: ; in, P This indicates the number of character identities included in a training batch. K This refers to the number of images randomly selected from each sample in each training batch. It is an interval parameter, [z] + Let max(z,0) represent the maximum value. Used to calculate Euclidean distance. Represents the most difficult positive sample, and represents the anchor point. Compared with positive samples The maximum distance between them Represents the most difficult negative sample, and represents the anchor point. With negative samples The minimum distance between them; In addition, to enable the model to better learn modality-invariant features, a center clustering loss is introduced. The formula for calculating the center clustering loss is as follows: ; in N To train the number of identities, It is marked as in the current batch The mean of the characteristics, P This indicates the number of character identities included in a training batch. It is the minimum interval between centers; During the inference phase, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the images in the library.

Images