Cross-modal pedestrian re-identification method based on style-independent multi-axis feature collaborative embedding

Abstract

The invention discloses a cross-modal pedestrian re-identification method based on style-independent multi-axis feature embedding, and relates to the technical field of pedestrian re-identification. The invention introduces an enhanced style independent feature module (ESF) aiming at the limitation of mode specific feature extraction. According to the module, by using an IN-guided style-independent weight map, style-independent discriminative features are enhanced in the channel dimension and the space dimension, and the influence of style change is effectively relieved; in order to relieve the problem of insufficient discriminative features in the feature embedding process, the invention provides a multi-axis collaborative embedding module which can effectively aggregate multi-dimensional discriminative feature information.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian re-identification, and in particular to a cross-modal pedestrian re-identification method based on collaborative embedding of style-independent multi-axis features. Background Art

[0002] The core goal of person re-identification (ReID) technology is to identify and track the appearance trajectory of a specific pedestrian through multiple non-overlapping cameras. In traditional scenarios, ReID mainly relies on visible light imaging images for recognition. However, when the ambient lighting conditions are insufficient (such as at night), it is difficult for visible light cameras to provide reliable identity clues, affecting the accuracy of recognition. In recent years, due to the application limitations of single-modal pedestrian re-identification technology in practical scenarios, visible-infrared person re-identification (VI-ReID) technology has been proposed. VI-ReID realizes cross-modal pedestrian recognition by combining the characteristics of visible light (VIS) and infrared (IR) images. Its goal is to retrieve a specific pedestrian in one modality image in another modality image.

[0003] The following references are proposed:

[0004] [1]Wu,A.,Zheng,W.-S.,Yu,H.-X.,Gong,S.,Lai,J.:Rgb-infrared cross-modality person reidentification.In:Proceedings of the IEEE InternationalConference on Computer Vision,pp.5380–5389(2017)

[0005] [2]Ye, M., Shen, J., Shao, L.: Visible-infrared person re-identificationvia homogeneous augmented tri-modal learning. IEEE Transactions on Information Forensics and Security 16,728739(2020)

[0006] [3]Zhong,X.,Lu,T.,Huang,W.,Ye,M.,Jia,X.,Lin,C.-W.:Grayscaleenhancement colorization network for visible-infrared personreidentification.IEEE Transactions on Circuits and Systems for VideoTechnology 32(3),14181430(2021)

[0007] [4]Ye,M.,Shen,J.,J.Crandall,D.,Shao,L.,Luo,J.:Dynamic dual-attentiveaggregation learning for visible-infrared person re-identification.In:Computer Vision–ECCV 2020:16th European Conference,Glasgow,UK,August 23–28,2020,Proceedings,Part XVII 16,pp.229–247(2020).Springer

[0008] [5]Zhang,Y.,Yan,Y.,Li,J.,Wang,H.:Mrcn:A novel modality restitutionand compensation network for visible-infrared person reidentification.In:Proceedings of the AAAI Conference on Artificial Intelligence,vol.37,pp.3498–3506(2023)

[0009] [6] Wu, Q., Dai, P., Chen, J., Lin, C.-W., Wu, Y., Huang, F., Zhong, B., Ji, R.: Discover crossmodality nuances for visible-infrared personreidentification. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 4330–4339 (2021)

[0010] The above literature summarizes that Wu et al. [1] first defined the VI-ReID problem and open-sourced the SYSU-MM01 dataset, which played a crucial role in promoting subsequent research. However, the core challenge of VI-ReID lies in how to reduce the modality difference between two heterogeneous images.

[0011] Existing VI-ReID methods can be divided into two categories. Image-level methods: narrow the differences between different modalities through image generation technology. For example, Ye et al. [2] generated grayscale images to assist training, and GECNet [3] attempted to establish a connection between VIS and IR modalities by generating intermediate grayscale images. The shortcomings of these methods include: 1) Noise problem: Due to the lack of real cross-modal comparison images, the generated images may contain mismatched noise, resulting in suboptimal performance. 2) High computational cost: The image generation or translation process requires a lot of computing resources, which limits practical applications. Feature-level methods: extract modality-shared features by designing network architectures, and use metric learning constraints to reduce modality differences. For example: the DDAG [4] method adopts a modality weighting module, but may introduce interference information due to style differences between modalities. MRCN [5] recovers and compensates modality features through the instance normalization (IN) module, but generating multiple branches increases computational complexity. MPANet[6] also designs a modality mitigation module through IN, but ignores the potential role of modality-specific features and only focuses on modality-shared features, resulting in the lack of discriminability of the extracted features.

[0012] Therefore, a new solution to the above problems needs to be proposed. Summary of the invention

[0013] The purpose of the present invention is to provide a cross-modal pedestrian re-identification method based on collaborative embedding of style-independent multi-axis features to solve the technical problems raised in the background technology.

[0014] To achieve the above object, the present invention provides the following technical solution: a cross-modal person re-identification method based on collaborative embedding of style-independent multi-axis features, comprising at least the following steps:

[0015] S1: Image feature extraction, i.e. extracting modality-shared features that are independent of style;

[0016] S2: Enhanced style-independent features. The backbone network of the ResNet-50 convolutional neural network has four stages, namely residual block 1 to residual block 4. The feature maps extracted from residual blocks 1, 2 and 3 are respectively input into the Enhance Style-independent Features module (ESF) to enhance the style-independent discriminative feature information in the two dimensions of channel and spatial. The Enhanced Style-independent Features module is the ESF module.

[0017] S3: To alleviate the modality difference, the feature map extracted by the third ESF module is input into the Multi-axis feature Collaborative Embedding (MCE) module, which embeds the feature map into a style-independent feature space to model the multi-axis information.

[0018] S4: Multi-loss joint optimization is adopted. Under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss and center clustering loss, the visible light-infrared pedestrian re-identification performance of the entire network is better.

[0019] Furthermore, the S1 at least comprises the following steps:

[0020] The SMCENet model first constructs two mode-specific branches: convolutional layer Conv1 and convolutional layer Conv2;

[0021] Then use two special branches, Conv1 and Conv2, to extract visible light and infrared mode-specific features of modality that are relevant to style;

[0022] Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality shared features.

[0023] Furthermore, the operation process of the ESF module in S2 at least includes the following steps:

[0024] For the feature map obtained by the residual block Where C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0025] First, F is input into three independent convolutional layers to obtain three new feature maps Q, K and V respectively, and then the feature dimensions of the three new feature maps are deformed so that Where N = H × W;

[0026] Then, a softmax function is applied between Q and K to calculate the channel attention map For S b Each position of is calculated as follows:

[0027]

[0028] where i and j represent the row and column coordinates of S, <·> represents the inner product, and d is the scaling factor. Then S b Matrix multiplication with V gives F c ′:

[0029] F c ′=S b ×V (2)

[0030] F c ′ is transformed so that F c ′∈R C×H×W , in order to further enhance the nonlinear expression ability, a multilayer perceptron is applied on Fc′; finally, F is added to the output of the channel attention through matrix addition, which is expressed as:

[0031] F c =MLP(F c ′)+F (3)

[0032] Refer to the above channel attention F c The acquisition method of spatial attention is obtained in the same way Then we get the output F of spatial attention p ;

[0033] The F described above is strengthened by an instance normalization-guided generated style-independent weight map. c and F p Features that are unrelated to the two different attention styles;

[0034] First, the feature instance normalization (IN) operation is applied to F;

[0035] Then, the 1*1 convolution Conv_1 with an output channel of 1 and the Sigmoid function are used to transform the normalized features of the instance to obtain a style-independent weight map:

[0036] map=Sigmoid(Conv_1(IN(F))) (4)

[0037] where map∈R H×W ,In order to aggregate these two style-independent features, the obtained style-independent weight map is used to perform pixel-by-pixel multiplication operations with the outputs of channel attention and spatial attention respectively;

[0038] Using parameter α c and β p Adaptively adjust the contribution of the two attention features to obtain the output of the ESF module The i-th channel of can be expressed as:

[0039]

[0040] Where i=1,…,C represents the i-th channel, and ⊙ represents matrix dot product.

[0041] Furthermore, the operation process of the MCE in S3 at least includes the following steps:

[0042] The input of the MCE module is the features extracted by the third ESF module.

[0043] The MCE module first passes three separate 3×3 convolutional layers to Embed into three independent spaces, and then use matrix transposition or identity mapping operations to generate embedded features in the three axis directions This process is described as follows:

[0044]

[0045] Among them, Conv represents a 3×3 convolutional layer, R H (·) and R W (·) indicates a 90° counterclockwise transposition along the H and W axes, respectively, and IM indicates the feature identity mapping operation;

[0046] The feature embedding optimization process of the MCE module for different axes includes two operations: adaptive compression transformation and excitation transformation;

[0047] In order to describe the problem clearly, the high axis is used as an example to describe the process;

[0048] For the adaptive compression transform operation, first Deform so that Where N = W × C, then use global average pooling and global maximum pooling to aggregate feature information along the N axis, respectively, to generate two different abstract features, and then input the two abstract features into the adaptive aggregation mechanism to generate compressed information. This process is expressed as:

[0049]

[0050] in G(·) and M(·) represent global average pooling and global maximum pooling respectively, α H and β H are two trainable floating parameters corresponding to the H axis, which can assign different weights to the features of average pooling and maximum pooling, thereby improving the discriminability of the output feature descriptor. sq (·) represents the feature compression transformation operation;

[0051] For feature excitation operation, given a high-axis feature operator Considering only the interaction between each channel and its k neighbors, the high-axis feature weight The calculation is as follows:

[0052]

[0053] Conv1(·,k) represents a convolution operation with a kernel size of 1×k. The quantitative relationship between the vector lengths m establishes a nonlinear mapping θ(·):

[0054]

[0055] It means rounding down to the nearest odd value to ensure that the convolution kernel size is always an odd value and is closest to the calculated result. In all experiments, λ and γ are set to 1.5 and 1 respectively;

[0056] Next, Generate attention weights about the high axis through the Sigmoid activation function

[0057]

[0058] Then A H and Multiply row by row to get the enhanced feature map That is, F′ H The i-th row of can be obtained by formula (11).

[0059]

[0060] Then for F′ H Deform so that

[0061] Use F′ H The enhanced features F′ in the other two dimensions are obtained by W and F′C ;

[0062] Next, F′ H , F′ W , F′ C Perform a matrix transposition operation in the opposite direction of the previous operation along the H, W, and C axes to obtain the feature map F″ H , F″ W , F″ C ;

[0063] Finally, the features enhanced in different axis directions are dynamically summed to obtain the final multi-axis collaborative embedding feature F″:

[0064]

[0065] Among them, μ1, μ2 and μ3 represent three trainable hyperparameters, which are used to dynamically adjust the feature contribution in different axis directions;

[0066] In addition, in order to ensure that the MCE module can capture the specific feature representations of different axes in different axis directions, the orthogonal separation loss is applied to ensure that non-repeated features can be learned in different axis directions;

[0067] The separation loss is defined as follows:

[0068]

[0069] Where m and n are the mth and nth feature vectors for generating embedded features, respectively, and i is the number of channels for generating embedded features.

[0070] Furthermore, the S4 at least includes the following steps:

[0071] Once F″ is obtained after multi-axis collaborative embedding by the MCE module, F″ is input into the last convolutional layer, and then the classification feature f is obtained after global average pooling and BN layer. i ;

[0072] Then two basic losses are calculated: identity classification loss comes from cross entropy loss and hard triplet loss;

[0073] The calculation formula for cross entropy loss is as follows:

[0074]

[0075] in y i represents the one-hot encoding of the classification label of the i-th sample and the corresponding true label, N is the number of training identities, and Q(·) represents f i Belong to y i The probability of

[0076] The hard triplet loss is used to optimize the distance between samples as follows:

[0077]

[0078] Where P represents the number of person identities contained in a training batch, K refers to the number of randomly selected images for each sample in each training batch, α is an interval parameter, [z] + represents max(z,0), D(·) is used to calculate the Euclidean distance, Represents the most difficult positive sample, representing the anchor point With positive samples The maximum distance between represents the most difficult negative sample, representing the anchor point With negative samples The minimum distance between

[0079] In addition, in order to enable the model to better learn modality-invariant features, the center cluster loss is introduced. The calculation formula of the center cluster loss is as follows:

[0080]

[0081] Where N is the number of training identities, h yi is the one marked as y in the current batch i The mean of the features, P represents the number of person identities contained in a training batch, and ρ is the minimum interval between centers;

[0082] In the inference stage, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the gallery image.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] 1. Aiming at the limitations of pattern-specific feature extraction, an enhanced style-independent feature module (ESF) is introduced. This module enhances the style-independent discriminative features in both channel and spatial dimensions by using IN-guided style-independent weight maps, effectively alleviating the impact of style changes.

[0085] 2. In order to alleviate the problem of insufficient discriminative features in the feature embedding process, a multi-axis collaborative embedding module is proposed, which can effectively aggregate the discriminative feature information of multiple dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0087] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0089] There is a pre-set in the design process of the present invention:

[0090] make Represent the visible light image and infrared image in the cross-modal pedestrian Re-ID dataset, respectively, where N v and N r are the number of samples in the two modes respectively. There are N=N in the data set. v +N r samples, the corresponding true value label set Where N p is the number of identities. Given a query image of a specific person, the goal of cross-modal person re-identification is to find a batch of image sequences from the image library of another modality based on similarity, where the higher the ranking, the higher the match with the specific person.

[0091] See also Figure 1 , a cross-modal person re-identification method based on style-independent multi-axis feature embedding includes at least the following steps:

[0092] S1: Image feature extraction, i.e. extracting modality-shared features that are independent of style;

[0093] S2: Enhanced style-independent features. The backbone network of the ResNet-50 convolutional neural network has four stages, namely residual block 1 to residual block 4. The feature maps extracted from residual blocks 1, 2 and 3 are respectively input into the Enhance Style-independent Features module (ESF) to enhance the style-independent discriminative feature information in the two dimensions of channel and spatial. The Enhanced Style-independent Features module is the ESF module.

[0094] S3: To alleviate the modality difference, the feature map extracted by the third ESF module is input into the Multi-axis feature Collaborative Embedding (MCE) module, which embeds the feature map into a style-independent feature space to model the multi-axis information.

[0095] S4: Multi-loss joint optimization is adopted. Under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss and center clustering loss, the visible light-infrared pedestrian re-identification performance of the entire network is better.

[0096] S1 includes at least the following steps:

[0097] The SMCENet model first constructs two mode-specific branches: convolutional layer Conv1 and convolutional layer Conv2;

[0098] Then use two special branches, Conv1 and Conv2, to extract visible light and infrared mode-specific features of modality that are relevant to style;

[0099] Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality shared features.

[0100] The operation process of the ESF module in S2 includes at least the following steps:

[0101] For the feature map obtained by the residual block Where C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0102] First, F is input into three independent convolutional layers to obtain three new feature maps Q, K and V respectively. Then, the feature dimensions of the three new feature maps are deformed so that Q, K, Where N = H × W;

[0103] Then, a softmax function is applied between Q and K to calculate the channel attention map For S b Each position of is calculated as follows:

[0104]

[0105] where i and j represent the row and column coordinates of S, <·> represents the inner product, and d is the scaling factor. Then S b Matrix multiplication with V gives F′:

[0106] Fc ′=S b ×V (2)

[0107] F c ′ is transformed so that F c ′∈R C×H×W , in order to further enhance the nonlinear expression ability, a multilayer perceptron is applied on Fc′; finally, F is added to the output of the channel attention through matrix addition, which is expressed as:

[0108] F c =MLP(F c ′)+F (3)

[0109] Refer to the above channel attention F c The acquisition method of spatial attention is obtained in the same way Then we get the output F of spatial attention p ;

[0110] The F described above is strengthened by an instance normalization-guided generated style-independent weight map. c and F p Features that are unrelated to the two different attention styles;

[0111] First, the feature instance normalization (IN) operation is applied to F;

[0112] Then, the 1*1 convolution Conv_1 with an output channel of 1 and the Sigmoid function are used to transform the normalized features of the instance to obtain a style-independent weight map:

[0113] map=Sigmoid(Conv_1(IN(F))) (4)

[0114] where map∈R H×W ,In order to aggregate these two style-independent features, the obtained style-independent weight map is used to perform pixel-by-pixel multiplication operations with the outputs of channel attention and spatial attention respectively;

[0115] Using parameter α c and β p Adaptively adjust the contribution of the two attention features to obtain the output of the ESF module The i-th channel of can be expressed as:

[0116]

[0117] Where i=1,…,C represents the i-th channel, and ⊙ represents matrix dot product.

[0118] The operation flow of MCE in S3 includes at least the following steps:

[0119] The input of the MCE module is the features extracted by the third ESF module.

[0120] The MCE module first passes three separate 3×3 convolutional layers to Embed into three independent spaces, and then use matrix transposition or identity mapping operations to generate embedded features in the three axis directions This process is described as follows:

[0121]

[0122] Among them, Conv represents a 3×3 convolutional layer, R H (·) and R W (·) indicates a 90° counterclockwise transposition along the H and W axes, respectively, and IM indicates the feature identity mapping operation;

[0123] The feature embedding optimization process of the MCE module for different axes includes two operations: adaptive compression transformation and excitation transformation;

[0124] In order to describe the problem clearly, the high axis is used as an example to describe the process;

[0125] For the adaptive compression transform operation, first Deform so that Where N = W × C, then use global average pooling and global maximum pooling to aggregate feature information along the N axis, respectively, to generate two different abstract features, and then input the two abstract features into the adaptive aggregation mechanism to generate compressed information. This process is expressed as:

[0126]

[0127] in G(·) and M(·) represent global average pooling and global maximum pooling respectively, α H and β H are two trainable floating parameters corresponding to the H axis, which can assign different weights to the features of average pooling and maximum pooling, thereby improving the discriminability of the output feature descriptor. sq (·) represents the feature compression transformation operation;

[0128] For feature excitation operation, given a high-axis feature operator Considering only the interaction between each channel and its k neighbors, the high-axis feature weight The calculation is as follows:

[0129]

[0130] Conv1(·,k) represents a convolution operation with a kernel size of 1×k. The quantitative relationship between the vector lengths m establishes a nonlinear mapping θ(·):

[0131]

[0132] It means rounding down to the nearest odd value to ensure that the convolution kernel size is always an odd value and is closest to the calculated result. In all experiments, λ and γ are set to 1.5 and 1 respectively;

[0133] Next, Generate attention weights about the high axis through the Sigmoid activation function

[0134]

[0135] Then A H and Multiply row by row to get the enhanced feature map That is, F′ H The i-th row of can be obtained by formula (11).

[0136]

[0137] Then for F′ H Deform so that

[0138] Use F′ H The enhanced features F′ in the other two dimensions are obtained by W and F′ C ;

[0139] Next, F′ H , F′ W , F′ C Perform a matrix transposition operation in the opposite direction of the previous operation along the H, W, and C axes to obtain the feature map F″ H , F″ W , F″ C ;

[0140] Finally, the features enhanced in different axis directions are dynamically summed to obtain the final multi-axis collaborative embedding feature F″:

[0141] F″=μ1F″ H +μ2F″ W +μ3F″ c (12)

[0142] Among them, μ1, μ2 and μ3 represent three trainable hyperparameters, which are used to dynamically adjust the feature contribution in different axis directions;

[0143] In addition, in order to ensure that the MCE module can capture the specific feature representations of different axes in different axis directions, the orthogonal separation loss is applied to ensure that non-repeated features can be learned in different axis directions;

[0144] The separation loss is defined as follows:

[0145]

[0146] Where m and n are the mth and nth feature vectors for generating embedded features, respectively, and i is the number of channels for generating embedded features.

[0147] S4 at least includes the following steps:

[0148] Once F″ is obtained after multi-axis collaborative embedding by the MCE module, F″ is input into the last convolutional layer, and then the classification feature f is obtained after global average pooling and BN layer. i ;

[0149] Then two basic losses are calculated: identity classification loss comes from cross entropy loss and hard triplet loss;

[0150] The calculation formula for cross entropy loss is as follows:

[0151]

[0152] in y i represents the one-hot encoding of the classification label of the i-th sample and the corresponding true label, N is the number of training identities, and Q(·) represents f i Belong to y i The probability of

[0153] The hard triplet loss is used to optimize the distance between samples as follows:

[0154]

[0155] Where P represents the number of person identities contained in a training batch, K refers to the number of randomly selected images for each sample in each training batch, α is an interval parameter, [z] + represents max(z,0), D(·) is used to calculate the Euclidean distance, Represents the most difficult positive sample, representing the anchor point With positive samples The maximum distance between represents the most difficult negative sample, representing the anchor point With negative samples The minimum distance between

[0156] In addition, in order to enable the model to better learn modality-invariant features, the center cluster loss is introduced. The calculation formula of the center cluster loss is as follows:

[0157]

[0158] Where N is the number of training identities, h yi is the one marked as y in the current batch i The mean of the features, P represents the number of person identities contained in a training batch, and ρ is the minimum interval between centers;

[0159] In the inference stage, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the gallery image.

[0160] For hard triplet loss[7], identity loss[8], orthogonal separation loss[9], and center clustering loss

[10] , please refer to the following references:

[0161] [7]Hermans,A.,Beyer,L.,Leibe,B.:In defense of the triplet loss forperson re-identification.arXiv preprint arXiv:1703.07737(2017)

[0162] [8] Luo, H., Gu, Y., Liao,

[0163] [9]Zhang,Y.,Wang,H.:Diverse embedding expansion network and low-lightcross-modality benchmark for visible-infrared person re-identification.In:Proceedings of the IEEE / CVF Conference on Computer Vision and PatternRecognition,pp.2153–2162(2023)

[0164]

[10] Wu, Q., Dai, P., Chen, J., Lin, C.-W., Wu, Y., Huang, F., Zhong, B., Ji, R.: Discover crossmodality nuances for visible-infrared personreidentification. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 4330–4339 (2021).

[0165] In summary:

[0166] The enhanced style-independent feature module (ESF) in the present invention enhances the style-independent discriminative features in both channel and spatial dimensions by using the style-independent weight map guided by IN, thereby effectively alleviating the impact of style changes.

[0167] The Multi-axis feature Collaborative Embedding module (MCE) in the present invention performs multi-dimensional modeling on the features in the three axes of height, width and channel during the feature embedding stage, then aggregates the feature information on multiple axes and embeds them into a style-independent feature space, thereby effectively reducing the impact of modal differences.

[0168] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A cross-modal person re-identification method based on collaborative embedding of style-independent multi-axis features, characterized by: At least the following steps are included: S1: Image feature extraction, i.e. extracting modality-shared features that are independent of style; S2: Enhanced style-independent features. The backbone network of the ResNet-50 convolutional neural network has four stages, namely residual block 1 to residual block 4. The feature maps extracted from residual blocks 1, 2, and 3 are respectively input into the enhanced style-independent feature module to enhance the style-independent discriminative feature information in both channel and spatial dimensions. The enhanced style-independent feature module is the ESF module. S3: To alleviate the modality difference, the feature map extracted by the third ESF module is input into the multi-axis feature co-embedding module, i.e., the MCE module, which embeds the feature map into a style-independent feature space to model the multi-axis information. S4: Multi-loss joint optimization is adopted. Under the joint supervision of hard triplet loss, identity loss, orthogonal separation loss and center clustering loss, the visible light-infrared pedestrian re-identification performance of the entire network is better.

2. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 1, characterized in that: The S1 at least comprises the following steps: The SMCENet model first constructs two mode-specific branches: convolutional layer Conv1 and convolutional layer Conv2; Then, two special branches, Conv1 and Conv2, are used to extract visible light V and infrared mode-specific features of modality that are relevant to style; Then, a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset is used as the backbone to extract style-independent modality shared features.

3. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 2, characterized in that: The operation process of the ESF module in S2 at least includes the following steps: For the feature map obtained by the residual block Where C, H, and W represent the number of channels, height, and width of the feature map, respectively; First, F is input into three independent convolutional layers to obtain three new feature maps Q, K and V respectively, and then the feature dimensions of the three new feature maps are deformed so that Where N = H × W; Then, a softmax function is applied between g and K to calculate the channel attention map For S b Each position of is calculated as follows: where i and j represent the row and column coordinates of S, <·> represents the inner product, and d is the scaling factor; Then S b Multiply the matrix with V to get F′ c : F′ c =S b ×V (2) F′ c Transform so that F′ c ∈R C×H×W In order to further enhance the nonlinear expression ability, in F′ c Apply multi-layer perceptron on Finally, F is added to the output of the channel attention through matrix addition, which is expressed as: F c =MLP(F′ c )+F (3) Refer to the above channel attention F c The acquisition method of spatial attention is obtained in the same way Then we get the output F of spatial attention p ; The F described above is strengthened by an instance normalization-guided generated style-independent weight map. c and F p Features that are unrelated to the two different attention styles; First, the feature instance normalization (IN) operation is applied to F; Then, the 1*1 convolution Conv_1 with an output channel of 1 and the Sigmoid function are used to transform the normalized features of the instance to obtain a style-independent weight map: map=Sigmoid(Conv_1(IN(F)))(4) where map∈R H×W ,In order to aggregate these two style-independent features, the obtained style-independent weight map is used to perform pixel-by-pixel multiplication operations with the outputs of channel attention and spatial attention respectively; Using parameter α c and β p Adaptively adjust the contribution of the two attention features to obtain the output of the ESF module The i-th channel of can be expressed as: Where i=1,…, C represents the i-th channel, and ⊙ represents matrix dot product.

4. The cross-modal person re-identification method based on style-independent multi-axis feature collaborative embedding according to claim 3 is characterized by: The operation process of the MCE in S3 at least includes the following steps: The input of the MCE module is the features extracted by the third ESF module. The MCE module first passes three separate 3×3 convolutional layers to Embed into three independent spaces, and then use matrix transposition or identity mapping operations to generate embedded features in the three axis directions This process is described as follows: Among them, Conv represents a 3×3 convolutional layer, R H (·) and R W (·) indicates a 90° counterclockwise transposition along the H and W axes, respectively, and IM indicates the feature identity mapping operation; The feature embedding optimization process of the MCE module for different axes includes two operations: adaptive compression transformation and excitation transformation; In order to describe the problem clearly, the high axis is used as an example to describe the process; For the adaptive compression transform operation, first Deform so that Where N = W × C, and then use global average pooling and global maximum pooling to aggregate feature information along the N axis, respectively, to generate two different abstract features, and then input the two abstract features into the adaptive aggregation mechanism to generate compressed information. This process is expressed as: in G(·) and M(·) represent global average pooling and global maximum pooling respectively, α H and β H are two trainable floating parameters corresponding to the H axis, which can assign different weights to the features of average pooling and maximum pooling, thereby improving the discriminability of the output feature descriptor. sq (·) represents the feature compression transformation operation; For feature excitation operation, given a high-axis feature operator Considering only the interaction between each channel and its k neighbors, the high-axis feature weight The calculation is as follows: Conv1(·, k) represents a convolution operation with a kernel size of 1×k. The quantitative relationship between the vector lengths m establishes a nonlinear mapping θ(.): It means rounding down to the nearest odd value to ensure that the convolution kernel size is always an odd value and is closest to the calculated result. In all experiments, λ and γ are set to 1.5 and 1 respectively; Next, Generate attention weights about the high axis through the Sigmoid activation function Then A H and Multiply row by row to get the enhanced feature map That is, F′ H The i-th row of can be obtained by formula (11); Then transform F′H so that Use F′ H The enhanced features F′W and F′C of the other two dimensions are obtained in this way; Next, F′ H , F′W, F′C perform matrix transposition operation in the opposite direction of the previous operation along the H, W, C axes to obtain the feature map F″ H , F″W, F″C; Finally, the features enhanced in different axis directions are dynamically summed to obtain the final multi-axis collaborative embedding feature F″: F″=μ1F″ H +μ2F″ W +μ3F″ C (12) Among them, μ1, μ2 and μ3 represent three trainable hyperparameters, which are used to dynamically adjust the feature contribution in different axis directions; In addition, in order to ensure that the MCE module can capture the specific feature representations of different axes in different axis directions, the orthogonal separation loss is applied to ensure that non-repeated features can be learned in different axis directions; The separation loss is defined as follows: Where m and n are the mth and nth feature vectors for generating embedded features, respectively, and i is the number of channels for generating embedded features.

5. The cross-modal person re-identification method based on collaborative embedding of style-independent multi-axis features according to claim 4, characterized in that: The S4 at least comprises the following steps: Once F″ is obtained after multi-axis collaborative embedding by the MCE module, F″ is input into the last convolutional layer, and then the classification feature f is obtained after global average pooling and BN layer. i ; Then two basic losses are calculated: identity classification loss comes from cross entropy loss and hard triplet loss; The calculation formula for cross entropy loss is as follows: in y i represents the one-hot encoding of the classification label of the i-th sample and the corresponding true label, N is the number of training identities, and Q(·) represents f i Belong to y i probability; The hard triplet loss is used to optimize the distance between samples as follows: Where P represents the number of person identities contained in a training batch, K refers to the number of randomly selected images for each sample in each training batch, α is an interval parameter, [z]+ represents max(z, 0), and D(·) is used to calculate the Euclidean distance. Represents the most difficult positive sample, representing the anchor point With positive samples The maximum distance between represents the most difficult negative sample, representing the anchor point With negative samples The minimum distance between In addition, in order to enable the model to better learn modality-invariant features, the center cluster loss is introduced. The calculation formula of the center cluster loss is as follows: Where N is the number of training identities, is the one marked as y in the current batch i The mean of the features, P represents the number of person identities contained in a training batch, and ρ is the minimum interval between centers; In the inference stage, cross-modal retrieval is performed by calculating the cosine similarity between the feature vector of the image to be searched and the feature vector of the gallery image.

Images