A cross-modal pedestrian re-identification method based on multi-modal common feature space exploration

Abstract

This invention discloses a cross-modal person re-identification method based on multimodal shared feature space exploration. It includes a unimodal feature extraction network, a multimodal shared feature space exploration network, and an auxiliary training module for multimodal feature interaction. Training images are preprocessed to obtain training samples. The unimodal feature extraction network and the multimodal shared feature space exploration network are constructed. The unimodal feature extraction network extracts unimodal features, while the multimodal shared feature space exploration network extracts modal shared features from multiple modal shared feature spaces and performs appropriate fusion. The training of both networks is conducted under the supervision of classification loss and metric loss. The obtained feature descriptions are used for similarity calculation during testing. This invention can reduce modal differences between features, and the interacting features are also constrained by the loss function, enhancing the network's ability to extract common modal features of pedestrians. This invention significantly improves the accuracy of cross-modal person re-identification tasks.

Description

Technical Field

[0001] This invention belongs to the fields of image processing and pattern recognition technology, and specifically relates to a cross-modal pedestrian re-identification method based on the exploration of multimodal common feature space. Background Technology

[0002] Pedestrian re-identification, as an important subtask in computer vision systems, aims to achieve cross-camera retrieval of target pedestrians, that is, given an image of a pedestrian, matching that pedestrian in scenes captured by other cameras. Pedestrian re-identification can utilize richer pedestrian features, overcoming the limitations of face recognition which only focuses on facial information. Combined with pedestrian detection and tracking technologies, pedestrian re-identification has broad prospects in the field of intelligent surveillance and can be applied to intelligent security systems, intelligent missing person search systems, and intelligent commercial systems.

[0003] Pedestrian re-identification based on visible light images or videos has reached the level of industrial applications. However, this method is easily affected by lighting conditions, limiting its practical application. Infrared cameras, on the other hand, are highly robust to varying lighting environments, compensating for the shortcomings of visible light images. Furthermore, most cameras on the market today have considered the difficulty of scene monitoring under insufficient lighting conditions with visible light cameras and have added infrared shooting modes, automatically switching between visible light and infrared shooting modes based on the lighting conditions. Therefore, research on visible light-infrared cross-modal pedestrian re-identification is of great significance for the practical application of pedestrian re-identification.

[0004] Currently, the visible-infrared cross-modal person re-identification task mainly utilizes deep learning algorithms to extract modality-independent pedestrian-related features from visible light and infrared images, thereby achieving cross-modal pedestrian retrieval. The challenges of this task include intra-class variations caused by changes in pedestrian pose and shooting angle, and modal differences resulting from the different imaging principles of infrared and visible light cameras. The basic approach to addressing these challenges is to reduce modal differences and obtain common modal features related to pedestrians in the images. Most existing methods use convolutional neural networks to map single-modal features to the same feature space to extract common modal features for pedestrian matching. However, these methods do not sufficiently mine common modal features, neglecting the complementary relationships between common modal features extracted from single-modal features mapped to different feature spaces, resulting in poor algorithm performance for cross-modal person re-identification. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a cross-modal pedestrian re-identification method based on multimodal common feature space exploration, so as to solve the problem that the existing methods do not fully mine the modal common features and further improve the accuracy of cross-modal pedestrian re-identification.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A cross-modal person re-identification method based on multimodal shared feature space mining is proposed. First, a single-modal feature extraction network is used to obtain single-modal pedestrian features at different levels. Second, the multi-branch feature extraction network of a multimodal shared space exploration network maps the single-modal features at different levels to different modal shared feature spaces to extract multiple modal shared features. Then, the feature fusion module of the multimodal shared space exploration network fuses the multiple modal shared features based on the complementary relationships between global and global, local and local, and local and global features. Finally, the fused features are used for pedestrian matching. During model training, a multimodal feature interaction module is used to reduce modal differences between features and improve the network's ability to extract modality-independent features.

[0008] Multi-branch feature extraction network implementation, including:

[0009] The modality-shared detail feature space branch takes shallow detail features of a single modality as input and outputs a modality-shared feature map.

[0010] The semantic feature space branch shared by modalities takes deep semantic features of a single modality as input and outputs a feature map shared by modalities.

[0011] A global feature space branch shared by all modalities takes as input global pedestrian features of a single modality and outputs a feature vector shared by all modalities.

[0012] A cross-modal person re-identification method based on multimodal shared feature space exploration, specifically including:

[0013] (1) Collect visible light pedestrian images as visible light modal training images, and collect infrared pedestrian images as infrared modal training images;

[0014] (2) Construct a multimodal common feature space exploration network model, which combines multiple modal common feature spaces to obtain the feature representation of pedestrians for pedestrian re-identification;

[0015] (3) Construct an auxiliary training module for multimodal feature interaction. This module utilizes the information transmission between pedestrian features to reduce the modal differences between different modal features and improve the performance of the feature extraction network.

[0016] Perform the same preprocessing operation on visible light and infrared images: add pixels with a width of l and a value of 0 to each side of the input image, and use random cropping to obtain the same image size, and then randomly flip the image horizontally.

[0017] Preprocessing operations also include random erasure and grayscale conversion of the visible light image.

[0018] The multimodal shared feature space includes the global feature space, semantic feature space, and detailed feature space shared by all modalities.

[0019] The multimodal shared feature space is constructed after single-modal feature extraction, and the visible light modal features F i R and infrared modal characteristics F i I (i∈1,2,3,4,5) are extracted using two independent ResNet50 networks, where i represents different levels of the feature extraction network.

[0020] The multimodal shared feature space exploration network model first uses a multi-branch feature extraction network to map the global features, semantic features, and detailed features of a single modality to the shared feature space of multiple modalities to mine complementary shared features. Then, a supplementary and enhanced fusion module is used to fully utilize the complementary relationships between the shared features of multiple modalities. The specific method is as follows:

[0021] (1) The multi-branch feature extraction network includes a global feature space branch common to the modalities, a semantic feature space branch common to the modalities, and a detailed feature space branch common to the modalities.

[0022] (2) The global feature space branch shared by modes will reveal the deep features of visible light modes. Deep features of infrared modes The compressed global features are mapped to the modal shared feature space to mine the modal shared global pedestrian features. The mapping method is a fully connected layer network with shared parameters between the two modalities.

[0023] (3) The semantic feature space shared by modes will be branched to include the semantic features of visible light modes. Semantic features of infrared modes Mapping to a modality-shared feature space to mine modality-shared pedestrian features, the mapping method includes the last two convolutional blocks of ResNet50;

[0024] (4) The modal common detail feature space branch will include the detail features of the visible light modes. and detailed features of infrared modes Mapping to a modality-shared feature space to mine modality-shared pedestrian features, the mapping method includes the last three convolutional layers of ResNet50;

[0025] (5) The supplementary and enhanced fusion module, after acquiring multiple modal common features with complementary relationships, utilizes the potential complementary relationships between details, details and global features, and global features extracted from the multi-feature space to obtain richer pedestrian features and improve model performance. The complementary relationships between details exist between modal common features extracted from the detail feature space and the semantic feature space, and the complementary relationships between details and global features, and global features exist between modal common features extracted from the three feature spaces.

[0026] The multimodal shared feature space exploration network learns under the constraints of classification loss and metric loss. The classification loss refers to the multi-class cross-entropy loss function, and the metric loss refers to the center-based triplet loss function.

[0027] The classification loss function is expressed as:

[0028]

[0029] Where y represents the label, p represents the predicted value, N represents the total number of samples, and C represents the total number of pedestrian identities in the dataset, which is the number of categories.

[0030] The loss is expressed as:

[0031]

[0032] Where M1 and M2 represent the visible light mode and the infrared mode, The eigenvectors representing visible light modes, The eigenvector representing the infrared mode. Let ρ represent the visible light class center and infrared class center of the pedestrian feature with identity l, respectively, and ρ be the threshold.

[0033] Multimodal feature interaction includes intramodal and intermodal feature interaction. Intramodal interaction improves the discriminative power of features by exchanging information between features within the same modality. Intermodal interaction reduces the modal differences between features by exchanging information between features from different modalities. The multimodal feature interaction module consists of a self-attention structure. The input of the module is pedestrian features from two modalities. This module mines the similarity between features, so that the features of one modality can perceive some information from the features of another modality, thereby reducing the modal differences between features.

[0034] The output of multimodal feature interaction is constrained by classification loss and metric loss to ensure the discriminative power of pedestrian features while reducing modal differences between features.

[0035] The trained cross-modal pedestrian re-identification network was tested using a publicly available dataset, and the results were obtained.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention provides a cross-modal person re-identification method based on multimodal shared feature space exploration. By mapping single-modal features at different levels to three different feature spaces for modal shared feature mining and utilizing the complementary relationships between features extracted from the three modal shared feature spaces, a richer description of pedestrian modal shared features is obtained. During training, a multimodal feature interaction module enables intramodal and intermodal interactions of features, further reducing modal differences between features without increasing the number of parameters required for testing. By utilizing a multimodal shared feature space exploration network and a multimodal feature interaction module, this invention significantly improves the performance of cross-modal person re-identification algorithms. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of a cross-modal pedestrian re-identification method based on multimodal shared feature space exploration disclosed in this invention;

[0040] Figure 2 This is a network diagram of the algorithm for a cross-modal person re-identification method based on multimodal shared feature space exploration proposed in this invention.

[0041] Figure 3 This is the feature fusion module proposed in this invention;

[0042] Figure 4 This is the multimodal feature interaction module proposed in this invention. Detailed Implementation

[0043] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0044] The terms "comprising" and "having," and any variations thereof, in the specification, embodiments, claims, and drawings of this invention are intended to cover non-exclusive inclusion, such as including a series of steps or units.

[0045] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0046] This invention presents a cross-modal person re-identification method based on multimodal shared feature space exploration. It utilizes a single-modal feature extraction network to extract single-modal features from the input image. Then, a multi-branch feature extraction network maps single-modal features at different levels to three different modal shared feature spaces for modal shared feature extraction. Next, a feature fusion module fuses features extracted from the three feature spaces using complementary relationships to obtain richer modal shared features. Finally, during training, multimodal feature interaction is used to reduce modal differences between features, further improving the algorithm performance of cross-modal person re-identification.

[0047] like Figure 1 As shown, the present invention specifically includes the following steps:

[0048] (1) Collect and preprocess the cross-modal pedestrian re-identification dataset to obtain training samples, wherein visible light pedestrian images are used as visible light domain training images and infrared pedestrian images are used as infrared domain training images.

[0049] In this embodiment, the same preprocessing operation is performed on both visible light and infrared images: a pixel with a width of 1 and a value of 0 is added to each side of the input image, random cropping is used to obtain the same image size, and the image is randomly flipped horizontally. In this embodiment, the image size is 288*144.

[0050] (2) Visible light feature extraction network and infrared feature extraction network are used to extract single-modal features of visible light images and infrared images respectively, to obtain visible light image features at different levels. and infrared image features Where i = 1, 2, 3, 4, 5, and sp represents modal-specific features.

[0051] (3) The multimodal common feature space exploration network first uses a multi-branch feature extraction network to map the features of visible light and infrared modes to different modal common feature spaces to extract multiple modal common features. Then, the feature fusion module is used to perform feature fusion, and the fused features are used for pedestrian re-identification.

[0052] In this embodiment, as Figure 2 As shown, the multimodal shared feature space exploration network obtains the final representation of a pedestrian through the following steps:

[0053] 1) Using a parameter-sharing fully connected layer, the global features of visible and infrared modes are mapped to a modal-shared global feature space to extract the modal-shared global features F.R,gf and F I,gf The mathematical expression is as follows:

[0054]

[0055] m represents modal information, which can be visible light mode R and infrared mode I;

[0056] gf represents global features;

[0057] GAP stands for Global Average Pooling Operation;

[0058] FC(*,α gf ) represents the fully connected layer and its parameters.

[0059] 2) The last two convolutional blocks of ResNet50 are used to map the semantic features of the visible light and infrared modes to the shared semantic feature space of the modes to extract the shared modal features F. R,sf and F I,sf The mathematical expression is as follows:

[0060]

[0061] ConvB5(*,α sf5 ) and ConvB4(*,α sf4 ) represent the last two convolutional blocks of ResNet50 and their parameters, respectively;

[0062] sf represents semantic features;

[0063] 3) The last three convolutional blocks of ResNet50 are used to map the detailed features of the visible light and infrared modalities to the modality-shared semantic feature space to extract the modality-shared features F. R,df and F I,df The mathematical expression is as follows:

[0064]

[0065] ConvB5(*,α df5 ), ConvB4(*,α df4 ) and ConvB3(*,α df3 ) represent the last three convolutional blocks of ResNet50 and their parameters, respectively;

[0066] df represents detailed features;

[0067] 4) Modal common features F were obtained. m,gf F m,sf and F m,df Subsequently, this invention uses a feature fusion module to fuse these features, and the fusion process takes into account feature map F. m,sfand F m,df The complementary relationship between local parts, F m,sf and F m,df The acquired global features and F m,gf The complementary relationship between global and global systems, F m,sf and F m,df The acquired local features and F m,gf The complementary relationship between local and global features yielded a richer description of pedestrian characteristics. For example... Figure 3 As shown, according to feature map F m,sf and F m,df The complementary relationship between local parts, the present invention uses F m,sf and F m,df Using shared information as guidance, the two feature maps are fused. This enriches the feature representation while maintaining the discriminative power of the features.

[0068]

[0069]

[0070]

[0071]

[0072] This represents a convolutional layer and its parameters;

[0073] ⊙ indicates element-wise multiplication. This represents element-wise addition.

[0074] F m,com F represents m,sf and F m,df Shared information;

[0075] This represents a Batch Normalization (BN) layer and its corresponding parameters;

[0076] F m,cpl This represents the fused feature map.

[0077] According to F m,sf and F m,df The acquired global features and F m,gf Based on the complementary relationship between global features, this invention uses a cascaded approach to obtain a rich global feature description.

[0078] F m,sg =FC(GAP(F m,sf ),α sf )

[0079] F m,dg=FC(GAP(F m,df ),α df )

[0080] F m,g =Cat(F m,gf ,F m,sg ,F m,dg )

[0081] FC(*,α sf ) represents the fully connected layer of the semantic branch and its corresponding parameters;

[0082] FC(*,α df ) represents the fully connected layer of the detail branch and its corresponding parameters;

[0083] Cat indicates a cascading operation;

[0084] F m,g Represents the global features after fusion;

[0085] According to F m,sf and F m,df The acquired local features and F m,gf Based on the complementary relationship between local and global features, this invention obtains a holistic representation of pedestrians' local and global characteristics by combining local and global features.

[0086]

[0087]

[0088]

[0089] F m =Cat(F m,g ,F m,l )

[0090] AP stands for Average Pooling Operation;

[0091] p represents a local feature, k represents the kth local feature, and k = 1, 2, 3, 4, 5, 6;

[0092] This represents the fully connected layer corresponding to the k-th local feature and its parameters;

[0093] F m,l Represents the local features after fusion;

[0094] F m This represents the overall characteristics of the pedestrian after final fusion.

[0095] 5) When acquiring common features and local features of multiple modalities of pedestrians, this invention makes the feature F...m,gf F m,sg F m,dg and Apply metric constraints to the corresponding classification result S m,gf S m,sg S m,dg and By imposing classification constraints, the network can extract features that are irrelevant to pedestrian-related modalities.

[0096] The classification loss function is expressed as:

[0097]

[0098] Where y represents the label, p represents the predicted value, N represents the total number of samples, and C represents the total number of pedestrian identities in the dataset, which is the number of categories.

[0099] The loss is expressed as:

[0100]

[0101] Where M1 and M2 represent the visible light mode and the infrared mode, The eigenvectors representing visible light modes, The eigenvector representing the infrared mode. Let ρ represent the visible light class center and infrared class center of the pedestrian feature with identity l, respectively, and ρ be the threshold.

[0102] Specifically, the loss function used in this invention is:

[0103]

[0104]

[0105] K represents the number of local features.

[0106] 6) During training, this invention uses a multimodal feature interaction module to achieve information transfer between features, enabling one modality to perceive the pedestrian features of the current modality and another modality, thereby enhancing the discriminative power of the features and reducing modal differences between features. The multimodal feature interaction module is implemented using a self-attention model, transferring information between features by calculating the similarity between input features. This invention uses two multimodal feature interaction modules to complete the interaction between global features and local features respectively, such as... Figure 4 As shown, the explanation will be based on global features.

[0107] K agg,g =W K,g F agg,g

[0108] Qagg,g =W Q,g F agg,g

[0109] V agg,g =W V,g F agg,g

[0110]

[0111] F agg,g An aggregated representation of multiple global features of the same pedestrian;

[0112] W K,g W Q,g W V,g This represents three different fully connected layer parameters;

[0113] K agg,g Q agg,g V agg,g These represent the three intermediate features in the self-attention model;

[0114] Softmax represents the Softmax activation function, used for normalization;

[0115] C represents the number of channels;

[0116] After feature interaction, the resulting global feature F o,g The feature extraction network is decomposed according to the original aggregation form, and then the decomposed features are constrained by classification constraints and metric constraints to further optimize the feature extraction network. Local features are processed using an operation similar to that used for global features.

[0117] 7) Verify the effectiveness of the proposed cross-modal person re-identification method based on specific modal feature compensation. Test the trained cross-modal person re-identification network using a publicly available dataset and obtain the corresponding results.

[0118] In this embodiment, to verify the effectiveness of the proposed pedestrian re-identification method, performance metrics are evaluated using the public datasets SYSU-MM01 and RegDB.

[0119] The technical effects of the present invention will be further explained below with reference to simulation experiments:

[0120] 1. Simulation conditions: All simulation experiments were conducted on the operating system Ubuntu 16.04.5, the hardware environment was Nvidia GeForce GTX1080Ti GPU, and the PyTorch deep learning framework was used.

[0121] 2. Simulation content and result analysis:

[0122] The results of experiments conducted on two public cross-modal person re-identification datasets, SYSU-MM01 and RegDB, comparing the present invention with existing cross-modal person re-identification methods based on shared modal feature learning, are presented in Tables 1 and 2. The simulation results are objectively evaluated using recognized metrics.

[0123] Table 1 shows the experimental results on the SYSU-MM01 dataset.

[0124]

[0125]

[0126] Table 2 shows the experimental results on the RegDB dataset.

[0127]

[0128]

[0129] Table 3 Ablation experimental results on the SYSU-MM01 dataset

[0130] Methods R1 R10 R20 mAP Baseline 64.03 94.53 98.15 59.92 Baseline+MBFE 69.12 95.81 98.68 65.08 Baseline+MBFE 69.36 95.96 98.65 65.5 Baseline+MBFE+CFB 70.71 96.02 98.65 66.36 Baseline+MFI 68.17 95.42 98.42 63.5 Baseline+MBFE+CFB+MFI 71.84 96.56 99.12 68.28

[0131] in:

[0132] R1, R10, R20, and mAP represent the Top-1 pedestrian image recognition accuracy, Top-10 pedestrian image recognition accuracy, Top-20 pedestrian image recognition accuracy, and average retrieval accuracy, respectively.

[0133] All-Search refers to pedestrian re-identification in panoramic mode, including indoor and outdoor camera scenes;

[0134] Indoor-Search indicates pedestrian re-identification in indoor mode;

[0135] Single-shot means that only one image is selected for each pedestrian identity in the image library;

[0136] Baseline, MFI, MBFE, and CFB represent the base network, the multi-branch feature extraction sub-network, and the fusion module of the multimodal common feature space exploration network, respectively.

[0137] R1, R10, R20, and mAP are all better the higher they are. Tables 1 and 2 show that on the two public datasets, this invention achieves optimal performance across all metrics, significantly improving the performance of cross-modal person re-identification. Table 3 shows that the multimodal shared feature space exploration network and multimodal feature interaction module of this invention jointly improve the accuracy of the cross-modal person re-identification task, further enhancing the performance of the base network, fully demonstrating the effectiveness and superiority of the method presented in this invention.

[0138] The embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments, and various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

[0139] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A cross-modal person re-identification method based on multimodal shared feature space exploration, characterized in that, include: (1) Collect visible light pedestrian images as visible light modal training images and collect infrared pedestrian images as infrared modal training images; (2) Construct a multimodal shared feature space exploration network model, which combines multiple modal shared feature spaces to obtain pedestrian feature representations for pedestrian re-identification; the multimodal shared feature space includes a modal shared global feature space, a semantic feature space, and a detail feature space; the multimodal shared feature space is constructed after single-modal feature extraction, visible light modal features and infrared modal characteristics Extraction was performed using two independent ResNet50 arrays. This represents different layers of the feature extraction network; (3) Construct an auxiliary training module for multimodal feature interaction. This module uses information transmission between pedestrian features to reduce the modal differences between different modal features and improve the performance of the feature extraction network. The multimodal shared feature space exploration network model first uses a multi-branch feature extraction network to map the global features, semantic features, and detailed features of a single modality to the shared feature space of multiple modalities to mine complementary shared features. Then, a supplementary and enhanced fusion module is used to fully utilize the complementary relationships between the shared features of multiple modalities. The specific method is as follows: 1) The multi-branch feature extraction network includes a global feature space branch common to all modalities, a semantic feature space branch common to all modalities, and a detailed feature space branch common to all modalities; 2) The global feature space branch shared by modes will reveal the deep features of visible light modes. Deep features of infrared modes The compressed global features are mapped to the modal shared feature space to mine the modal shared global pedestrian features. The mapping method is a fully connected layer network with shared parameters between the two modalities. 3) The semantic feature space branching shared by modes will include the semantic features of visible light modes. Semantic features of infrared modes Mapping to a modality-shared feature space to mine modality-shared pedestrian features, the mapping method includes the last two convolutional blocks of ResNet50; 4) The shared detail feature space branch of the modes will include the detail features of the visible light modes. and detailed features of infrared modes Mapping to a modality-shared feature space to mine modality-shared pedestrian features, the mapping method includes the last three convolutional layers of ResNet50; 5) The supplementary and enhanced fusion module, after acquiring multiple modal common features with complementary relationships, utilizes the potential complementary relationships between details, local and global features, and global features extracted from the modal common features in the multi-feature space to obtain richer pedestrian features and improve the performance of the model. The complementary relationships between details exist between the modal common features extracted from the detail feature space and the semantic feature space, and the complementary relationships between local and global features and global features exist between the modal common features extracted from the three feature spaces.

2. The cross-modal person re-identification method based on multimodal shared feature space exploration according to claim 1, characterized in that, Perform the same preprocessing operation on both visible light and infrared images: add a width of [value] to each side of the input image. l The image is then randomly cropped to the same size, and then randomly horizontally flipped.

3. The cross-modal pedestrian re-identification method based on multimodal shared feature space exploration according to claim 2, characterized in that, Preprocessing operations also include random erasure and grayscale conversion of the visible light image.

4. The cross-modal person re-identification method based on multimodal common feature space exploration according to claim 1, characterized in that, The multimodal shared feature space exploration network learns under the constraints of classification loss and metric loss. The classification loss refers to the multi-class cross-entropy loss function, and the metric loss refers to the center-based triplet loss function. The classification loss function is expressed as: in, Indicates a label, Indicates the predicted value. Represents the total number of samples. This represents the total number of pedestrian identities in the dataset, which is also the number of categories. The loss is expressed as: in, and Representing visible light modes and infrared modes, The eigenvectors representing visible light modes, The eigenvector representing the infrared mode. They represent the identities as follows: Visible light class center and infrared class center of pedestrian characteristics, The threshold value is used.

5. The cross-modal person re-identification method based on multimodal shared feature space exploration according to claim 1, characterized in that, Multimodal feature interaction includes intramodal and intermodal feature interaction. Intramodal interaction improves the discriminative power of features by exchanging information between features within the same modality; intermodal interaction reduces the modal differences between features by exchanging information between features from different modalities. The multimodal feature interaction module consists of a self-attention structure. The input of the module is pedestrian features from two modalities. By mining the similarity between features, the module enables the features of one modality to perceive some information from the features of the other modality, thereby reducing the modal differences between features.

6. The cross-modal person re-identification method based on multimodal shared feature space exploration according to claim 5, characterized in that, The output of multimodal feature interaction is constrained by classification loss and metric loss to ensure the discriminative power of pedestrian features while reducing modal differences between features.

7. The cross-modal person re-identification method based on multimodal common feature space exploration according to claim 1, characterized in that, The trained cross-modal pedestrian re-identification network was tested using a publicly available dataset, and the results were obtained.

Images