Cross-modal pedestrian re-identification method and system based on feature reconstruction

Abstract

The present application relates to a kind of cross-modal pedestrian re-identification methods based on feature reconstruction, comprising the following steps: 1) from the visible light picture and infrared picture of multiple pedestrians in data set pair extraction, form visible light training dataset and infrared training dataset;2) construct cross-modal pedestrian re-identification network model based on feature reconstruction, the cross-modal pedestrian re-identification network model mainly includes specific feature extraction module, multi-scale feature extraction module, Token perception's multi-scale feature fusion module and cross-modal feature reconstruction module;Cross-modal pedestrian re-identification network model is trained by visible light training dataset and infrared training dataset, and the model parameter of generalization is obtained;3) the cross-modal pedestrian re-identification network model of training is used for cross-modal retrieval, and cross-modal pedestrian re-identification is realized.The method and system are conducive to obtaining more stable, robust, accurate cross-modal pedestrian re-identification result.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a cross-modal person re-identification method and system based on feature reconstruction. Background Technology

[0002] Pedestrian re-identification, a key technology in intelligent video surveillance systems, aims to identify identical pedestrians within disjoint camera systems. Its applications are widespread, including large venues such as airports, shopping malls, and campuses. Previously, much work focused on pedestrian re-identification in visible light scenarios, neglecting real-world lighting variations. Modern camera systems automatically switch between visible and infrared light modes based on real-time lighting conditions to ensure 24 / 7 monitoring. Currently, less attention is paid to cross-modal pedestrian re-identification, which involves finding infrared (visible) images of identical pedestrians within an infrared (visible) light search database based on a visible (infrared) image of the target pedestrian. Cross-modal pedestrian re-identification not only addresses common challenges such as varying pedestrian poses, occlusion, camera angle differences, cluttered backgrounds, and lighting variations, but also tackles the issue of modal differences between images.

[0003] Currently, cross-modal person re-identification can be divided into two categories: image-based methods and feature-based methods. Image-based methods aim to transfer images from one modality to another. Li et al. (D.Li, X.Wei, X.Hong, Y.Gong, Infrared-visible cross-modal person re-identification with an Xmodality, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020, pp.4610–4617.) designed a lightweight shared network to learn modal cues in visible light images and then use these cues to generate intermediate modal images. Wang et al. (Z.Wang, Z.Wang, Y.Zheng, Y.Chuang, S.Satoh, Learning to reduce dual-level discrepancy for infrared-visible person re-identification, in: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, 2019, pp.618–626.) proposed D... 2RL models employ adversarial learning to extract modal information of pedestrians in different modalities. Then, a generative network transfers this learned modal information to create intermediate modal pedestrian images, which are provided to the network as additional modal images for further learning, thus reducing modal differences. However, adversarial learning not only results in images with discontinuous features and lost semantic information, but also leads to network convergence problems.

[0004] For feature-based learning methods, the goal is to learn the shared features of pedestrians across different modalities, thereby reducing the negative impact of modal differences. Currently, in order to obtain robust shared pedestrian features, many methods use convolutional networks or Transformer networks as the basic backbone network. For example, Zhu et al. (Y. Zhu, Z. Yang, L. Wang, S. Zhao, X. Hu, D. Tao, Hetero-center loss for cross-modality person re-identification, Neurocomputing 386(2020)97–109.) designed a simple but high-performance network based on a CNN with hetero-center loss to reduce intra-class cross-modal differences, thereby obtaining pedestrian discrimination features. Furthermore, Liang et al. (T.Liang,Y.Jin,Y.Gao,W.Liu,S.Feng,T.Wang,Y.Li,CMTR:Cross-modalityTransformer for visible-infrared person re-identification,arXiv preprintarXiv:2110.08994(2021).) introduced a pure Transformer structure into cross-modal person re-identification to uncover distinctive features of pedestrians. Hybrid models combining convolutional and Transformer structures also compensate for the lack of long-range modeling capabilities in convolutional networks and the insensitivity of Transformers to local features. Chen et al. (C. Chen, M. Ye, M. Qi, J. Wu, J. Jiang, C. Lin, Structure-aware positional Transformer for visible-infrared person re-identification, IEEE Trans. Image Process. 31(2022) 2352–2364.) proposed a structure-aware positional Transformer model, SPOT, combined with CNN to explore the structural features of the human body in different modalities, thereby obtaining modality-invariant features. However, existing cross-modal person re-identification methods still lack in the extraction of multi-scale features. Furthermore, the connections between pedestrian features in different modalities have not been well explored. Summary of the Invention

[0005] The purpose of this invention is to provide a cross-modal pedestrian re-identification method and system based on feature reconstruction, which is beneficial to obtaining more stable, robust and accurate cross-modal pedestrian re-identification results.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a cross-modal person re-identification method based on feature reconstruction, comprising the following steps:

[0007] 1) Extract visible light and infrared images of multiple pedestrians in pairs from the dataset to form visible light training dataset and infrared training dataset;

[0008] 2) Construct a cross-modal pedestrian re-identification network model based on feature reconstruction. The cross-modal pedestrian re-identification network model mainly includes a specific feature extraction module, a multi-scale feature extraction module, a token-aware multi-scale feature fusion module, and a cross-modal feature reconstruction module. The cross-modal pedestrian re-identification network model is trained using visible light training datasets and infrared training datasets to obtain generalizable model parameters.

[0009] 3) The trained cross-modal person re-identification network model is used for cross-modal retrieval to achieve cross-modal person re-identification.

[0010] Further, in step 1), the dataset is the RegDB cross-modal pedestrian re-identification dataset, and M visible light images and M infrared images of N pedestrians are extracted from the RegDB cross-modal pedestrian re-identification dataset in a paired manner.

[0011] Furthermore, in step 2), the implementation method of the cross-modal pedestrian re-identification network model is as follows:

[0012] A) Pedestrian features are extracted from the input visible light image and infrared image using separate specific feature extraction modules, and then the extracted pedestrian features are simultaneously input into the multi-scale feature extraction module;

[0013] B) The multi-scale feature extraction module extracts pedestrian features from visible light images and infrared images at multiple different scales.

[0014] C) The multi-scale pedestrian features are fed into the token-aware multi-scale feature fusion module. The token-aware multi-scale feature fusion module uses a learnable token sequence to model the relationship between multi-scale pedestrian features from both local and global perspectives, reducing the interference of irrelevant pedestrian features at different scales. The bidirectional interaction process from local and global perspectives is repeated multiple times to obtain the final visible light and infrared multi-scale feature relationship map and the visible light and infrared token sequence containing multi-scale information.

[0015] D) The obtained multi-scale feature relationship map is combined with the original pedestrian features and fed into the last feature extraction module of the multi-scale feature extraction module for further feature learning. Then, pooling and horizontal segmentation are performed to obtain the visible light and infrared global features and local features of the pedestrian.

[0016] E) Input the visible light and infrared global features and local features of pedestrians, as well as the visible light and infrared token sequences containing multi-scale information, into the cross-modal feature reconstruction module to perform cross-modal feature reconstruction and discover the relationship between pedestrian features in different modalities;

[0017] F) To reduce the noise generated by pedestrian features during reconstruction, a feature reconstruction loss is constructed. The loss is calculated on the reconstructed features and the target modality features. The error between the two is minimized by an optimizer to enhance the connection between the features of the two modalities.

[0018] Further, in step B), the multi-scale feature extraction module includes four feature extraction modules: Stage-1, Stage-2, Stage-3, and Stage-4. The pedestrian feature size extracted by the specific feature extraction module is 3*288*144. After passing through the first feature extraction module Stage-1, the feature map size is 256*72*36. After passing through the second feature extraction module Stage-2, the feature map size is 512*36*18. After passing through the third feature extraction module Stage-3, the feature map size is 1024*18*9.

[0019] Furthermore, in step C, adaptive pooling is used to unify the pedestrian features at different scales before concatenation. A bidirectional hybrid structure of convolution and Transformer is used to model the multi-scale features of pedestrians, reducing the interference of irrelevant pedestrian features at different scales. Learnable token sequences are used to discover the relationships between the multi-scale features of pedestrians from both local and global perspectives. For the multi-scale features M of pedestrians in visible light... vis The process of fusion, shifting from a local perspective to a global perspective, can be represented as:

[0020] T′ vis =LN(FFN(MHA(T,FL(M vis ), FL(M vis ))))+T)

[0021] Where T represents the learnable token sequence with a quantity of 6, FL represents the operation of flattening the 3D pedestrian features into 2D features, MHA represents the multi-head attention mechanism, FFN represents the feedforward operation, and LN represents the layer normalization operation.

[0022] The process of shifting from a global perspective to a local perspective can be represented as:

[0023] M′ vis =Conv(RS(MHA(FL(M) vis ), T′ vis , T′ vis )+Mvis )

[0024] Where Conv represents the convolution operation, and RS represents the operation of converting two-dimensional features into three-dimensional features.

[0025] Further, in step E), the implementation method of the cross-modal feature reconstruction module is as follows:

[0026] Using the visible light and infrared token sequence T′ containing multi-scale pedestrian information obtained in step C), vis Visible and infrared global features of pedestrians and local features Reconstruction is performed to enhance the relationship between the two modal features; the cross-modal reconstruction of global features yields the following features. Represented as:

[0027]

[0028] Where Attn represents the attention mechanism, T′ ir[0] This indicates the use of the infrared token sequence of the first pedestrian, W. Qh , and This means converting the corresponding features into a Query, Key, and Value matrix; similarly, the formula above... Replace with That is, to obtain the local features after cross-modal feature reconstruction.

[0029] Furthermore, in step F), the specific method for constructing the feature reconstruction loss is as follows: calculate the difference between the reconstructed pedestrian features and the target features to obtain the feature reconstruction loss. Updating the network model using an optimizer is represented as follows:

[0030]

[0031] Where L1 represents the Manhattan distance, N p This indicates the number of pedestrian feature level segments.

[0032] The present invention also provides a cross-modal pedestrian re-identification system based on feature reconstruction, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-mentioned method steps.

[0033] Compared with the prior art, the present invention has the following beneficial effects: The method and system effectively utilize multi-scale feature learning and cross-modal feature reconstruction, which can obtain generalized and robust pedestrian features. It can not only effectively solve the problems of pose change and object occlusion, but also alleviate the negative impact of modal differences on model performance degradation. Attached Figure Description

[0034] Figure 1 This is an architecture diagram of the cross-modal pedestrian re-identification network model based on feature reconstruction in an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0038] This embodiment provides a cross-modal person re-identification method based on feature reconstruction, including the following steps:

[0039] 1) Extract visible light and infrared images of multiple pedestrians in pairs from the dataset to form visible light training dataset and infrared training dataset.

[0040] 2) Construct a cross-modal person re-identification network model based on feature reconstruction. This model mainly includes a specific feature extraction module, a multi-scale feature extraction module, a token-aware multi-scale feature fusion module, and a cross-modal feature reconstruction module. Its architecture is as follows: Figure 1 As shown, the cross-modal pedestrian re-identification network model was trained using visible light and infrared training datasets to obtain generalizable model parameters.

[0041] 3) The trained cross-modal person re-identification network model is used for cross-modal retrieval to achieve cross-modal person re-identification.

[0042] In step 1), the dataset is the RegDB cross-modal pedestrian re-identification dataset. M visible light images and M infrared images of N pedestrians are extracted from the RegDB cross-modal pedestrian re-identification dataset in a paired manner.

[0043] In step 2), the implementation method of the cross-modal person re-identification network model is as follows:

[0044] A) Pedestrian features are extracted from the input visible light image and infrared image using separate specific feature extraction modules, and then the extracted pedestrian features are simultaneously input into the multi-scale feature extraction module.

[0045] B) The multi-scale feature extraction module extracts pedestrian features from visible light and infrared images at multiple different scales.

[0046] C) The multi-scale pedestrian features are fed into the token-aware multi-scale feature fusion module. The token-aware multi-scale feature fusion module uses a small number of learnable token sequences to model the relationship between multi-scale pedestrian features from a local and global perspective, reducing the interference of irrelevant pedestrian features at different scales. The local and global perspective bidirectional interaction process is repeated multiple times to obtain the final visible light and infrared multi-scale feature relationship map and visible light and infrared token sequences containing multi-scale information.

[0047] D) The obtained multi-scale feature relationship map is combined with the original pedestrian features and fed into the last feature extraction module of the multi-scale feature extraction module for further feature learning. Then, pooling and horizontal segmentation are performed to obtain the visible light and infrared global features and local features of the pedestrian.

[0048] E) Input the visible light and infrared global features and local features of pedestrians, as well as the visible light and infrared token sequences containing multi-scale information, into the cross-modal feature reconstruction module to perform cross-modal feature reconstruction and discover the relationship between pedestrian features in different modalities.

[0049] F) To reduce the noise generated by pedestrian features during reconstruction, a feature reconstruction loss is constructed. The loss is calculated on the reconstructed features and the target modality features. The error between the two is minimized by an optimizer to enhance the connection between the features of the two modalities.

[0050] In step B), the multi-scale feature extraction module includes four feature extraction modules: Stage-1, Stage-2, Stage-3, and Stage-4. The pedestrian feature extracted by the specific feature extraction module is 3*288*144 in size. After passing through the first feature extraction module Stage-1, the feature map size is 256*72*36. After passing through the second feature extraction module Stage-2, the feature map size is 512*36*18. After passing through the third feature extraction module Stage-3, the feature map size is 1024*18*9.

[0051] In step C, adaptive pooling is used to scale and unify pedestrian features at different scales before concatenation. A bidirectional hybrid structure of convolution and Transformer is used to model multi-scale pedestrian features, reducing interference from irrelevant pedestrian features at different scales. Learnable token sequences are used to discover relationships between multi-scale pedestrian features from both local and global perspectives. The visible light multi-scale pedestrian features M are used as an example. vis Taking the fusion as an example, the process of shifting from a local perspective to a global perspective can be represented as follows:

[0052] T′ vLs =LN(FFN(MHA(T,FL(M) VLs ),FL(M VLs ))))+T)

[0053] Where T represents the learnable token sequence with a quantity of 6, FL represents the operation of flattening the 3D pedestrian features into 2D features, MHA represents the multi-head attention mechanism, FFN represents the feedforward operation, and LN represents the layer normalization operation.

[0054] The process of shifting from a global perspective to a local perspective can be represented as:

[0055] M′ vis =Conv(RS(MHA(FL(M) vis ), T′ vis , T′ vis )+M vis )

[0056] Where Conv represents the convolution operation, and RS represents the operation of converting two-dimensional features into three-dimensional features.

[0057] In step E), the implementation method of the cross-modal feature reconstruction module is as follows:

[0058] Using the visible light and infrared token sequence T′ containing multi-scale pedestrian information obtained in step C), vis Visible and infrared global features of pedestrians and local features Reconstruction is performed to enhance the relationship between the two modal features; the cross-modal reconstruction of global features yields the following features. Represented as:

[0059]

[0060] Where Attn represents the attention mechanism, T′ ir[0] This indicates the use of the infrared token sequence of the first pedestrian. and This means converting the corresponding features into a Query, Key, and Value matrix; similarly, the formula above... Replace with That is, to obtain the local features after cross-modal feature reconstruction.

[0061] In step F), the specific method for constructing the feature reconstruction loss is as follows: calculate the difference between the reconstructed pedestrian features and the target features to obtain the feature reconstruction loss. Updating the network model using an optimizer is represented as follows:

[0062]

[0063] Where L1 represents the Manhattan distance, N p This indicates the number of pedestrian feature level segments.

[0064] This embodiment also provides a cross-modal pedestrian re-identification system based on feature reconstruction, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-described method steps.

[0065] In this embodiment, the RegDB dataset was used for comparative verification under the setting of pedestrian visible light image search for infrared image. Table 1 shows the comparison results of the proposed method with other cross-modal pedestrian re-identification methods on the RegDB dataset. As can be seen from Table 1, the proposed method has higher accuracy and robustness compared with other cross-modal pedestrian re-identification methods, specifically showing the best Rank-1 and mAP.

[0066] Table 1

[0067]

[0068]

[0069] In Table 1, MAUM corresponds to the method proposed by J. Liu et al. (J. Liu, Y. Sun, F. Zhu, H. Pei, Y. Yang, W. Li, Learning memory-augmented unidirectional metrics for cross-modalityperson re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022, pp. 19366–19375.)

[0070] MPANet corresponds to the method proposed by Q. Wu et al. (Q. Wu, P. Dai, J. Chen, C. Lin, Y. Wu, F. Huang, B. Zhong, R. Ji, Discover cross-modality nuances for visible-infrared person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 4330–4339.)

[0071] NFS corresponds to the method proposed by Y. Chen et al. (Y. Chen, L. Wan, Z. Li, Q. Jing, Z. Sun, Neural feature search for RGB infrared person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 587–597.)

[0072] SPOT corresponds to the method proposed by C. Chen et al. (C. Chen, M. Ye, M. Qi, J. Wu, J. Jiang, C. Lin, Structure-aware positional Transformer for visible-infrared person re-identification, IEEE Trans. Image Process. 31(2022) 2352–2364.)

[0073] AGW corresponds to the method proposed by M. Ye et al. (M. Ye, J. Shen, G. Lin, T. Xiang, L. Shao, SCHoi, Deep learning for person re-identification: A survey and outlook, IEEE Trans. Pattern Anal. Mach. Intell 44(6)(2022)2872–2893.)

[0074] DDAG corresponds to the method proposed by M. Ye et al. (M. Ye, J. Shen, DJ Crandall, L. Shao, J. Luo, Dynamic dual-attentive aggregation learning for visible-infrared person re-identification, in: Proceedings of the European Conference on Computer Vision, 2020, pp. 229–247.)

[0075] D-HSME corresponds to the method proposed by Y. Hao et al. (Y. Hao, N. Wang, J. Li, X. Gao, HSME: Hypersphere manifold embedding for visible thermal person re-identification, in: Proceedings of the AAAI conference on artificial intelligence, 2019, pp. 8385–8392.)

[0076] MSPAC corresponds to the method proposed by C. Zhang et al. (C. Zhang, H. Liu, W. Guo, M. Ye, Multi-scale cascading network with compact feature learning for RGB-infrared person re-identification, in: Proceedings of the IEEE International Conference on Pattern Recognition, 2021, pp. 8679–8686.)

[0077] CMGN corresponds to the method proposed by J. Jiang et al. (J. Jiang, K. Jin, M. Qi, Q. Wang, J. Wu, C. Chen, Across-modal multi-granularity attention network for RGB-IR person re-identification, Neurocomputing 406(2020)59–67.)

[0078] SDL corresponds to the method proposed by K. Kansal et al. (K. Kansal, A.V. Subramanyam, Z. Wang, S. Satoh, SDL: Spectrum disentangled representation learning for visible-infrared person re-identification, IEEE Trans. Circuits Syst. Video Technol. 30(10)(2020) 3422–3432.)

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A feature reconstruction based cross-modal pedestrian re-identification method, characterized in that, The method comprises the following steps: 1) extracting visible light pictures and infrared pictures of multiple pedestrians from a data set in pairs to form a visible light training data set and an infrared training data set; 2) constructing a cross-modal pedestrian re-identification network model based on feature reconstruction, wherein the cross-modal pedestrian re-identification network model comprises a specific feature extraction module, a multi-scale feature extraction module, a Token-aware multi-scale feature fusion module and a cross-modal feature reconstruction module; training the cross-modal pedestrian re-identification network model through the visible light training data set and the infrared training data set to obtain model parameters that can be generalized; 3) using the trained cross-modal pedestrian re-identification network model for cross-modal retrieval to realize cross-modal pedestrian re-identification; In step 2), the implementation method of the cross-modal pedestrian re-identification network model is as follows: A) extracting pedestrian features from input visible light pictures and infrared pictures through independent specific feature extraction modules, and then inputting the extracted pedestrian features into a multi-scale feature extraction module; B) the multi-scale feature extraction module extracts pedestrian features of the visible light pictures and the infrared pictures through multiple feature extraction modules of different scales; C) sending the multi-scale pedestrian features into a Token-aware multi-scale feature fusion module, which adopts a learnable Token sequence to model the relationship between the multi-scale pedestrian features from a local and global perspective in a bidirectional interaction manner, thereby reducing the interference of irrelevant pedestrian features in different scales; repeating the local and global perspective bidirectional interaction process multiple times to obtain a final visible light and infrared multi-scale feature relationship graph and a visible light and infrared Token sequence containing multi-scale information; D) combining the obtained multi-scale feature relationship graph with the original pedestrian features, and sending the combination into the last feature extraction module of the multi-scale feature extraction module for further feature learning, and then performing pooling and horizontal segmentation to obtain visible light and infrared global features and local features of the pedestrian; E) inputting the visible light and infrared global features and local features of the pedestrian and the visible light and infrared Token sequence containing multi-scale information into a cross-modal feature reconstruction module for cross-modal feature reconstruction to explore the relationship between the pedestrian features in different modalities; F) in order to reduce the noise generated by the pedestrian features in the reconstruction process, a feature reconstruction loss is constructed to calculate the loss of the reconstructed features and the target modality features, and an optimizer is used to minimize the error between the two to enhance the relationship between the features in the two modalities; In step B), the multi-scale feature extraction module comprises four feature extraction modules, namely Stage-1, Stage-2, Stage-3 and Stage-4; The size of the pedestrian features extracted by the specific feature extraction module is 3*288*144, the size of the feature map is 256*72*36 after passing through the first feature extraction module Stage-1, the size of the feature map is 512*36*18 after passing through the second feature extraction module Stage-2, and the size of the feature map is 1024*18*9 after passing through the third feature extraction module Stage-3. In step C, different scale features of pedestrians are unified by adaptive pooling and then spliced, and a bidirectional hybrid structure of convolution and Transformer is used to model the multi-scale features of pedestrians, reducing the interference of irrelevant features of pedestrians at different scales; The multi-scale features of the pedestrian are locally and globally related by using a learnable Token sequence; and The process of switching from a local perspective to a global perspective is represented as: wherein, is represented as a learnable Token sequence, the number is set to 6, represents a two-dimensional feature operation of flattening three-dimensional pedestrian features, represents a multi-head attention mechanism, represents a forward feedback operation, represents a layer normalization operation; The process of turning from a global perspective to a local perspective is represented as: wherein, denotes a convolution operation, denotes a two-dimensional feature to three-dimensional feature operation; In step E, the implementation method of the cross-modal feature reconstruction module is: The visible, infrared token sequence comprising pedestrian multiscale information obtained in step C) is reconstructed The visible, infrared global feature of the pedestrian and local feature is reconstructed to enhance the relationship between the two modal features. The feature obtained by the cross-modal reconstruction of the global feature is denoted as: wherein, denotes the attention mechanism, denotes the use of the first pedestrian's infrared Token sequence, , and denotes the conversion of the respective features into Query, Key and Value matrices; by analogy, replacing with in the above formula, we obtain the local feature reconstructed across modalities.

2. The feature reconstruction based cross-modal pedestrian re-identification method according to claim 1, characterized in that, In step 1, the data set is a RegDB cross-modal pedestrian re-identification data set, and M visible light pictures and M infrared pictures of N pedestrians are extracted from the RegDB cross-modal pedestrian re-identification data set in pairs.

3. The feature reconstruction based cross-modal pedestrian re-identification method according to claim 1, characterized in that, In step F), the specific method of constructing the feature reconstruction loss is: calculating the difference between the reconstructed pedestrian feature and the target feature to obtain the feature reconstruction loss updating the network model by using the optimizer, denoted as: wherein, denotes the Manhattan distance, denotes the number of pedestrian feature level splits.

4. A feature reconstruction based cross-modal pedestrian re-identification system characterized in that, The computer program instructions stored in the memory and capable of being executed by the processor can implement the method of any one of claims 1-3 when the processor executes the computer program instructions. The computer program instructions stored in the memory and capable of being executed by the processor can implement the method of any one of claims 1-3 when the processor executes the computer program instructions.

Images