Cross-modal pedestrian re-identification method based on spatial data enhancement and symmetric mutual attention

Abstract

The application discloses a cross-modal pedestrian re-identification method based on spatial data enhancement and symmetric mutual attention, and the implementation scheme is as follows: for the spatial data enhancement method of the cross-modal pedestrian, the information related to the pedestrian is generated through the way of partition image mixing; a double-flow deep learning model for the cross-modal pedestrian re-identification is constructed, and the mixed image is decomposed into modality sharing features and modality specific features; the symmetric mutual attention module is used to fuse the modality sharing features and the modality specific features, and more discriminative pedestrian features are generated. Under the premise of not introducing additional information, the application improves the discrimination accuracy of the model and enhances the search ability of the pedestrian under different modalities through spatial data enhancement and feature information compensation, and can be used in the fields of security monitoring, automatic driving, album classification and human-computer interaction.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and further relates to a cross-modal pedestrian re-identification method based on spatial data enhancement and symmetric mutual attention in the technical field of image recognition. The present application can be used for identifying whether the pedestrian images obtained from monitoring videos of different cameras and different angles in security monitoring and intelligent transportation are the same pedestrian. BACKGROUND

[0002] Pedestrian re-identification aims to accurately identify and distinguish different pedestrian identities from images or videos, and match and identify images or videos of the same pedestrian obtained at different cameras or time periods. Cross-modal pedestrian re-identification refers to the task of identifying pedestrian identities between different perception modalities. The goal of this task is to achieve accurate pedestrian identity recognition by extracting shared features and specific modal features of pedestrians. However, due to the differences in color, background, and light intensity of the same pedestrian under visible light cameras and infrared cameras, the posture, appearance, and occlusion of the obtained pedestrians also differ, which brings great difficulty and challenge to the task of cross-modal pedestrian re-identification.

[0003] Wang et al. proposed a method for cross-modal pedestrian re-identification using image instance modal compensation JSIA in their published paper "Cross-modality paired-images generation for RGB-infrared person re-identification" (2020, published in AAAI, volume 7, volume 34). The implementation steps of this method are: false pedestrian image generation, dual-modal information integration, color information merging, and cross-modal pedestrian target recognition. By generating corresponding modal false images through a generative adversarial network, the method compensates for the changes in modal information in cross-modal pedestrian re-identification. Secondly, the generated images and real pedestrian images are input into a neural network, so that the network model cannot distinguish modal information. The disadvantage of this method is that the generation of false images is a global property and does not distinguish pedestrian information, resulting in low quality of generated false images, introducing too much noise information, and causing misclassification of pedestrian categories in cross-modal pedestrian re-identification.

[0004] A cross-modal pedestrian re-identification method based on channel reorganization and attention mechanism is proposed in the patent document "Cross-modal pedestrian re-identification method based on channel reorganization and attention mechanism" (application number CN 202210651623.X, application publication number CN 115100678 A) applied by Henan University. The steps implemented by this method are: constructing a pedestrian feature vector, feature matching, model training, channel reorganization, and similarity matching. This method first extracts the feature vector of the input data and performs feature matching on the cross-modal pedestrian re-identification model; uses grouping convolution and channel reorganization operations to mine the shared features of pedestrian images in infrared and visible light modalities; finally, uses the attention mechanism to mine the local features of pedestrians, and uses the feature vector similarity to perform feature matching. The deficiency of this method is that the model ignores the available pedestrian information in the modal-specific features, resulting in partial loss of pedestrian information, which reduces the recognition accuracy in the cross-modal pedestrian re-identification task. SUMMARY

[0005] The present application aims to address the deficiencies of the prior art by proposing a cross-modal pedestrian re-identification method based on spatial data enhancement and symmetric mutual attention, which solves the problem of high pedestrian misjudgment rate caused by generating additional noise information in different modalities and the loss of pedestrian information in modal-specific features.

[0006] The idea of achieving the purpose of the present application is that the present application unifies the read-in visible light pedestrian image and infrared pedestrian image to the size of [144, 288], and performs segmentation operation on the picture to obtain 288 [12, 12] image blocks. The obtained image blocks are divided into central region, sub-central region and outer ring region, and different mixing probabilities are set for these three regions respectively to perform image mixing operation to obtain spatial data enhanced visible light pedestrian picture and infrared pedestrian picture, which reduces the introduction of irrelevant noise, thereby solving the problem of high pedestrian misjudgment rate caused by generating additional noise information in the cross-modal pedestrian re-identification task. The present application uses a double-flow deep learning network to design a modal decomposition loss function to constrain the mixed pedestrian picture to be decomposed into modal shared features and modal unique features, ensuring the difference between the features. The present application designs a mutual attention mechanism to fuse the pedestrian information in the modal shared features and the modal unique features, uses a group of convolution layers to project the features into a computable matrix, cross-multiplies and fuses the computable matrix to obtain a mixed feature, sets a learnable parameter, and fuses the mixed feature, thereby solving the problem of loss of pedestrian information in modal-specific features.

[0007] The specific steps of the present application include the following:

[0008] Step 1, construct a symmetric mutual attention module:

[0009] The symmetric cross attention module is built by connecting the modal specific branch and the modal sharing branch in parallel, and then connecting the cross point multiplication unit and the fusion unit in series; the modal specific branch and the modal sharing branch have the same structure and are composed of the first, second and third convolution layers in parallel; the cross point multiplication unit is composed of a point multiplication operation and a Softmax classification layer in series; the convolution kernel size of the first to third multiplication layers is set to 1*1, and the step size is set to 1;

[0010] Step 2, build a main network of a double-flow deep learning network by connecting an input layer, a first convolution unit, a pooling unit, a residual combination block, and a second convolution unit in series, and set the parameters of the double-flow deep learning main network;

[0011] Step 3, build a double-flow deep learning subnetwork:

[0012] The double-flow deep learning subnetwork is built by connecting an infrared light module and a visible light module in parallel; the visible light module and the infrared light module are both composed of a third convolution unit, a fourth convolution unit, a fifth convolution unit, a full connection unit, and a Softmax classification unit; and the parameters of the double-flow deep learning subnetwork are set.

[0013] Step 4, build a double-flow deep learning network:

[0014] The second convolution unit of the main network and the third convolution unit of the visible light module of the subnetwork are cascaded as a first branch; the second convolution unit of the main network and the third convolution unit of the infrared light module are cascaded as a second branch; the first branch, the second branch, the fourth convolution unit of the subnetwork, and the fifth convolution unit of the subnetwork are connected in series to form a third branch; the third branch and the modal sharing branch of the symmetric cross attention module are connected in series to form a fourth branch; the third branch and the modal specific branch of the symmetric cross attention module are connected in series to form a fifth branch; the fourth branch and the fifth branch, the cross point multiplication unit of the symmetric cross attention module, and the Softmax classification layer of the symmetric cross attention module are cascaded to form a double-flow deep learning network.

[0015] Step 5, generate a spatial data enhanced cross-modal pedestrian training set:

[0016] Select a continuous 20-second pedestrian visible light and infrared light video; randomly select one frame of image from each of the two videos; crop the smallest rectangular image of the area occupied by each pedestrian from the selected image; normalize all rectangular images to [288, 144] pixels; label all images of the same pedestrian in the visible light and infrared light pedestrian images as the real label of the same class of pedestrian images; each class contains at least one pedestrian image; and perform segmentation, partitioning, and mixing operations on the labeled pedestrian images in sequence to obtain a spatial data enhanced cross-modal pedestrian training set.

[0017] Step 6, training a double-flow deep learning network:

[0018] The cross-modal pedestrian training set is input into the double-flow deep learning network, the weights of the network are iteratively updated using the Adam optimizer until the network loss function converges, and a trained network is obtained;

[0019] Step 7, input the image of the query target pedestrian to obtain the matching image:

[0020] After the image of the query target pedestrian is subjected to the same segmentation operation, partition operation and mixing operation as in step 5, the output features are input into the trained double-flow deep learning network to calculate the Euclidean distance between the target pedestrian image feature vector and the pedestrian image feature vector obtained by training, and the top 20 candidate pedestrian images after sorting are used as the matching images after pedestrian re-identification.

[0021] Compared with the prior art, the present application has the following advantages:

[0022] First, the present application generates mixed pictures after spatial data enhancement of the read-in visible light pedestrian image and infrared pedestrian image, overcomes the defect of high pedestrian misjudgment rate caused by additional noise information generated under different modalities in the prior art in cross-modal pedestrian re-identification, and reduces the misjudgment rate of cross-modal pedestrian re-identification.

[0023] Second, the present application constructs a symmetric mutual attention module to fuse the pedestrian information in the modal shared features and modal unique features decomposed from the double-flow deep learning network, overcomes the lack of pedestrian information in the modal specific features in the prior art, and makes the pedestrian information in the modal shared features and modal unique features combined in the present application, thereby improving the recognition accuracy in cross-modal pedestrian re-identification. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is the implementation flowchart of the present application;

[0025] Figure 2 is a spatial data enhancement schematic diagram in the present application;

[0026] Figure 3 is a schematic diagram of the symmetric mutual attention module in the present application;

[0027] Figure 4 is a network structure diagram in the present application. DETAILED DESCRIPTION

[0028] The present application will be further described in detail below in combination with the drawings and examples.

[0029] Referring to Figure 1 , the implementation steps of the embodiment of the present application are further described in detail.

[0030] Step 1, construct a symmetric cross-attention module:

[0031] Reference Figure 2 , the symmetric cross-attention module of the present application is described.

[0032] Build a symmetric cross-attention module composed of a modal-specific branch and a modal-shared branch in parallel, and then in series with a cross-point multiplication unit and a fusion unit; the modal-specific branch and the modal-shared branch have the same structure and are composed of first, second and third convolution layers in parallel, the cross-point multiplication unit is composed of a point multiplication operation and a Softmax classification layer in series; the convolution kernel size of the first to third multiplication layers is set to 1x1, and the step is set to 1; two branches can obtain two three-channel features, six channels are represented as Q sh , K sh , V sh and Q sp , K sp , V sp , through the cross-point multiplication unit, the mixed features are obtained by using the formula and , F sp-sh and F sh-sp represent the fused pedestrian features, d represents the feature dimension, and the fusion unit uses the formula F C = ω1F sp-sh + ω2F sh-sp to fuse the two mixed features, where ω1 and ω2 are weight parameters, and F C represents the fused pedestrian features.

[0033] Step 2, build a main network of a dual-flow deep learning network composed of an input layer, a first convolution unit, a pooling unit, a residual combination block, and a second convolution unit in series, and set the parameters of the dual-flow deep learning main network; the main network can perform pre-perception processing on the input visible light pedestrian image and infrared light pedestrian image to generate modal primary features, which contain shallow information of pedestrians.

[0034] The first convolution unit is composed of 3 serial convolution layers; the convolution kernel size of the 3 convolution layers of the first convolution unit is set to 7x7, and the step is set to 2; the pooling layer is a maximum pooling, the sliding window size of the pooling layer is set to 3x3, the step is set to 2, and the padding size is 1; the residual combination block is composed of 16 residual blocks in series, the convolution kernel size of the residual blocks in the residual combination block is set to 1x1, and the step is set to 1; the second convolution unit is composed of 64 serial convolution layers, the convolution kernel size of the 64 convolution layers of the second convolution unit is set to 3x3, the step is set to 1, and the padding size is set to 1.

[0035] Step 3, constructing a dual-flow deep learning subnetwork:

[0036] The dual-flow deep learning subnetwork is built by connecting the infrared light module and the visible light module in parallel, and the visible light module and the infrared light module are both composed of a third convolution unit, a fourth convolution unit, a fifth convolution unit, a full connection unit and a Softmax classification unit, and the parameters of the dual-flow deep learning subnetwork are set. The modal primary features generated by the main network are respectively input into the visible light module and the infrared light module of the dual-flow deep learning subnetwork, and the cross-modal primary features are decomposed into modal shared features and modal specific features.

[0037] The third convolution unit, the fourth convolution unit and the fifth convolution unit are respectively composed of 128, 256 and 512 serial convolution layers; the full connection unit is composed of 6 parallel full connection layers; the convolution kernel sizes of the third to fifth convolution units are respectively set to 3*3, 3*3 and 1*1, and the step sizes are all set to 1.

[0038] Step 4, constructing a dual-flow deep learning network:

[0039] The second convolution unit of the main network and the third convolution unit of the visible light module of the subnetwork are cascaded as a first branch, and the second convolution unit of the main network and the third convolution unit of the infrared light module are cascaded as a second branch; the first branch, the second branch, the fourth convolution unit of the subnetwork and the fifth convolution unit of the subnetwork are sequentially connected to form a third branch, the third branch and the modal shared branch of the symmetric cross-attention module are connected to form a fourth branch, the third branch and the modal specific branch of the symmetric cross-attention module are connected to form a fifth branch, and the fourth branch, the fifth branch, the cross-point multiplication unit of the symmetric cross-attention module and the Softmax classification layer of the symmetric cross-attention module are cascaded to form a dual-flow deep learning network.

[0040] Step 5, generating a cross-modal pedestrian training set after spatial data enhancement:

[0041] Reference Figure 3 The spatial data enhancement in the present application is described as follows:

[0042] Selecting continuous 20-second pedestrian visible light and infrared light videos, randomly selecting one frame of image from each of the two videos, cropping the minimum rectangular image of the area occupied by each pedestrian from the selected image, normalizing all the rectangular images to [288, 144] pixels, labeling all the images of the same pedestrian in the visible light and infrared light pedestrian images as the real labels of the same class of pedestrian images, each class containing at least one pedestrian image, and sequentially performing segmentation operation, partition operation and mixing operation on the labeled pedestrian images to obtain a cross-modal pedestrian training set after spatial data enhancement.

[0043] The segmentation operation refers to cutting the labeled pedestrian image into image blocks with a size of [12, 12], so that the pedestrian picture is uniformly divided into 288 image blocks; the division operation refers to dividing each segmented image block into a center region, a sub-center region and an outer ring region, the center region contains 64 image blocks, the sub-center region contains 96 image blocks, and the outer ring region contains 128 image blocks; the mixing operation refers to generating an image mask according to the probability of the appearance of pedestrians obtained by counting the divided image blocks, and using the formula The image mask is mixed, wherein, represents the spatial data enhanced image of the Pth pedestrian in the Vth visible light image, ∑ represents the summation operation, M i represents the Ith image mask in the region set χ, represents the dot product operation, represents the image block of the Pth pedestrian in the Ith infrared light image, represents the image block of the Pth pedestrian in the Vth visible light image.

[0044] Step 6, training the dual-flow deep learning network:

[0045] Referring to Figure 4 , the network training structure of the application is described:

[0046] The cross-modal pedestrian training set is input into the dual-flow deep learning network, the Adam optimizer is used to iteratively update the weights of the network, until the network loss function converges, and the trained network is obtained;

[0047] The network loss function is L Total =L ID +L WRT +α*L MFD , wherein L ID represents the pedestrian classification loss function, L WRT represents the weighted triplet loss function, L MFD represents the feature decomposition loss function, and α represents a weight parameter with a value of 0.2; the pedestrian classification loss function L ID constrains the accuracy of classification of pedestrians in each modality in the feature extraction process, the weighted triplet loss function L WRT reduces the difference between modalities and enhances the correlation between positive samples; the feature decomposition loss function L MFD ensures the difference between modality shared features and modality specific features.

[0048] The pedestrian classification loss function L ID calculates the cross-entropy loss using the pedestrian class generated by the network and the real label, and the formula is Wherein, Y represents a real pedestrian label, P(F C ) represents a pedestrian label predicted by a network, and CE(*) represents a cross-entropy loss; the weighted triplet loss function L WRT , by learning feature vectors of different categories, positive samples of the same category are closer in the feature space, and negative samples of different categories are far away from each other, and the formula is Wherein and represent a positive sample set and a negative sample set under the current sample, and represent the Euclidean distance from the current sample to the positive sample and the negative sample, and W P,i and W N,i represent the weight of the distance; the feature decomposition loss function is , represents the distance between the Pth pedestrian modal shared features, , represents the distance between the Pth pedestrian modal specific features, , represents the distance between the Pth pedestrian modal specific features and the modal shared features.

[0049] Step 7, input the image of the query target pedestrian to obtain a matching image:

[0050] After the image of the query target pedestrian is subjected to the same segmentation operation, partition operation and mixing operation as in step 5, the output features obtained by inputting the image into the trained double-flow deep learning network are inputted to calculate the Euclidean distance between the target pedestrian image feature vector and the pedestrian image feature vector obtained by training, and the top 20 candidate pedestrian images after sorting are taken as the matching images after pedestrian re-identification.

[0051] The effect of the present application will be further described in combination with a simulation experiment:

[0052] 1. Simulation experiment conditions:

[0053] The hardware platform of the simulation experiment of the present application is: the processor is Intel i7 5930k CPU, the main frequency is 3.5GHz, and the memory is 24GB.

[0054] The software platform of the simulation experiment of the present application is: Windows 10 operating system and python 3.6.

[0055] The input image used in the simulation experiment of the present application is SYSU-MM01 cross-modal pedestrian re-identification image, which contains pedestrian data captured by 4 visible light cameras and 2 infrared light cameras, and the image contains 296 pedestrians, and the image format is jpg.

[0056] 2. Simulation content and result analysis:

[0057] The simulation experiment of the present application is to respectively perform re-identification on the input SYSU-MM01 image by using the present application and three prior arts (end-to-end alignment generative adversarial network AlignGAN, pixel-level image alignment network LbA, and similarity inference measurement network SIM), and obtain classification accuracy.

[0058] In the simulation experiment, the three prior arts used are:

[0059] The existing end-to-end alignment generative adversarial network AlignGAN refers to the cross-modal pedestrian re-identification method proposed by Mao et al. in "Mao, Xudong, Qing Li, and Haoran Xie. Aligngan: Learning to align cross-domain images with conditional generative adversarial networks. arXiv preprint arXiv:1707.01400 (2017)." and is referred to as end-to-end alignment generative adversarial network AlignGAN.

[0060] The existing pixel-level image alignment network LbA refers to the cross-modal pedestrian re-identification method proposed by Park et al. in "Park, Hyunjong, Sanghoon Lee, Junghyup Lee, and Bumsub Ham. Learning by aligning: Visible-infrared person re-identification using cross-modal correspondences. In Proceedings of the IEEE / CVF international conference on computer vision, pp. 12046-12055. 2021." and is referred to as pixel-level image alignment network LbA.

[0061] The existing similarity inference metric network SIM refers to the cross-modality person re-identification method proposed by Jia et al. in "Jia, Mengxi, Yunpeng Zhai, Shijian Lu, Siwei Ma, and Jian Zhang. A similarity inference metric for RGB-infrared cross-modality person re-identification. arXiv preprint arXiv:2007.01504 (2020)."

[0062] The classification results of the three methods are evaluated respectively by using three evaluation indexes (person first ranking accuracy, person tenth ranking accuracy, and person category classification accuracy). The person first ranking accuracy Rank1, the person tenth ranking accuracy Rank10, and the person category classification accuracy mAP are calculated by using the following formula, and all the calculation results are drawn into Table 1.

[0063]

[0064]

[0065]

[0066] Table 1. Quantitative analysis table of classification results of the present application and each prior art in simulation experiment

[0067]

[0068] It can be seen from Table 1 that the person first ranking accuracy Rank1 of the present application is 67.42%, the person tenth ranking accuracy Rank10 is 94.61%, and the person category classification accuracy Map is 62.23%. These three indexes are higher than those of the three prior art methods, proving that the present application can obtain higher cross-modality person re-identification accuracy.

[0069] The above simulation experiment shows that the method of the present application can extract person features by using the built double-flow deep learning network, can effectively complete the missing modality specific information of the person by using the symmetric mutual attention module, and can effectively mix the visible light person image and the infrared light person image by using the spatial data enhancement, so that the network pays more attention to the person region, thereby achieving lower misjudgment rate and higher recognition accuracy.

Claims

1. A cross-modal pedestrian re-identification method based on spatial data enhancement and symmetric attention, characterized in that, The symmetric mutual attention module is constructed to generate a cross-modal pedestrian training set after spatial data enhancement. Step 1, constructing a symmetric mutual attention module: A symmetric mutual attention module is built by connecting a modal-specific branch and a modal-shared branch in parallel, and then connecting a cross-point multiplication unit and a fusion unit in sequence; the modal-specific branch and the modal-shared branch have the same structure and are composed of first, second and third convolution layers in parallel; the cross-point multiplication unit is composed of a point multiplication operation and a Softmax classification layer in series; the convolution kernel size of the first to third multiplication layers is set to 1*1, and the step is set to 1; Step 2, building a main network of a double-flow deep learning network by connecting an input layer, a first convolution unit, a pooling unit, a residual combination block, and a second convolution unit in sequence, and setting the parameters of the double-flow deep learning main network; Step 3, constructing a double-flow deep learning subnetwork: A double-flow deep learning subnetwork is built by connecting an infrared light module and a visible light module in parallel; the visible light module and the infrared light module are composed of a third convolution unit, a fourth convolution unit, a fifth convolution unit, a full connection unit, and a Softmax classification unit, and the parameters of the double-flow deep learning subnetwork are set; Step 4, constructing a double-flow deep learning network: The second convolution unit of the main network and the third convolution unit of the visible light module of the subnetwork are cascaded as a first branch, and the second convolution unit of the main network and the third convolution unit of the infrared light module are cascaded as a second branch; the first branch, the second branch, the fourth convolution unit of the subnetwork, and the fifth convolution unit of the subnetwork are connected in sequence to form a third branch; the third branch and the modal-shared branch of the symmetric mutual attention module are connected in series to form a fourth branch; the third branch and the modal-specific branch of the symmetric mutual attention module are connected in series to form a fifth branch; the fourth branch, the fifth branch, the cross-point multiplication unit of the symmetric mutual attention module, and the Softmax classification layer of the symmetric mutual attention module are cascaded to form a double-flow deep learning network; Step 5, generating a cross-modal pedestrian training set after spatial data enhancement: Selecting 20 seconds of continuous pedestrian visible light and infrared light videos, randomly selecting one frame of image from each of the two videos, cropping the minimum rectangular image of each pedestrian area from the selected image, normalizing all rectangular images to [288, 144] pixels, labeling all images of the same pedestrian in the visible light and infrared light pedestrian images as the same class of pedestrian image, and each class containing at least one pedestrian image, performing segmentation operation, partition operation and mixing operation on the labeled pedestrian images to obtain a cross-modal pedestrian training set after spatial data enhancement; Step 6, training the double-flow deep learning network: The cross-modal pedestrian training set is input into the double-flow deep learning network, the Adam optimizer is used to iteratively update the weights of the network until the network loss function converges, and a trained network is obtained; Step 7, inputting the image of the query target pedestrian to obtain a matching image: After the image of the query target pedestrian is segmented, partitioned and mixed through the same operation as step 5, the output feature obtained by inputting the image into the trained dual-flow deep learning network is calculated to obtain the Euclidean distance between the image feature vector of the target pedestrian and the pedestrian image feature vector obtained by training, and the top 20 candidate pedestrian images in the order are taken as the matching images after pedestrian re-identification.

2. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The first convolution unit in step 2 is composed of three serial convolution layers; the convolution kernel size of the three convolution layers of the first convolution unit is all set to 7*7, and the step is all set to 2.

3. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The pooling layer in step 2 is maximum pooling, the sliding window size of the pooling layer is set to 3*3, the step is set to 2, and the padding size is 1.

4. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The residual combination block in step 2 is composed of 16 residual blocks in series, and the convolution kernel size of the residual blocks in the residual combination block is all set to 1*1, and the step is all set to 1.

5. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The second convolution unit in step 2 is composed of 64 serial convolution layers, and the convolution kernel size of the 64 convolution layers of the second convolution unit is all set to 3*3, the step is all set to 1, and the padding size is 1.

6. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The third convolution unit, the fourth convolution unit and the fifth convolution unit in step 3 are respectively composed of 128, 256 and 512 serial convolution layers; the full connection unit is composed of 6 parallel full connection layers; the convolution kernel size of the third to fifth convolution units is respectively set to 3*3, 3*3 and 1*1, and the step is all set to 1.

7. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The segmentation operation in step 5 refers to cutting the labeled pedestrian image into image blocks with a size of [12, 12], so that the pedestrian picture is uniformly segmented into 288 image blocks.

8. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The partition operation in step 5 refers to dividing each segmented image block into a center region, a sub-center region and an outer ring region, the center region contains 64 image blocks, the sub-center region contains 96 image blocks, and the outer ring region contains 128 image blocks.

9. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention of claim 1, wherein, The mixing operation described in step 5 refers to generating an image mask according to the probability of the appearance of pedestrians obtained by statistics of the partitioned image blocks, using the formula The image mask is mixed, wherein, represents the spatial data enhanced image of the Pth pedestrian in the Vth visible light image, ∑ represents the summation operation, M i represents the Ith image mask in the region set χ, represents the point multiplication operation, represents the image block of the Pth pedestrian in the Ith infrared light image, the image block of the Pth pedestrian in the Vth visible light image.

10. The cross-modal pedestrian re-identification method based on spatial data augmentation and symmetric attention according to claim 1, characterized in that, The network loss function described in step 6 is L Total = L ID + L WRT + a * L MFD , wherein L ID represents a pedestrian classification loss function, L WRT represents a weighted triplet loss function, L MFD represents a feature decomposition loss function, and a represents a weight parameter with a value of 0.2; the feature decomposition loss function is represents a distance between Pth pedestrian modal shared features, represents a distance between Pth pedestrian modal specific features, represents a distance between Pth pedestrian modal specific features and modal shared features.

Images