A cross-modal person re-identification method and system based on distribution space alignment
By adopting the distributed spatial alignment method in cross-modal pedestrian recognition, the problem of large differences between visible light and infrared images is solved, the recognition accuracy and anti-interference ability are improved, and the dependence of manual labeling data is reduced.
Patent Information
- Application Number
- CN202111401063.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-24
- Publication Date
- 2025-04-15
- Estimated Expiration
- 2041-11-24
AI Technical Summary
The prior art is difficult to effectively handle the differences between visible light and infrared images in cross-modal pedestrian re-identification, and requires a large number of manually labeled data sets, resulting in high costs and high task difficulty.
A cross-modal pedestrian re-identification method based on distribution space alignment is adopted. By constructing a feature extractor and a modal sharing model, self-supervised and cross-modal stage learning is used to reduce the distribution distance between visible light and infrared modes, and the model is corrected through a small amount of real-named data.
Improve the accuracy and anti-interference ability of cross-modal pedestrian re-identification in complex environments, reduce the dependence on manual labeled data, and improve the recognition efficiency and accuracy.
Smart Images
Figure CN114419529B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to a cross-modal pedestrian re-identification method and system based on distribution space alignment. Background Art
[0002] Single-mode pedestrian re-identification based on visible light is only applicable to scenes with sufficient lighting. When the light is insufficient, infrared cameras need to be used to take infrared pedestrian images. Therefore, it is necessary to solve the problem of cross-modal pedestrian re-identification of visible light and infrared, and to face the problem of large differences between visible light images and infrared images. In addition, supervised models require a large number of manually annotated cross-modal datasets, and the cost of cross-modal pedestrian identity annotation is much higher than that of single-mode, and the task is more difficult. Summary of the invention
[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a cross-modal pedestrian re-identification method and system based on distribution space alignment, which can perform cross-modal pedestrian re-identification tasks in both visible light and infrared modes and provide recognition capabilities in complex environments.
[0004] The first technical solution adopted by the present invention is: a cross-modal pedestrian re-identification method based on distribution space alignment, comprising the following steps:
[0005] Constructing a feature extractor and training the feature extractor based on the first training sample to obtain a trained feature extractor;
[0006] Acquire the modal image and extract pedestrian modal features based on the trained feature extractor;
[0007] Cluster the images to be tested according to pedestrian modal features to obtain a dataset with pseudo annotations;
[0008] Constructing a modality sharing model and training the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model;
[0009] Training the first modality sharing model based on the second training sample to obtain a trained modality sharing model;
[0010] The input image is recognized according to the trained modality sharing model to complete the pedestrian re-identification task.
[0011] Furthermore, the first training sample is a training sample with real annotations, and the step of constructing a feature extractor and training the feature extractor based on the first training sample to obtain a trained feature extractor specifically includes:
[0012] Construct visible light and infrared modality feature extractors based on ResNet50;
[0013] The feature extractor is trained based on the first training sample and the model parameters are adjusted according to the feature loss function to obtain a trained feature extractor.
[0014] Furthermore, the visible light and infrared modal feature extractors respectively include three branch networks with different granularities, and the branch networks share the ResNet50 backbone network. The step of acquiring the modal image and extracting the pedestrian modal features based on the trained feature extractor specifically includes:
[0015] Obtain dataset images of visible light and infrared corresponding modalities, and obtain convolution features through the backbone network;
[0016] The convolutional features are respectively passed into three branch networks with different granularities for maximum pooling to obtain block features of corresponding granularities;
[0017] The block features of the corresponding granularity are reduced in dimension through a 1x1x256 convolution kernel to obtain the reduced-dimensional features;
[0018] The reduced-dimensional features are input into the fully connected layer at the end of the branch network for classification, and the pedestrian modal features are output.
[0019] Furthermore, the step of clustering the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations specifically includes:
[0020] Cluster the modal images according to pedestrian modal features and density clustering algorithm to obtain pseudo-annotations of the modal images;
[0021] A dataset with pseudo annotations is generated according to the pseudo annotations of the modality images and the corresponding modality images.
[0022] Furthermore, the step of constructing a modality sharing model and training the modality sharing model based on the pseudo-annotated data set to obtain a first modality sharing model specifically includes:
[0023] Construct a modality sharing model and train it based on a dataset with pseudo annotations to obtain the corresponding modality classification output and pedestrian depth features;
[0024] Calculate the cross entropy loss function based on the modal classification output;
[0025] Calculate the triplet loss function based on the pedestrian depth features;
[0026] Calculate the modal distribution space alignment loss function based on pedestrian depth features;
[0027] The parameters of the modality sharing model are adjusted by combining the cross entropy loss function, the triplet loss function and the modality distribution space alignment loss function to obtain the first modality sharing model.
[0028] Furthermore, the calculation formula of the cross entropy loss function is as follows:
[0029]
[0030] In the above formula, F(·) represents the classification output of the modality sharing model, C represents the number of pedestrian categories, X represents the input image, y represents the pedestrian pseudo-annotation generated in the self-supervision stage, j represents the pedestrian pseudo-annotation corresponding to the image, and e represents a natural constant.
[0031] Furthermore, the formula for calculating the modal distribution space alignment loss function based on pedestrian depth features is expressed as follows:
[0032]
[0033] In the above formula, ||·|| p represents the p-order norm, Φ(F(X RGB ),F(X IR )) means including all and F(X RGB) ,F(X IR ) is the set of all joint distributions of the boundary distribution, X RGB represents the visible light modality sample, X IR represents the near-infrared modal sample, p represents the norm, x represents the sample of the visible light modal distribution space, and x' represents the sample of the near-infrared modal distribution space.
[0034] Furthermore, the step of training the first modality sharing model based on the second training sample as a training sample with a real annotation to obtain a trained modality sharing model specifically includes:
[0035] Generate pseudo positive sample pairs based on the first modality shared model;
[0036] Preliminarily correcting the first modality sharing model according to the pseudo-positive sample pair to obtain a preliminarily corrected modality sharing model;
[0037] The initially corrected modality sharing model is trained based on the second training sample to obtain a trained modality sharing model.
[0038] Furthermore, the step of generating a pseudo positive sample pair based on the first modality sharing model specifically includes:
[0039] According to the first modality sharing model, a distance matrix between same-modality samples and a distance matrix between cross-modality samples are obtained;
[0040] Select two cross-modal sample images and calculate k same-modal nearest neighbors and k cross-modal nearest neighbors of the selected samples according to the distance matrix between same-modal samples and the distance matrix between cross-modal samples, and obtain k infrared modal neighbors of the infrared modal sample image, k visible light modal neighbors of the infrared modal sample image, k infrared modal neighbors of the visible light modal sample image, and k visible light modal neighbors of the visible light modal sample image;
[0041] Calculate the Jaccard similarity between the k infrared modal neighbors of the infrared modal sample image and the k infrared modal neighbors of the visible light modal sample image, and the Jaccard similarity between the k visible light modal neighbors of the infrared modal sample image and the k visible light modal neighbors of the visible light modal sample image, and weight them to obtain the total Jaccard similarity of the cross-modal sample pairs;
[0042] Sort the total Jaccard similarity according to the cross-modal samples to obtain the cross-modal neighbor sequence of each sample;
[0043] A preset number of images are taken according to the cross-modal neighbor sequence of each sample to obtain cross-modal pseudo-positive sample pairs.
[0044] The second technical solution adopted by the present invention is: a cross-modal pedestrian re-identification system based on distribution space alignment, comprising:
[0045] A first construction module is used to construct a feature extractor and train the feature extractor based on the first training sample to obtain a trained feature extractor;
[0046] A feature extraction module, used to obtain modal images and extract pedestrian modal features based on the trained feature extractor;
[0047] The clustering module is used to cluster the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations;
[0048] A second construction module is used to construct a modality sharing model and train the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model, and train the first modality sharing model based on a second training sample to obtain a trained modality sharing model;
[0049] The recognition module is used to recognize the input image based on the trained modality sharing model and complete the pedestrian re-identification task.
[0050] The beneficial effects of the method and system of the present invention are as follows: the present invention decomposes the weakly supervised visible light-infrared cross-modal pedestrian re-identification problem into two-stage sub-problems: the self-supervision stage solves the learning of the feature representation of view invariance and the cross-modal stage solves the learning of the feature representation of modality invariance. By aligning the spatial distribution of the cross-modal pedestrian depth features, the distribution distance between the visible light modality and the infrared modality is minimized, the two modalities are aligned at the metric level, and a small amount of real annotated data is used to correct the models of the self-supervision stage and the cross-modal stage, so as to solve the problem of reduced recognition rate caused by huge differences in image styles in real cross-modal scenes, and improve the anti-interference ability and recognition ability of the cross-modal pedestrian re-identification method model. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 It is a flow chart of the steps of a cross-modal pedestrian re-identification method based on distribution space alignment of the present invention;
[0052] Figure 2 It is a structural block diagram of a cross-modal pedestrian re-identification system based on distribution space alignment of the present invention. DETAILED DESCRIPTION
[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0054] The weakly supervised visible light-infrared cross-modal pedestrian re-identification of the present invention is a weakly supervised, cross-domain retrieval task, which decomposes the retrieval task into two-stage tasks: in the self-supervised stage, a feature extractor that fits the two-modal pedestrian data set is trained by a "clustering-fine-tuning" method, and a small amount of data with real annotations is used to correct the model, learn the viewpoint invariant feature representation and pseudo-annotate the training data; in the cross-modal stage, pedestrian deep features and the outputs of the two modal classifiers are used as prediction labels, and the cross entropy loss function is calculated according to the modal classifier output. The triplet loss function and the modal distribution space alignment loss function are calculated according to the pedestrian deep features, the two modal distributions are aligned, and then the model is corrected according to the cross-modal pseudo-positive samples and a small amount of cross-modal positive samples with real annotations, thereby improving the accuracy of cross-modal pedestrian matching.
[0055] Reference Figure 1 The present invention provides a cross-modal person re-identification method based on distribution space alignment, which comprises the following steps:
[0056] Self-supervision stage:
[0057] S1, constructing a feature extractor and training the feature extractor based on the first training sample to obtain a trained feature extractor;
[0058] S2, acquiring a modal image and extracting pedestrian modal features based on a trained feature extractor;
[0059] S3, clustering the images to be tested according to pedestrian modal features to obtain a dataset with pseudo annotations;
[0060] Cross-modal stage:
[0061] S4, constructing a modality sharing model and training the modality sharing model based on the dataset with pseudo annotations to obtain a first modality sharing model;
[0062] S5. Training the first modality sharing model based on the second training sample to obtain a trained modality sharing model;
[0063] S6. Recognize the input image according to the trained modality sharing model to complete the pedestrian re-identification task.
[0064] Further as a preferred embodiment of the method, the first training sample is a training sample with real annotations, and the step of constructing a feature extractor and training the feature extractor based on the first training sample to obtain a trained feature extractor specifically includes:
[0065] Construct visible light and infrared modality feature extractors based on ResNet50;
[0066] The feature extractor is trained based on the first training sample and the model parameters are adjusted according to the feature loss function to obtain a trained feature extractor.
[0067] Specifically, the large-scale single-modal pedestrian re-identification dataset Market1501 is used to pre-train the feature extractors of the two modalities respectively. Based on the pedestrian identity features and pedestrian modal features corresponding to the feature extractor outputs, the cross-entropy loss function is calculated for the pedestrian identity features, and the triplet loss function is calculated for the modal features. The minimum value of the sum of the cross-entropy loss function and the triplet loss function is used as the target to optimize the training model. After each round of training using pseudo-annotated data is completed, a small amount of real annotated samples are loaded for training to further correct the model.
[0068] As a further preferred embodiment of the present method, taking the visible light modality feature extractor as an example, the step of calculating the cross entropy loss function is expressed as follows:
[0069]
[0070] In the above formula, C represents the number of pedestrian categories, N represents branches with different granularity, and FRGB (·) represents the classification output of the RGB modality sharing model, X represents the input image, represents the pedestrian pseudo-annotation generated in the self-supervision stage, j represents the pedestrian pseudo-annotation corresponding to the image, and e represents a natural constant.
[0071] As a further preferred embodiment of the present method, taking the visible light modality feature extractor as an example, the step of calculating the triple feature loss function is expressed as follows:
[0072]
[0073] In the above formula, the X a represents the selected sample instance, X p represents the positive sample of the selected sample, X n represents the negative sample of the selected sample, the F RGB (i) (X a ) represents the pedestrian modal features corresponding to the selected instance, and the F RGB (i) (X p ) represents the pedestrian modal features corresponding to the positive sample of the selected instance, and the F RGB (i) (X n ) represents the pedestrian modal features corresponding to the negative samples of the selected instances, P represents the number of selected individuals, K represents the number of instance images of each individual, and α represents the minimum threshold value of the distance between the selected samples and the positive and negative samples.
[0074] The total feature loss function expression in the self-supervision stage is as follows:
[0075] L = argmin(αL c +βL t ) pseudo +argmin(αL c +βL t ) true
[0076] The α and β represent hyperparameters that can be manually optimized and adjusted. c represents the cross entropy loss, L t represents the triplet feature loss.
[0077] As a further preferred embodiment of the method, the visible light and infrared modal feature extractors respectively include three branch networks with different granularities, and the branch networks share a ResNet50 backbone network. The step of acquiring a modal image and extracting pedestrian modal features based on the trained feature extractor specifically includes:
[0078] Obtain dataset images of visible light and infrared corresponding modalities, and obtain convolution features through the backbone network;
[0079] The convolutional features are respectively passed into three branch networks with different granularities for maximum pooling to obtain block features of corresponding granularities;
[0080] The block features of the corresponding granularity are reduced in dimension through a 1x1x256 convolution kernel to obtain the reduced-dimensional features;
[0081] The reduced-dimensional features are input into the fully connected layer at the end of the branch network for classification, and the pedestrian modal features are output.
[0082] As a further preferred embodiment of the method, the step of clustering the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations specifically includes:
[0083] Cluster the modal images according to pedestrian modal features and density clustering algorithm to obtain pseudo-annotations of the modal images;
[0084] The clustering expression is as follows:
[0085]
[0086] The X represents the input image, the represents the pseudo annotation generated in the unsupervised stage, and the F RGB represents the visible light modal feature extractor, Π represents the clustering target formula, the density clustering algorithm is adopted in this method, and J represents the clustering loss function corresponding to the clustering algorithm.
[0087] Specifically, clustering is performed to obtain pseudo-labels for each sample of the two modalities, which are combined with pedestrian images to form a new labeled dataset. The new dataset is then input into the corresponding modality feature extractor, and the feature loss function is calculated to continue to optimize the fine-tuning model, performing multiple iterations of "clustering-fine-tuning".
[0088] A dataset with pseudo annotations is generated according to the pseudo annotations of the modality images and the corresponding modality images.
[0089] Further as a preferred embodiment of the method, the step of constructing a modality sharing model and training the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model specifically includes:
[0090] Construct a modality sharing model and train it based on a dataset with pseudo annotations to obtain the corresponding modality classification output and pedestrian depth features;
[0091] Calculate the cross entropy loss function based on the modal classification output;
[0092] Calculate the triplet loss function based on the pedestrian depth features;
[0093] Calculate the modal distribution space alignment loss function based on pedestrian depth features;
[0094] The parameters of the modality sharing model are adjusted by combining the cross entropy loss function, the triplet loss function and the modality distribution space alignment loss function to obtain the first modality sharing model.
[0095] As a further preferred embodiment of the method, the calculation formula of the cross entropy loss function is as follows:
[0096]
[0097] In the above formula, F(·) represents the classification output of the modality sharing model, C represents the number of pedestrian categories, X represents the input image, y represents the pedestrian pseudo-annotation generated in the self-supervision stage, j represents the pedestrian pseudo-annotation corresponding to the image, and e represents a natural constant.
[0098] As a further preferred embodiment of the method, the calculation formula of the triplet loss function is as follows:
[0099]
[0100] In the above formula, the X a represents the selected sample instance, X p represents the positive sample of the selected sample, X n represents the negative sample of the selected sample, the F (i) (X a ) represents the pedestrian depth features extracted by the selected instance through the modality sharing model, and the F (i) (X p ) represents the pedestrian depth features extracted by the modality sharing model from the positive sample of the selected instance, and the F (i) (X n ) represents the pedestrian depth features extracted by the negative sample of the selected instance through the modality sharing model, P represents the number of selected individuals, K represents the number of instance images of each individual, and α represents the minimum threshold of the distance between the selected sample and the positive and negative samples.
[0101] As a further preferred embodiment of the method, the formula for calculating the modal distribution space alignment loss function based on pedestrian depth features is expressed as follows:
[0102]
[0103] In the above formula, ||·|| p represents the p-order norm, Φ(F(X RGB ),F(X IR )) means including all and F(X RGB ),F(X IR ) is the set of all joint distributions of the boundary distribution, X RGB represents the visible light modality sample, X IR represents the near-infrared modal sample, p represents the norm, x represents the sample of the visible light modal distribution space, and x' represents the sample of the near-infrared modal distribution space.
[0104] Specifically, the distribution distance of the pedestrian depth features in the visible light and infrared modes is calculated, and the distribution of the pedestrian depth features in the two modes is brought closer through the modal distribution space alignment loss function, which solves the problem of the large distribution distance of the pedestrian depth features caused by the large difference in cross-modal image styles.
[0105] Further as a preferred embodiment of the method, the second training sample is a training sample with a real annotation, and the step of training the first modality sharing model based on the second training sample to obtain the trained modality sharing model specifically includes:
[0106] Generate pseudo positive sample pairs based on the first modality shared model;
[0107] Preliminarily correcting the first modality sharing model according to the pseudo-positive sample pair to obtain a preliminarily corrected modality sharing model;
[0108] The initially corrected modality sharing model is trained based on the second training sample to obtain a trained modality sharing model.
[0109] As a further preferred embodiment of the method, the step of generating a pseudo positive sample pair based on the first modality sharing model specifically includes:
[0110] According to the first modality sharing model, a distance matrix between same-modality samples and a distance matrix between cross-modality samples are obtained;
[0111] Select two cross-modal sample images and calculate k same-modal nearest neighbors and k cross-modal nearest neighbors of the selected samples according to the distance matrix between same-modal samples and the distance matrix between cross-modal samples, and obtain k infrared modality neighbors of the infrared modality sample image, k visible light modality neighbors of the infrared modality sample image, k infrared modality neighbors of the visible light modality sample image, and k visible light modality neighbors of the visible light modality sample image;
[0112] Calculate the Jaccard similarity between the k infrared modal neighbors of the infrared modal sample image and the k infrared modal neighbors of the visible light modal sample image, and the Jaccard similarity between the k visible light modal neighbors of the infrared modal sample image and the k visible light modal neighbors of the visible light modal sample image, and weight them to obtain the total Jaccard similarity of the cross-modal sample pairs;
[0113] Sort the total Jaccard similarity according to the cross-modal samples to obtain the cross-modal neighbor sequence of each sample;
[0114] A preset number of images are taken according to the cross-modal neighbor sequence of each sample to obtain cross-modal pseudo-positive sample pairs.
[0115] like Figure 2 As shown, a cross-modal person re-identification system based on distribution space alignment includes:
[0116] A first construction module is used to construct a feature extractor and train the feature extractor based on the first training sample to obtain a trained feature extractor;
[0117] A feature extraction module, used to obtain modal images and extract pedestrian modal features based on the trained feature extractor;
[0118] The clustering module is used to cluster the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations;
[0119] A second construction module is used to construct a modality sharing model and train the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model, and train the first modality sharing model based on a second training sample to obtain a trained modality sharing model;
[0120] The recognition module is used to recognize the input image based on the trained modality sharing model and complete the pedestrian re-identification task.
[0121] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0122] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A cross-modal person re-identification method based on distribution space alignment, characterized in that: The following steps are involved: Constructing a feature extractor and training the feature extractor based on the first training sample to obtain a trained feature extractor; specifically including: Construct visible light and infrared modality feature extractors based on ResNet50; The visible light and infrared modality feature extractors respectively include three branch networks with different granularities, and the branch networks share the ResNet50 backbone network; Training a feature extractor based on the first training sample and adjusting model parameters according to a feature loss function to obtain a trained feature extractor; Obtain modal images and extract pedestrian modal features based on the trained feature extractor; specifically including: Obtain dataset images of visible light and infrared corresponding modalities, and obtain convolution features through the backbone network; The convolutional features are respectively passed into three branch networks with different granularities for maximum pooling to obtain block features of corresponding granularities; The block features of the corresponding granularity are reduced in dimension through a 1x1x256 convolution kernel to obtain the reduced-dimensional features; The reduced-dimensional features are input into the fully connected layer at the end of the branch network for classification, and the pedestrian modal features are output; Cluster the images to be tested according to pedestrian modal features to obtain a dataset with pseudo annotations; Constructing a modality sharing model and training the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model; Training the first modality sharing model based on the second training sample to obtain a trained modality sharing model; The input image is recognized according to the trained modality sharing model to complete the pedestrian re-identification task.
2. According to claim 1, a cross-modal person re-identification method based on distribution space alignment is characterized in that: The first training sample is a training sample with real annotations.
3. According to claim 1, a cross-modal person re-identification method based on distribution space alignment is characterized in that: The step of clustering the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations specifically includes: Cluster the modal images according to pedestrian modal features and density clustering algorithm to obtain pseudo-annotations of the modal images; A dataset with pseudo annotations is generated according to the pseudo annotations of the modality images and the corresponding modality images.
4. According to claim 3, a cross-modal pedestrian re-identification method based on distribution space alignment is characterized in that: The step of constructing a modality sharing model and training the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model specifically includes: Construct a modality sharing model and train it based on a dataset with pseudo annotations to obtain the corresponding modality classification output and pedestrian depth features; Calculate the cross entropy loss function based on the modal classification output; Calculate the triplet loss function based on the pedestrian depth features; Calculate the modal distribution space alignment loss function based on pedestrian depth features; The parameters of the modality sharing model are adjusted by combining the cross entropy loss function, the triplet loss function and the modality distribution space alignment loss function to obtain the first modality sharing model.
5. According to claim 4, a cross-modal person re-identification method based on distribution space alignment is characterized in that: The calculation formula of the cross entropy loss function is as follows: In the above formula, F(·) represents the classification output of the modality sharing model, C represents the number of pedestrian categories, X represents the input image, y represents the pedestrian pseudo-annotation generated in the self-supervision stage, j represents the pedestrian pseudo-annotation corresponding to the image, and e represents a natural constant.
6. According to claim 5, a cross-modal pedestrian re-identification method based on distribution space alignment is characterized in that: The formula for calculating the modal distribution space alignment loss function based on pedestrian depth features is as follows: In the above formula, ‖·‖ p represents the p-order norm, Φ(F(X RGB ),F(X IR )) means including all and F(X RGB ),F(X IR ) is the set of all joint distributions of the boundary distribution, X RGB represents the visible light modality sample, X IR represents the near-infrared modal sample, p represents the norm, x represents the sample in the visible light modal distribution space, x ′ Represents a sample of the near-infrared modal distribution space.
7. The cross-modal person re-identification method based on distribution space alignment according to claim 6, characterized in that: The step of training the first modality sharing model based on the second training sample as a training sample with a real annotation to obtain a trained modality sharing model specifically includes: Generate pseudo positive sample pairs based on the first modality shared model; Preliminarily correcting the first modality sharing model according to the pseudo-positive sample pair to obtain a preliminarily corrected modality sharing model; The initially corrected modality sharing model is trained based on the second training sample to obtain a trained modality sharing model.
8. The cross-modal person re-identification method based on distribution space alignment according to claim 7, characterized in that: The step of generating a pseudo positive sample pair based on the first modality sharing model specifically includes: According to the first modality sharing model, a distance matrix between same-modality samples and a distance matrix between cross-modality samples are obtained; Select two cross-modal sample images and calculate k same-modal nearest neighbors and k cross-modal nearest neighbors of the selected samples according to the distance matrix between same-modal samples and the distance matrix between cross-modal samples, and obtain k infrared modal neighbors of the infrared modal sample image, k visible light modal neighbors of the infrared modal sample image, k infrared modal neighbors of the visible light modal sample image, and k visible light modal neighbors of the visible light modal sample image; Calculate the Jaccard similarity between the k infrared modal neighbors of the infrared modal sample image and the k infrared modal neighbors of the visible light modal sample image, and the Jaccard similarity between the k visible light modal neighbors of the infrared modal sample image and the k visible light modal neighbors of the visible light modal sample image, and weight them to obtain the total Jaccard similarity of the cross-modal sample pairs; Sort the total Jaccard similarity according to the cross-modal samples to obtain the cross-modal neighbor sequence of each sample; A preset number of images are taken according to the cross-modal neighbor sequence of each sample to obtain cross-modal pseudo-positive sample pairs.
9. A cross-modal person re-identification system based on distribution space alignment, characterized in that: include: A first construction module is used to construct a feature extractor and train the feature extractor based on the first training sample to obtain a trained feature extractor; Specifically include: Construct visible light and infrared modality feature extractors based on ResNet50; The visible light and infrared modality feature extractors respectively include three branch networks with different granularities, and the branch networks share the ResNet50 backbone network; Training a feature extractor based on the first training sample and adjusting model parameters according to a feature loss function to obtain a trained feature extractor; The feature extraction module is used to obtain the modal image and extract the pedestrian modal features based on the trained feature extractor; specifically, it includes: Obtain dataset images of visible light and infrared corresponding modalities, and obtain convolution features through the backbone network; The convolutional features are respectively passed into three branch networks with different granularities for maximum pooling to obtain block features of corresponding granularities; The block features of the corresponding granularity are reduced in dimension through a 1x1x256 convolution kernel to obtain the reduced-dimensional features; The reduced-dimensional features are input into the fully connected layer at the end of the branch network for classification, and the pedestrian modal features are output; The clustering module is used to cluster the images to be tested according to the pedestrian modal features to obtain a dataset with pseudo annotations; A second construction module is used to construct a modality sharing model and train the modality sharing model based on a dataset with pseudo annotations to obtain a first modality sharing model, and train the first modality sharing model based on a second training sample to obtain a trained modality sharing model; The recognition module is used to recognize the input image based on the trained modality sharing model and complete the pedestrian re-identification task.