Unsupervised cross-modal person re-identification method based on clustering optimization

Abstract

The present invention discloses an unsupervised cross-modal pedestrian re-identification method based on clustering optimization, including: 1) alleviating the huge modal difference between visible light images and infrared images by means of dual-modal channel data enhancement of visible light modality and infrared light modality, thereby improving the accuracy of cross-modal cluster matching; 2) eliminating the noise influence that may be brought by unsupervised clustering through intra-modal denoising and cross-modal matching and denoising, and obtaining reliable cross-modal cluster center matching pairs; 3) eliminating the negative impact of different camera styles on clustering by dividing the camera center, and further optimizing the clustering; 4) designing a training and testing method for an unlabeled cross-modal pedestrian re-identification dataset without relying on a labeled source domain dataset for pre-training. The recognition average mean accuracy of this method has been improved by more than 10% compared with previous methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian re-identification, and specifically relates to an unsupervised cross-modal pedestrian re-identification method based on clustering optimization. Background Art

[0002] Video surveillance is a crucial component of security. Using surveillance cameras to monitor a target area effectively ensures safety. The vast number of surveillance cameras generates massive amounts of video data, posing a challenge to data processing. Traditional methods, relying on manual analysis of video footage captured by surveillance cameras, are extremely time-consuming and labor-intensive. Intelligent video surveillance leverages technologies in computer vision, pattern recognition, and machine learning, employing artificial intelligence algorithms to automatically analyze video content.

[0003] Person re-identification (ReID) is a crucial technology in intelligent video surveillance. Its goal is to identify the same person across a network of cameras. Given a query image of a person, the algorithm must automatically find all images captured by all cameras that show the same person as the query image. Early person re-ID methods achieved excellent performance in the visible light mode using deep learning models and large-scale datasets.

[0004] In order to cope with the 24-hour monitoring environment, especially the nighttime environment, many surveillance cameras are equipped with both visible light mode and infrared mode. The purpose of cross-modal pedestrian re-identification is to solve the pedestrian image matching problem between the visible light modality and the infrared modality. A large number of existing cross-modal pedestrian re-identification methods usually rely on manually annotated cross-modal datasets, which is usually labor-intensive and time-consuming, especially for the annotation of infrared datasets. Therefore, it is particularly important to train the cross-modal ReID model through unsupervised methods. Specifically, it does not rely on manually annotated labels but trains the cross-modal ReID model through pseudo labels generated by the model's own clustering. The focus is on how to overcome modality differences and obtain cross-modal clustering belonging to the same pedestrian category.

[0005] Existing cross-modal pedestrian re-identification methods can be divided into two categories. One is to pre-train the model through a visible light annotated dataset and then migrate it to a cross-modal dataset. A typical method is H2H, which is pre-trained in the visible light source domain, and then obtains cross-modal shared labels through homogeneous training within the modality and heterogeneous label matching across modalities. A similar method is patent document CN114495004A, which uses a labeled source dataset for pre-training. The second category is methods that do not rely on annotated datasets for pre-training, such as ADCA, which obtains single-modal pseudo labels and cluster centers through unsupervised clustering within the modality, and then brings together the cluster centers of samples from different modalities through cross-modal matching. The existing methods have the following shortcomings:

[0006] The first type of method relies heavily on pre-training of labeled source domain datasets. Strictly speaking, it is not a completely unsupervised cross-modal pedestrian re-identification method and is not suitable for real-world scenarios.

[0007] The second type of method relies heavily on the accuracy of cross-modal pseudo-labels obtained by unsupervised clustering. Existing methods cannot guarantee the accuracy of cross-modal labels. Specifically, there are three shortcomings: 1) They do not take into account the huge modal differences between visible light and infrared images, resulting in the inability to obtain the same pseudo-label for pedestrians with the same identity in different modalities. For example, patent document CN116229510A states that even extracting a single-channel image cannot eliminate the modal difference, resulting in cross-modal clustering failure; 2) Existing methods also do not take into account intra-modal clustering errors caused by differences in camera styles. For example, the imbalance in the number of clusters caused by the fact that there are more visible light cameras than infrared cameras in the SYSU-MM01 dataset leads to a failure in cross-modal matching; 3) They do not take into account the problem of cluster noise and the presence of a small number of images of different categories in the cluster. Incorrect clustering and pseudo-labeling often have a negative impact on model training, and cross-modal label matching may also be noisy. Summary of the Invention

[0008] In view of the above, the object of the present invention is to provide an unsupervised cross-modal person re-identification method based on clustering optimization to improve recognition accuracy.

[0009] To achieve the above-mentioned object of the invention, an embodiment provides an unsupervised cross-modal person re-identification method based on clustering optimization, comprising the following steps:

[0010] Step 1: Obtain a set of visible light original images and a set of infrared original images of multiple target objects in visible light mode and infrared light mode, and perform channel enhancement on them respectively to obtain a set of visible light enhanced images and a set of infrared enhanced images;

[0011] Step 2: Extract features of each image set based on the dual-path deep learning model, and cluster each feature to obtain cluster centers and corresponding pseudo labels;

[0012] Step 3: Eliminate noise within the modality based on each cluster center and the corresponding pseudo label;

[0013] Step 4: After noise elimination, matching between cluster centers across modalities is achieved;

[0014] Step 5: The cluster centers corresponding to each image set of each modality are divided into different camera centers by camera labels;

[0015] Step 6: Combine the matching results between cluster centers and camera centers to construct intra-modality camera center contrast loss and cross-modality camera center contrast loss, and update the parameters of camera center and two-way deep learning model based on the contrast loss.

[0016] Step 7: Perform person re-identification based on the two-way deep learning model with updated parameters.

[0017] Preferably, in step 1, for the visible light original image, one of the three channels R, G, and B is randomly selected as the channel enhanced image to obtain a visible light enhanced image set; for the infrared original image, the infrared original image is processed by random illumination changes as the channel enhanced image to obtain an infrared enhanced image set.

[0018] Preferably, in step 2, the dual-path deep learning model includes a visible light layer and an infrared layer, and its parameter updates are calculated independently, wherein the visible light layer is used to perform feature extraction on the visible light original image set and the visible light enhanced image set in the visible light modality to obtain visible light features and visible light enhanced features, and the infrared layer is used to perform feature extraction on the infrared original image set and the infrared enhanced image set in the infrared modality to obtain infrared features and infrared enhanced features;

[0019] The visible light features and infrared features are clustered separately to obtain the cluster centers and corresponding pseudo labels, which can be expressed as follows:

[0020]

[0021] Among them, N k Indicates that the pseudo label y assigned to the kth category by the clustering algorithm k The number of samples, Indicates that the c-th image set belongs to the pseudo label y k The set of directional vectors, express The feature vector of the nth image sample in , Indicates that the pseudo label y is obtained by clustering the c-th image set k The number of pseudo-label types corresponds to the number of cluster centers. At this time, c takes the values ​​of v and i, which represent the visible light original image set and the infrared original image set respectively.

[0022] Preferably, in step 3, performing intra-modal noise elimination based on each cluster center and the corresponding pseudo label includes:

[0023] In each mode, each cluster center As a classifier, the classifier calculates the likelihood pseudo label for each image sample in the modality. If the likelihood pseudo label is different from the pseudo label of the original cluster, the image sample pair is regarded as a noise sample and deleted from the modality. The likelihood pseudo label is calculated by the following formula:

[0024]

[0025]

[0026] Among them, Y c represents the pseudo label set corresponding to the c-th image set, represents the pseudo label set Y c The jth pseudo label in , y represents the target label, Indicates that the pseudo label y is obtained by clustering the c-th image set j The cluster center of Indicates the pseudo label y obtained by clustering the c-th image set belonging to the z-th z The cluster center of z is 1-K, where K is the total number of pseudo labels. Represents the obtained likelihood pseudo label.

[0027] Preferably, in step 4, matching between cluster centers across modalities is achieved after noise elimination, including:

[0028] For the original image set, for each corresponding pseudo label The visible light cluster center All the corresponding visible light original image samples Calculate infrared likelihood pseudo labels through the classifier composed of infrared cluster centers The calculation formula is as follows:

[0029]

[0030]

[0031] Among them, Y i represents the pseudo label set corresponding to the infrared original image set, represents the pseudo label set Y i The jth pseudo label in , y represents the target label, Indicates the pseudo label y obtained by clustering the original infrared image set j The cluster center of Represents the pseudo label y belonging to the zth obtained by clustering the infrared original image set z The cluster center of z is 1-L, where L is the total number of pseudo labels.

[0032] The infrared likelihood pseudo-label set calculated for all corresponding visible light original image samples Among them, the infrared likelihood pseudo label to which the sample belongs most is The corresponding infrared cluster center As The matching quasi-infrared cluster center is calculated as follows:

[0033]

[0034]

[0035] For the matching quasi-infrared cluster centers All corresponding infrared original image samples Calculate the visible light likelihood pseudo label through the classifier composed of visible light cluster centers The corresponding visible light likelihood pseudo-label set calculated for all infrared original image samples Among them, the most common visible light likelihood pseudo label for the sample is The corresponding visible light cluster center As The matching quasi-visible light cluster center, when the quasi-visible light cluster center and visible light cluster centers When they are equal, then for the original image set, it is considered that The visible light cluster center and the infrared cluster center are matched.

[0036] Preferably, in step 5, each cluster center corresponding to each image set of each modality is divided into different camera centers by camera labels, which can be expressed as follows:

[0037]

[0038] in, Indicates a pseudo label The feature vector set of the c-th image set with camera label s, Indicates belonging The feature vector of the nth image sample, N ks represents the number of samples assigned to camera label s by the clustering algorithm, Indicates that the pseudo label y is obtained by clustering the c-th image set k And it belongs to the camera center of camera label s, then the value of c is v,i.

[0039] Preferably, in step 6, the camera center contrast loss within the modality is It is calculated by the following formula:

[0040]

[0041]

[0042]

[0043]

[0044] Among them, q v and q i In the training phase, the visible light original image and the infrared original image are input into the dual-path deep learning model to obtain the visible light features and infrared features, q va and q ia Represents the visible light enhancement features and infrared enhancement features obtained by inputting the corresponding visible light enhanced image and infrared enhanced image into the dual-path deep learning model, and Respectively represent and q v and q i The set of all visible and infrared camera centers with the same cluster center, p j Indicates that q v or q i N nearest homomodal clusters that do not have the same cluster center neg Camera center, and Respectively and The number of camera centers in , τ represents a learnable hyperparameter, Indicates belonging The center of the sth visible light camera, Indicates belonging The center of the t-th infrared camera;

[0045] The cross-modal camera center contrast loss is It is calculated by the following formula:

[0046]

[0047]

[0048]

[0049]

[0050] in, Express expectations.

[0051] Preferably, in step 6, updating the parameters of the camera center and the two-way deep learning model according to the contrast loss includes:

[0052] Based on contrast loss and Constructing the total loss Use the total loss to update the parameters of the two-way deep learning model,

[0053]

[0054] Among them, λ cross It is a hyperparameter used to balance the intra-modal and cross-modal weights;

[0055] During the update process, the image sample vector of each batch is also used to update the center of each camera, which is specifically expressed as:

[0056]

[0057]

[0058]

[0059]

[0060] Among them, δ represents the number of iterations, β represents the updated weight, and the next round of training uses the updated camera centers. The arrow ← represents the update. represents the center of the visible light camera, Indicates that the light can be enhanced to the center of the camera, Indicates the center of the infrared camera, Indicates the center of the infrared-enhanced camera.

[0061] Preferably, in step 7, performing person re-identification based on the two-way deep learning model after parameter update includes:

[0062] The original visible light image or infrared image of the target object to be identified is used as the probe image sample. The probe image sample and the library image sample are input into the two-way deep learning model with updated parameters to identify the corresponding visible light features and infrared features. The visible light features or infrared features are selected to calculate the similarity with the features of the library sample image respectively, and all library sample images are sorted according to the similarity. The identity label of the library sample image with the highest ranking is selected as the identification identity label of the probe image sample.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1) By enhancing data from both visible and infrared modalities, the team mitigated the significant modality differences between visible and infrared images, improving cross-modal cluster matching accuracy.

[0065] 2) Through intra-modal denoising and cross-modal matching and denoising, the noise effect that may be caused by unsupervised clustering is eliminated, and reliable matching pairs of cross-modal cluster centers are obtained;

[0066] 3) The negative impact of different camera styles on clustering is eliminated by dividing the camera center, further optimizing the clustering;

[0067] 4) Without relying on pre-training on labeled source datasets, we designed a training and testing method for an unlabeled cross-modal person re-identification dataset. The mean average precision (mAP) improved by more than 10% compared to previous methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 1 is a flow chart of an unsupervised cross-modal person re-identification method based on clustering optimization provided by an embodiment;

[0070] Figure 2 is a flow chart of dual-modal channel data enhancement provided by an embodiment;

[0071] Figure 3 3 is a schematic diagram of denoising cross-modal matching provided by an embodiment. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0073] The inventive concept of the present invention is: in order to solve the shortcomings and defects in the existing unsupervised cross-modal pedestrian re-identification technology, the present invention provides an unsupervised cross-modal pedestrian re-identification method based on clustering optimization, and optimizes the unsupervised clustering of the existing method from three levels, respectively solving the clustering problems caused by large cross-modal differences, the influence of camera style differences and the influence of clustering noise. At the same time, an unsupervised cross-modal pedestrian re-identification model that does not rely on source domain pre-training is designed, and its performance and recognition accuracy exceed the existing methods.

[0074] The embodiment provides an unsupervised cross-modal pedestrian re-identification method based on clustering optimization, such as Figure 1As shown, the following steps are included:

[0075] Step 1: Obtain a visible light original image set and an infrared original image set of multiple target objects in visible light mode and infrared light mode and perform channel enhancement respectively to obtain a visible light enhanced image set and an infrared enhanced image set.

[0076] Specifically, the visible light image and infrared image are extracted from the image captured by the camera through the pedestrian target detection method and form a visible light original image set X v and infrared original image set X i , and it is necessary to mark the camera to which each image belongs. Based on the original image set, the modal difference between the visible light original image and the infrared original image is alleviated by means of data channel data enhancement. Figure 2 As shown in the figure, for the original visible light image, one of the three channels of R, G, and B is randomly selected as the channel enhanced image to form the visible light enhanced image set X va For infrared images, random illumination changes are used to process the original infrared images as channel enhanced images, and the infrared enhanced image set X is obtained. ia .

[0077] In step 2, the features of each image set are extracted based on the two-way deep learning model, and each feature is clustered to obtain each cluster center and the corresponding pseudo label.

[0078] In this embodiment, the parameter updates of the visible light layer and infrared layer of the dual-path deep learning model are calculated independently. Specifically, the public model Resnet pre-trained on ImageNet is used as the backbone network, wherein the visible light layer is used to update the visible light original image set X in the visible light modality. v and visible light enhanced image set X va Perform feature extraction to obtain visible light feature U v and visible light enhancement characteristics U va , the infrared layer is used to process the infrared original image set X under the infrared light modality i and infrared enhanced image set X ia Perform feature extraction to obtain infrared feature U i and infrared enhanced features U ia .

[0079] The visible light features and infrared features are clustered separately to obtain the cluster centers and corresponding pseudo labels, which can be expressed as follows:

[0080]

[0081] Among them, N k Indicates that the pseudo label y assigned to the kth category by the clustering algorithm k The number of samples, Indicates that the c-th image set belongs to the pseudo label y k The set of directional vectors, express The feature vector of the nth image sample in , Indicates that the pseudo label y is obtained by clustering the c-th image set k The number of pseudo-label types corresponds to the number of cluster centers. The value of c is v, i, which represent the visible light original image set and the infrared original image set respectively. They represent the visible light cluster center and infrared cluster center respectively. The specific clustering can adopt the DBSCAN clustering algorithm.

[0082] Step 3: Perform intra-modal noise elimination based on each cluster center and the corresponding pseudo-label.

[0083] In the embodiment, after obtaining each cluster center and the corresponding pseudo label, the noise elimination in the modality is performed. The specific process is:

[0084] like Figure 3 As shown in (a), within each mode, each cluster center As a classifier, the classifier calculates the likelihood pseudo label for each image sample in the modality. If the likelihood pseudo label is different from the pseudo label of the original cluster, the image sample pair is regarded as a noise sample and deleted from the modality. The likelihood pseudo label is calculated by the following formula:

[0085]

[0086]

[0087] Among them, Y c represents the pseudo label set corresponding to the c-th image set, represents the pseudo label set Y c The jth pseudo label in , y represents the target label, Indicates that the pseudo label y is obtained by clustering the c-th image set j The cluster center of Indicates the pseudo label y obtained by clustering the c-th image set belonging to the z-th z The cluster center of z is 1-K, where K is the total number of pseudo labels. Indicates the obtained likelihood pseudo-label, where c takes the value v, i, based only on the visible light cluster center Infrared Cluster Center Intra-modal denoising is performed on visible light raw image samples and infrared raw image samples.

[0088] Step 4: After noise removal, cross-modal cluster center matching is achieved.

[0089] like Figure 3 As shown in (b), for the original image set, for each corresponding pseudo label The visible light cluster center All the corresponding visible light original image samples Calculate infrared likelihood pseudo labels through the classifier composed of infrared cluster centers The calculation formula is as follows:

[0090]

[0091]

[0092] Among them, Y i represents the pseudo label set corresponding to the infrared original image set, represents the pseudo label set Y i The jth pseudo label in , y represents the target label, Indicates the pseudo label y obtained by clustering the original infrared image set j The cluster center of Represents the pseudo label y belonging to the zth obtained by clustering the infrared original image set z The cluster center of , the value of z is 1-L, L is the total number of pseudo labels;

[0093] Visible light original image set X v The corresponding feature vector set The infrared likelihood pseudo-label set calculated for all corresponding visible light original image samples Among them, the infrared likelihood pseudo label to which the sample belongs most is The corresponding infrared cluster center As The matching quasi-infrared cluster center is calculated as follows:

[0094]

[0095]

[0096] In order to eliminate noise interference, a two-way matching process will be carried out, specifically: for the matched quasi-infrared clustering centers All corresponding infrared original image samples The principles of formula (4) and formula (5) are used to calculate the visible light likelihood pseudo label through the classifier composed of visible light cluster centers. Infrared raw image set X i The corresponding feature vector set The corresponding visible light likelihood pseudo-label set calculated for all infrared original image samples Among them, the principles of formula (6) and formula (7) are used to convert the most visible light likelihood pseudo labels to which the samples belong The corresponding visible light cluster center As The matching quasi-visible light cluster center, when the quasi-visible light cluster center and visible light cluster centers When they are equal, then for the original image set, it is considered that is the matching visible light cluster center and infrared cluster center, otherwise Discarded as noise matching.

[0097] In step 5, each cluster center corresponding to each image set of each modality is divided into different camera centers by camera labels.

[0098] In this embodiment, each cluster center corresponding to each image set of each modality is divided into different camera centers by camera labels. The same cluster center may be divided into different camera centers. The contrast loss in step 6 is based on the camera center. Specifically, the formula used to divide the camera center is:

[0099]

[0100] in, Indicates a pseudo label The feature vector set of the c-th image set with camera label s, Indicates belonging The feature vector of the nth image sample, N ks represents the number of samples assigned to camera label s by the clustering algorithm, Indicates that the pseudo label y is obtained by clustering the c-th image set k And it belongs to the camera center of camera tag s. At this time, c takes the value of v, i, and only the visible light cluster center Infrared Cluster Center Perform camera center division. When calculating the loss, the visible light enhanced image reuses the visible light cluster center corresponding to the visible light original image. Infrared enhanced image reuses the infrared cluster center corresponding to the infrared original image

[0101] In step 6, the matching results between cluster centers and camera centers are combined to construct intra-modal camera center contrast loss and cross-modal camera center contrast loss, and the parameters of the camera center and the two-way deep learning model are updated based on the contrast loss.

[0102] In an embodiment, combining the matching results between cluster centers and camera centers to construct the camera center contrast loss within the modality includes: It is calculated by the following formula:

[0103]

[0104]

[0105]

[0106]

[0107] Among them, q v and q i In the training phase, the visible light original image and the infrared original image are input into the dual-path deep learning model to obtain the visible light features and infrared features, q va and q ia Represents the visible light enhancement features and infrared enhancement features obtained by inputting the corresponding visible light enhanced image and infrared enhanced image into the dual-path deep learning model, and Respectively represent and q v and q i The set of all visible and infrared camera centers with the same cluster center, p j Indicates that q v or q i N nearest homomodal clusters that do not have the same cluster center neg Camera center, and Respectively and The number of camera centers in , τ represents a learnable hyperparameter, Indicates belonging The center of the sth visible light camera, Indicates belonging The center of the t-th infrared camera, express expectations;

[0108] The cross-modal camera center contrast loss is It is calculated by the following formula:

[0109]

[0110]

[0111]

[0112]

[0113] Then based on the contrast loss and Constructing the total loss Using the total loss to update the parameters of the dual-path deep learning model using parameter gradient backpropagation, the total loss Expressed as:

[0114]

[0115] Among them, λ cross It is a hyperparameter used to balance the intra-modal and cross-modal weights, and is generally set to 1.2 to enhance the weight of cross-modal clustering.

[0116] During the update process, the image sample vector of each batch is also used to update the center of each camera, which is specifically expressed as:

[0117]

[0118]

[0119]

[0120]

[0121] Among them, δ represents the number of iterations, β represents the update weight, which is 0.1. The updated camera centers are used in the next round of training. The arrow ← represents the update. represents the center of the visible light camera, Indicates that the light can be enhanced to the center of the camera, Indicates the center of the infrared camera, Indicates the center of the infrared-enhanced camera.

[0122] In the specific experimental example, there are a total of 100 training rounds, and each training round repeats steps 2 to 5. The trained dual-path deep learning model is tested every 5 rounds. During the training iteration, the batch size of each training round is 64, and each modality image set and its enhanced image set are sampled from each category to form a training batch, that is, a training batch contains B v ,B va ,B i ,B ia There are a total of 64×4=256 training image samples. Repeat step 5, and iterate 200 times in each training round.

[0123] In an embodiment, the testing process of the dual-path deep learning model is as follows: the probe sample image and the library sample image are input into the parameter-optimized dual-path deep learning model to obtain the corresponding directional vector. Specifically, when the probe sample image is an infrared image, the library sample image is a visible light image; when the probe sample image is a visible light image, the library sample image is an infrared image. Then, the similarity between the probe sample image and each library sample image is calculated, and the library sample images are sorted based on the similarity. The identity label of the library sample image with the highest ranking is the identification identity label corresponding to the probe sample image. Finally, the mean average precision (mAP) between the identification identity label of the probe sample image and the true identity label is calculated. The mean average precision is used to evaluate the robustness of the model. After multiple rounds of training, the model with the highest mAP is saved as the final parameter-updated dual-path deep learning model for pedestrian re-identification.

[0124] Step 7: Perform person re-identification based on the two-way deep learning model with updated parameters.

[0125] In an embodiment, the process of pedestrian re-identification in the application stage is: the visible light original image or infrared original image of the target object to be identified is used as a probe image sample, the probe image sample and the library image sample are input into the two-way deep learning model with updated parameters, the corresponding visible light features and infrared features are identified, the visible light features or infrared features are selected to calculate the similarity with the features of the library sample image respectively, and all library sample images are sorted according to the similarity, and the identity label of the library sample image with the highest ranking is selected as the identification identity label of the probe image sample.

[0126] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An unsupervised cross-modal person re-identification method based on clustering optimization, characterized by: The following steps are involved: Step 1: Obtain a set of visible light original images and a set of infrared original images of multiple target objects in visible light mode and infrared light mode, and perform channel enhancement on them respectively to obtain a set of visible light enhanced images and a set of infrared enhanced images; Step 2: Extract features of each image set based on the dual-path deep learning model, and cluster each feature to obtain cluster centers and corresponding pseudo labels; Step 3: Eliminate noise within the modality based on each cluster center and the corresponding pseudo label; Step 4: After noise elimination, matching between cluster centers across modalities is achieved; Step 5: The cluster centers corresponding to each image set of each modality are divided into different camera centers by camera labels; Step 6: Combine the matching results between cluster centers and camera centers to construct intra-modality camera center contrast loss and cross-modality camera center contrast loss, and update the parameters of camera center and two-way deep learning model based on the contrast loss. Step 7: Perform person re-identification based on the two-way deep learning model with updated parameters.

2. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 1, characterized in that: In step 1, for the visible light original image, one of the three channels of R, G, and B is randomly selected as the channel enhanced image to obtain a visible light enhanced image set; for the infrared original image, the infrared original image is processed by random illumination changes as the channel enhanced image to obtain an infrared enhanced image set.

3. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 1, characterized in that: In step 2, the dual-path deep learning model includes a visible light layer and an infrared layer, and their parameter updates are calculated independently. The visible light layer is used to extract features from the visible light original image set and the visible light enhanced image set in the visible light modality to obtain visible light features and visible light enhanced features. The infrared layer is used to extract features from the infrared original image set and the infrared enhanced image set in the infrared modality to obtain infrared features and infrared enhanced features. The visible light features and infrared features are clustered separately to obtain the cluster centers and corresponding pseudo labels, which can be expressed as follows: Among them, N k Indicates that the pseudo label y assigned to the kth category by the clustering algorithm k The number of samples, Indicates that the c-th image set belongs to the pseudo label y k The set of directional vectors, express The feature vector of the nth image sample in , Indicates that the pseudo label y is obtained by clustering the c-th image set k The number of pseudo-label types corresponds to the number of cluster centers. At this time, c takes the values ​​of v and i, which represent the visible light original image set and the infrared original image set respectively.

4. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 3 is characterized in that: In step 3, noise removal within the modality is performed based on each cluster center and the corresponding pseudo-label, including: In each mode, each cluster center As a classifier, the classifier calculates the likelihood pseudo label for each image sample in the modality. If the likelihood pseudo label is different from the pseudo label of the original cluster, the image sample pair is regarded as a noise sample and deleted from the modality. The likelihood pseudo label is calculated by the following formula: Among them, Y c represents the pseudo label set corresponding to the c-th image set, represents the pseudo label set Y c The jth pseudo label in , y represents the target label, Indicates that the pseudo label y is obtained by clustering the c-th image set j The cluster center of Indicates the pseudo label y obtained by clustering the c-th image set belonging to the z-th z The cluster center of z is 1-K, where K is the total number of pseudo labels. Represents the obtained likelihood pseudo label.

5. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 3 is characterized in that: In step 4, after noise removal, cross-modal cluster center matching is achieved, including: For the original image set, for each corresponding pseudo label The visible light cluster center All the corresponding visible light original image samples Calculate infrared likelihood pseudo labels through the classifier composed of infrared cluster centers The calculation formula is as follows: Among them, Y i represents the pseudo label set corresponding to the infrared original image set, represents the pseudo label set Y i The jth pseudo label in , y represents the target label, Indicates the pseudo label y obtained by clustering the original infrared image set j The cluster center of Represents the pseudo label y belonging to the zth obtained by clustering the infrared original image set z The cluster center of z is 1-L, where L is the total number of pseudo labels. The infrared likelihood pseudo-label set calculated for all corresponding visible light original image samples Among them, the infrared likelihood pseudo label to which the sample belongs most is The corresponding infrared cluster center As The matching quasi-infrared cluster center is calculated as follows: For the matching quasi-infrared cluster centers All corresponding infrared original image samples Calculate the visible light likelihood pseudo label through the classifier composed of visible light cluster centers The corresponding visible light likelihood pseudo-label set calculated for all infrared original image samples Among them, the most common visible light likelihood pseudo label for the sample is The corresponding visible light cluster center As The matching quasi-visible light cluster center, when the quasi-visible light cluster center and visible light cluster centers When they are equal, then for the original image set, it is considered that The visible light cluster center and the infrared cluster center are matched.

6. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 3, characterized in that: In step 5, the cluster centers corresponding to each image set of each modality are divided into different camera centers by camera labels, which can be expressed as follows: in, Indicates a pseudo label The feature vector set of the c-th image set with camera label s, Indicates belonging The feature vector of the nth image sample, N ks represents the number of samples assigned to camera label s by the clustering algorithm, Indicates that the pseudo label y is obtained by clustering the c-th image set k And belongs to the camera center of camera tag s.

7. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 6, characterized in that: In step 6, The intra-modality camera center contrast loss is It is calculated by the following formula: Among them, q v and q i In the training phase, the visible light original image and the infrared original image are input into the dual-path deep learning model to obtain the visible light features and infrared features, q va and q ia Represents the visible light enhancement features and infrared enhancement features obtained by inputting the corresponding visible light enhanced image and infrared enhanced image into the dual-path deep learning model, and Respectively represent and q v and q i The set of all visible and infrared camera centers with the same cluster center, p j Indicates that q v or q i N nearest homomodal clusters that do not have the same cluster center neg Camera center, and Respectively and The number of camera centers in , τ represents a learnable hyperparameter, Indicates belonging The center of the sth visible light camera, Indicates belonging The center of the t-th infrared camera; The cross-modal camera center contrast loss is It is calculated by the following formula: in, Express expectations.

8. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 7, characterized in that: In step 6, the parameters of the camera center and the two-way deep learning model are updated based on the contrast loss, including: Based on contrast loss and Constructing the total loss Utilize total loss Update the parameters of the two-way deep learning model, Among them, λ cross It is a hyperparameter used to balance the intra-modal and cross-modal weights; During the update process, the image sample vector of each batch is also used to update the center of each camera, which is specifically expressed as: Among them, δ represents the number of iterations, β represents the updated weight, and the next round of training uses the updated camera centers. The arrow ← represents the update. represents the center of the visible light camera, Indicates that the light can be enhanced to the center of the camera, Indicates the center of the infrared camera, Indicates the center of the infrared-enhanced camera.

9. The unsupervised cross-modal person re-identification method based on clustering optimization according to claim 1, characterized in that: In step 7, person re-identification is performed based on the two-way deep learning model with updated parameters, including: The original visible light image or infrared image of the target object to be identified is used as the probe image sample. The probe image sample and the library image sample are input into the two-way deep learning model with updated parameters to identify the corresponding visible light features and infrared features. The visible light features or infrared features are selected to calculate the similarity with the features of the library sample image respectively, and all library sample images are sorted according to the similarity. The identity label of the library sample image with the highest ranking is selected as the identification identity label of the probe image sample.

Images