A visible-infrared pedestrian re-identification method based on personalized perception matching and optimization
By constructing regular and compact clusters and introducing personalized perception matching and optimization modules, the problem of neglecting personalized information in unsupervised visible light-infrared pedestrian re-identification is solved, improving the accuracy and robustness of cross-modal recognition and making it suitable for pedestrian re-identification under label-less conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUILIN UNIV OF AEROSPACE TECH
- Filing Date
- 2026-05-15
- Publication Date
- 2026-07-17
AI Technical Summary
Existing unsupervised visible-infrared pedestrian re-identification methods ignore personalized information within identity clusters during cross-modal matching, resulting in limited recognition performance in complex scenes, especially when affected by factors such as changes in illumination, occlusion, and pose changes, making it difficult to accurately distinguish pedestrians.
By constructing regular clusters and compact clusters, and combining the individual perception matching module and the individual perception optimization module, cross-modal matching and feature optimization are performed using intra-cluster individual information. Homogeneous and heterogeneous individual perception losses are introduced for joint optimization of the image encoder.
It improves the accuracy and robustness of pedestrian re-identification without manual annotation, reduces the cost of manual annotation, and enhances the reliability and feature discrimination ability of cross-modal recognition.
Smart Images
Figure CN122416490A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visible-infrared cross-modal pedestrian re-identification, and in particular to a visible-infrared pedestrian re-identification method based on personalized perception matching and optimization. Background Technology
[0002] Person re-identification (ReID) aims to retrieve images of a specific individual captured by multiple cameras from a large-scale image database. In practical applications, widely deployed cameras typically capture visible light images under well-lit conditions and automatically switch to infrared mode to generate infrared images under low-light conditions. To address this, visible light-infrared person re-identification has gained increasing attention. Benefiting from dedicated model architectures and diverse optimization strategies, existing supervised visible light-infrared person re-identification methods have achieved good performance on several public datasets. However, supervised methods typically rely heavily on manually labeled identity tags, and cross-modal identity labeling requires significant manpower and time costs, severely limiting their widespread application in real-world scenarios.
[0003] Currently, unsupervised visible-infrared pedestrian re-identification has become a research hotspot, and researchers have proposed various unsupervised visible-infrared pedestrian re-identification methods, achieving certain performance on public datasets. However, existing methods often neglect personalized information within identity clusters. Specifically, most methods first perform clustering within each modality to generate multiple clusters, treating all images within the same cluster as belonging to the same identity. Subsequently, the average value of all image features within a cluster is typically used as the cluster center, which is used not only for cross-modal cluster matching but also as the main reference for subsequent feature optimization. However, since the cluster center is obtained by averaging the features of all samples within the cluster, it usually only represents the common information of the cluster and is difficult to fully reflect the personalized features contained in some images. In real-world scenarios, pedestrian images within the same cluster may be affected by factors such as pose changes, occlusion levels, shooting angle, lighting conditions, modal differences, and image quality, thus exhibiting significant individual differences. These personalized features play an important role in accurately distinguishing different pedestrians. For example, when pedestrians wear similar clothing, have similar backgrounds, or the image quality is poor, relying solely on cluster-level common features can easily lead to identity confusion. Personalized information such as pose, local accessories, and contour details can provide crucial discriminatory criteria. Furthermore, existing cross-modal matching methods, when generating cross-modal matching relationships or pseudo-labels, often focus on the global feature similarity represented by the cluster center, neglecting the mining of fine-grained personalized information within the cluster. This results in inaccurate cross-modal matching and identity confusion in complex visible-infrared scenes, limiting further improvements in overall recognition performance. Summary of the Invention
[0004] In view of this, the present invention aims to overcome the limitations of the above-mentioned problems in the prior art and provide a personalized perception matching and optimization pedestrian re-identification method, which can improve the accuracy and robustness of pedestrian re-identification in cross-modal scenarios.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization includes the following steps:
[0007] Step S1: Acquire unlabeled cross-modal pedestrian images and extract image features;
[0008] In step S1, a set of unlabeled visible light images and a set of unlabeled infrared images are obtained, and feature extraction is performed on the images in both modalities using an image encoder to obtain a set of visible light image features and a set of infrared image features, specifically including:
[0009] Step S101: Obtain a set of visible light pedestrian images captured by a visible light camera and a set of infrared pedestrian images captured by an infrared camera.
[0010] Step S102: Input the visible light image and the infrared image into the image encoder respectively, and extract the corresponding depth feature representations;
[0011] Step S103: The extracted visible light image features and infrared image features are used as the basis for subsequent intramodal clustering, cross-modal matching and model optimization, respectively.
[0012] Step S2: Construct regular clusters and compact clusters;
[0013] In step S2, image features are clustered within each modality to obtain regular clusters and compact clusters located within the regular clusters. The centers of the regular clusters and compact clusters are calculated respectively, specifically including:
[0014] Step S201: Within the visible light mode and infrared mode, a clustering algorithm with a first distance threshold is used to cluster the image features to obtain regular clusters;
[0015] Step S202: Within the visible light mode and the infrared mode, a clustering algorithm with a second distance threshold is used to cluster the image features to obtain compact clusters, wherein the second distance threshold is less than the first distance threshold.
[0016] Step S203: Calculate the center of the regular cluster based on the image features in the regular cluster;
[0017] Step S204: Calculate the center of the compact cluster based on the image features in the compact cluster.
[0018] Step S3: Perform homogeneous identity comparison learning based on conventional cluster centers;
[0019] In step S3, a homogeneous identity comparison learning process is constructed based on the regular clustering results within each modality, enabling the image encoder to obtain initial identity-related feature extraction capabilities without manual identity labeling. Specifically, this includes:
[0020] Step S301: Construct the identity comparison loss within the visible light mode based on the conventional cluster centers in the visible light mode;
[0021] Step S302: Construct the identity comparison loss within the infrared modality based on the conventional cluster centers in the infrared modality;
[0022] Step S303: Add the identity comparison losses within the two modalities to obtain the total homogeneous identity comparison loss.
[0023] Step S4: Construct a personalized perception matching module;
[0024] In step S4, a personalized perception matching module, namely the PAM module, is constructed. This module is used to perform cross-modal clustering matching between visible light and infrared modes, and simultaneously utilizes the common information of regular clusters and the personalized information of compact clusters during the matching process. Step S4 specifically includes:
[0025] Step S401: Calculate the common cluster similarity between the centers of the common clusters of the visible light mode and the centers of the common clusters of the infrared mode to obtain the common similarity matrix;
[0026] Step S402: Calculate the compact cluster similarity between the centers of the visible light mode compact clusters and the centers of the infrared mode compact clusters to obtain the compact similarity matrix;
[0027] Step S403: Introduce the regular similarity matrix and the compact similarity matrix into the inter-modal allocation cost to construct the allocation cost matrix;
[0028] Step S404: Perform cross-modal clustering matching based on the allocation cost matrix to obtain the matching relationship between visible light conventional clusters and infrared conventional clusters.
[0029] Step S5: Construct a personalized perception optimization module;
[0030] In step S5, a personality-aware optimization module, namely the PAO module, is constructed. This module includes homogeneous personality-aware loss and heterogeneous personality-aware loss, which are used to introduce the personalized supervision signals provided by compact clusters into the feature learning process within the same modality and between different modalities, respectively. Step S5 specifically includes:
[0031] Step S501: Construct a homogeneous individuality perception loss within each modality, so that image features are close to the center of the compact cluster within their respective regular clusters within the same modality;
[0032] Step S502: Construct a heterogeneous individuality perception loss between the two modalities, so that the image features are close to the center of the compact cluster inside the corresponding cross-modal positive cluster in the cross-modal space;
[0033] Step S503: Homogeneous personality perception loss and heterogeneous personality perception loss are used together with identity contrast loss for image encoder optimization.
[0034] Step S6: Jointly optimize the image encoder and complete cross-modal re-identification;
[0035] In step S6, the image encoder is jointly optimized based on the above loss function, and the optimized image encoder is used to complete the visible light-infrared cross-modal pedestrian re-identification task. Step S6 specifically includes:
[0036] Step S601: In the intramodal homogeneous learning stage, the image encoder is optimized using homogeneous identity contrast loss and homogeneous personality perception loss.
[0037] Step S602: In the cross-modal heterogeneous learning stage, the image encoder is jointly optimized using homogeneous identity contrast loss, heterogeneous identity contrast loss, homogeneous personality perception loss and heterogeneous personality perception loss.
[0038] Step S603: During the training process, personality perception matching and cross-modal heterogeneous learning are performed alternately, so that the image encoder gradually acquires the ability to represent identity features robust to modal differences.
[0039] Step S604: In the inference phase, the image to be queried and the candidate images are input into the trained image encoder to extract image features, calculate the similarity or distance between cross-modal image features, and output the re-identification ranking results according to the similarity from high to low or the distance from small to large.
[0040] The technical solution of the present invention achieves at least the following beneficial effects:
[0041] First, this invention further constructs compact clusters based on conventional clustering, enabling the model to utilize not only common cluster-level information, but also more fine-grained personalized information within the clusters.
[0042] Second, by using a personalized perception matching module to simultaneously consider the similarity of conventional clusters and compact clusters, this invention can introduce personalized cues within clusters during cross-modal clustering matching, thereby improving the reliability of the matching relationship between visible light modes and infrared modes.
[0043] Third, this invention introduces the supervision signals provided by compact clusters into the homogeneous and heterogeneous learning processes through the individual perception optimization module, enabling the model to further learn modality-invariant and identity-related image features.
[0044] Fourth, this invention is applicable to visible light-infrared pedestrian re-identification scenarios without manual identification labels, which can reduce the cost of manual labeling and improve the feature discrimination ability and retrieval effect in unsupervised cross-modal pedestrian re-identification. Attached Figure Description
[0045] The above advantages of the present invention will be more readily understood from the following description of the embodiments in conjunction with the accompanying drawings, wherein:
[0046] Figure 1 This is an overall flowchart of the method of the present invention;
[0047] Figure 2 This is an overall structural diagram of the method of the present invention;
[0048] Figure 3 These are the F-value statistics from the SYSU-MM01 dataset;
[0049] Figure 4 These are the F-value statistics from the RegDB dataset;
[0050] Figure 5 This is the image feature distribution of the A2 method in an ablation experiment according to an embodiment of the present invention;
[0051] Figure 6 This is the image feature distribution of the A3 method in an ablation experiment according to an embodiment of the present invention. Detailed Implementation
[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0053] Figure 1 This is a flowchart of one embodiment of the present invention. Figure 2 This is an overall structural diagram of one embodiment of the present invention.
[0054] like Figure 1 and Figure 2 As shown, a visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization includes the following steps:
[0055] Step S1: Acquire unlabeled cross-modal pedestrian images and extract image features;
[0056] In step S1, a set of unlabeled visible light images and a set of unlabeled infrared images are obtained, and an image encoder is used to extract features from the images in the two modes to obtain a set of visible light image features and a set of infrared image features.
[0057] Furthermore, in one embodiment of the present invention, step S1 specifically includes:
[0058] Step S101: Obtain a set of visible light pedestrian images captured by a visible light camera and a set of infrared pedestrian images captured by an infrared camera.
[0059] Step S102: Input the visible light image and the infrared image into the image encoder respectively, and extract the corresponding depth feature representations;
[0060] Step S103: The extracted visible light image features and infrared image features are used as the basis for subsequent intramodal clustering, cross-modal matching and model optimization, respectively.
[0061] Specifically, the image encoder can employ a deep neural network comprising a modality-specific branch and a shared feature extraction branch. The modality-specific branch adapts to the imaging differences between visible light and infrared images, while the shared feature extraction branch learns identity-related features shared between the two modalities. This approach provides a unified feature representation space for subsequent cross-modal person re-identification tasks.
[0062] Step S2: Construct regular clusters and compact clusters;
[0063] In step S2, the image features are clustered within each modality to obtain regular clusters and compact clusters located within the regular clusters, and the centers of the regular clusters and compact clusters are calculated respectively.
[0064] Furthermore, in one embodiment of the present invention, step S2 specifically includes:
[0065] Step S201: Within the visible light mode and infrared mode, a clustering algorithm with a first distance threshold is used to cluster the image features to obtain regular clusters;
[0066] Step S202: Within the visible light mode and the infrared mode, a clustering algorithm with a second distance threshold is used to cluster the image features to obtain compact clusters, wherein the second distance threshold is less than the first distance threshold.
[0067] Step S203: Calculate the center of the regular cluster based on the image features in the regular cluster;
[0068] Step S204: Calculate the center of the compact cluster based on the image features in the compact cluster.
[0069] Specifically, the clustering algorithm can adopt the DBSCAN clustering algorithm. Let the first distance threshold be e, the second distance threshold be d, and d < e. Since the second distance threshold is more stringent, the compact clustering clusters are usually located inside a certain regular clustering cluster, which is used to represent the more fine-grained personalized feature distribution within the regular clustering cluster.
[0070] For the u-th regular clustering cluster in the visible light modality, its cluster center can be calculated according to Equation 1:
[0071] (1),
[0072] where, represents the center of the u-th regular clustering cluster in the visible light modality, represents the i-th visible light image feature within the regular clustering cluster, and N u represents the number of image features within the regular clustering cluster. Similarly, the centers of the regular clustering clusters in the infrared modality can be calculated .
[0073] For the h-th compact clustering cluster within the u-th regular clustering cluster in the visible light modality, its compact clustering cluster center can be calculated according to Equation 2:
[0074] (2),
[0075] where, [[ID=3l]] represents the center of the h-th compact clustering cluster within the u-th regular clustering cluster in the visible light modality, and N u,h represents the number of image features within the compact clustering cluster. Similarly, the compact clustering cluster centers in the infrared modality can be calculated .
[0076] Through the above method, the regular clustering cluster center is used to characterize the cluster-level common information, and the compact clustering cluster center is used to characterize the personalized information within the cluster.
[0077] Step S3: Perform homogeneous identity contrast learning based on the regular clustering cluster center;
[0078] In step S3, a homogeneous identity contrast learning process is constructed based on the regular clustering results within each modality, enabling the image encoder to obtain the initial identity-related feature extraction ability without artificial identity labels.
[0079] Furthermore, in an embodiment of the present invention, step S3 specifically includes:
[0080] Step S301, construct an identity contrast loss within the visible light modality according to the regular clustering cluster center in the visible light modality;
[0081] Step S302: Construct the identity comparison loss within the infrared modality based on the conventional cluster centers in the infrared modality;
[0082] Step S303: Add the identity comparison losses within the two modalities to obtain the total homogeneous identity comparison loss.
[0083] Specifically, for any image feature in the visible light mode The loss for homogeneous identity comparison can be expressed as calculated according to Formula 3:
[0084] (3),
[0085] in, Representation of features The center of the regular cluster to which it belongs, This represents the set of all conventional cluster centers in the visible light mode. Let represent the number of regular clusters in the visible light mode, τ represent the temperature parameter, and sim(·) represent the feature similarity function. Similarly, the homogeneous identity comparison loss for the infrared mode can be obtained. .
[0086] The total homogeneous identity contrast loss for the two modalities is calculated according to Formula 4:
[0087] (4),
[0088] This loss function can make image features within the same modality closer to the center of their respective regular clusters and farther away from the centers of other regular clusters, thereby improving the image encoder's initial learning ability for identity-related features.
[0089] Step S4: Construct a personalized perception matching module;
[0090] In step S4, a personalized perception matching module, namely the PAM module, is constructed. This module is used to perform cross-modal clustering matching between visible light and infrared modes, and simultaneously utilizes the common information of regular clusters and the personalized information of compact clusters during the matching process.
[0091] Furthermore, in one embodiment of the present invention, step S4 specifically includes:
[0092] Step S401: Calculate the common cluster similarity between the centers of the common clusters of the visible light mode and the centers of the common clusters of the infrared mode to obtain the common similarity matrix;
[0093] Step S402: Calculate the compact cluster similarity between the centers of the visible light mode compact clusters and the centers of the infrared mode compact clusters to obtain the compact similarity matrix;
[0094] Step S403: Introduce the regular similarity matrix and the compact similarity matrix into the inter-modal allocation cost to construct the allocation cost matrix;
[0095] Step S404: Perform cross-modal clustering matching based on the allocation cost matrix to obtain the matching relationship between visible light conventional clusters and infrared conventional clusters.
[0096] Specifically, the similarity between the p-th regular cluster in the visible light mode and the q-th regular cluster in the infrared mode is calculated according to Formula 5:
[0097] (5),
[0098] in, This represents the center of the p-th regular cluster in the visible light mode. This represents the center of the q-th conventional cluster in the infrared mode. This indicates the regular cluster similarity between the two.
[0099] For the compact clusters within the p-th regular cluster in the visible light mode, and the compact clusters within the q-th regular cluster in the infrared mode, their compact cluster similarity is calculated according to Formula 6:
[0100] (6),
[0101] in, This represents the center of the a-th compact cluster within the p-th regular cluster in the visible light mode. This represents the center of the b-th compact cluster within the q-th regular cluster in the infrared mode. This formula allows for the introduction of personalized similarity relationships between compact clusters within a regular cluster during the cluster matching process.
[0102] Furthermore, an allocation cost matrix M is constructed based on regular cluster similarity and compact cluster similarity, where the element in the p-th row and q-th column is calculated according to Formula 7:
[0103] (7),
[0104] Where, γ g and γ t These represent the weight coefficients of the regular cluster similarity term and the compact cluster similarity term, respectively. By adjusting γ... g and γ t It can control the degree of influence of cluster-level common information and intra-cluster personalized information in cross-modal matching.
[0105] Based on the allocation cost matrix, cross-modal clustering matching relationships can be obtained using graph matching or progressive graph matching methods. For example, when the p-th regular cluster in the visible light mode matches the q-th regular cluster in the infrared mode, the q-th regular cluster in the infrared mode is taken as the cross-modal positive cluster of the p-th regular cluster in the visible light mode.
[0106] The aforementioned personalized perception matching module can avoid the problem of insufficient information caused by relying solely on conventional cluster centers for matching, thereby improving the reliability of cross-modal clustering matching relationships.
[0107] Step S5: Construct a personalized perception optimization module;
[0108] In step S5, a personality-aware optimization module, namely the PAO module, is constructed. This module includes homogeneous personality-aware loss and heterogeneous personality-aware loss, which are used to introduce the personalized supervision signals provided by compact clusters into the feature learning process within the same modality and between different modalities, respectively.
[0109] Furthermore, in one embodiment of the present invention, step S5 specifically includes:
[0110] Step S501: Construct a homogeneous individuality perception loss within each modality, so that image features are close to the center of the compact cluster within their respective regular clusters within the same modality;
[0111] Step S502: Construct a heterogeneous individuality perception loss between the two modalities, so that the image features are close to the center of the compact cluster inside the corresponding cross-modal positive cluster in the cross-modal space;
[0112] Step S503: Homogeneous personality perception loss and heterogeneous personality perception loss are used together with identity contrast loss for image encoder optimization.
[0113] Specifically, for any image feature in the visible light mode The loss of perceived homogeneity is calculated according to Formula 8:
[0114] (8),
[0115] in, Representation of features The center of a compact cluster within a regular cluster. Let represent the set of all compact cluster centers in the visible light mode, and τ represent the temperature parameter. Similarly, the homogeneous individuality sensing loss in the infrared mode can be calculated. .
[0116] The total homogeneous individuality perception loss for both modalities is calculated according to Formula 9:
[0117] (9),
[0118] Through this loss function, the fine-grained personalized supervision signals provided by the compact clusters are introduced into the intramodal feature learning process, thereby improving the model's ability to utilize personalized identity cues within the clusters.
[0119] For any image feature in the visible light mode If the cross-modal positive cluster of its regular cluster in the infrared mode has been obtained by step S4, then the heterogeneous individuality perception loss is calculated according to formula 10:
[0120] (10)
[0121] in, Indicating infrared modes with The corresponding compact cluster centers within cross-modal positive clusters. This represents the set of centers of all compact clusters in the infrared modality. Similarly, the heterogeneous individuality perception loss corresponding to infrared image features can be calculated. .
[0122] The total heterogeneous individuality perception loss for both modalities is calculated according to Formula 11:
[0123] (11),
[0124] In one embodiment, to reduce the impact of erroneous cross-modal matching relationships on the training process, an alternating cross-modal contrastive learning approach can be adopted, that is, alternating the use of cross-modal supervision signals from visible light to infrared and from infrared to visible light in different training rounds to enhance the stability of the training process.
[0125] Step S6: Jointly optimize the image encoder and complete cross-modal re-identification;
[0126] In step S6, the image encoder is jointly optimized based on the above loss function, and the optimized image encoder is used to complete the visible light-infrared cross-modal pedestrian re-identification task.
[0127] Furthermore, in one embodiment of the present invention, step S6 specifically includes:
[0128] Step S601: In the intramodal homogeneous learning stage, the image encoder is optimized using homogeneous identity contrast loss and homogeneous personality perception loss.
[0129] Step S602: In the cross-modal heterogeneous learning stage, the image encoder is jointly optimized using homogeneous identity contrast loss, heterogeneous identity contrast loss, homogeneous personality perception loss and heterogeneous personality perception loss.
[0130] Step S603: During the training process, personality perception matching and cross-modal heterogeneous learning are performed alternately, so that the image encoder gradually acquires the ability to represent identity features robust to modal differences.
[0131] Step S604: In the inference phase, the image to be queried and the candidate images are input into the trained image encoder to extract image features, calculate the similarity or distance between cross-modal image features, and output the re-identification ranking results according to the similarity from high to low or the distance from small to large.
[0132] Specifically, in the intramodal homogeneous learning phase, the overall loss function is calculated according to Formula 12:
[0133] (12)
[0134] Among them, L hoc L represents the loss for homogeneous identity comparison. hop λ represents the loss of perceived homogeneity. hop The weighting coefficient represents the loss of perceived homogeneity.
[0135] In the cross-modal heterogeneous learning phase, the overall loss function is calculated according to Formula 13:
[0136] (13)
[0137] Among them, L hec L represents the heterogeneous identity contrast loss. hep λ represents the loss of perception of heterogeneous individuality. hep The weighting coefficient represents the loss of perception of heterogeneous individuality.
[0138] Through the above-mentioned overall loss function, the present invention can simultaneously utilize the cluster-level supervision signal provided by conventional clustering and the personalized supervision signal provided by compact clustering, enabling the image encoder to learn more stable, more discriminative, and modally invariant identity features under unsupervised conditions.
[0139] After model training is complete, for the input visible light query image and infrared candidate image, or infrared query image and visible light candidate image, feature representations are extracted using the trained image encoder, and the similarity between the query image features and candidate image features is calculated. Based on the similarity ranking, the candidate images with the highest ranking are output as the final cross-modal person re-identification results.
[0140] The following specific embodiment further illustrates the unsupervised cross-modal pedestrian re-identification method based on personalized perception matching and optimization proposed in this invention.
[0141] In this embodiment, the method of the present invention is trained and verified using the publicly available visible-infrared pedestrian re-identification dataset that includes visible light images and infrared images. The image encoder uses a backbone network based on ResNet-50, and two modality-specific branches are set at the front end of the network to adapt to the modality feature differences between visible light images and infrared images respectively. In the training stage, the DBSCAN clustering algorithm is used to perform intra-modal clustering on the visible light image features and infrared image features respectively, and the minimum sample number is set to 4. As Figure 3 and Figure 4 , for the RegDB dataset, the distance threshold corresponding to the regular clustering cluster is set to e = 0.3, and the distance threshold corresponding to the compact clustering cluster is set to d = 0.25, where d < e, to ensure that the compact clustering cluster is located inside the regular clustering cluster, so as to obtain more fine-grained personalized information within the cluster. The batch size during training is set to 128. Each time during training, 8 clustering clusters are randomly selected within each modality, and 16 images are randomly sampled from each clustering cluster. The Adam optimizer is used as the optimizer, and the learning rate is set to 0.00035; both the homogeneous learning stage and the heterogeneous learning stage are trained for 50 rounds; the temperature parameter τ is set to 0.05, and the weight parameters corresponding to the regular clustering cluster similarity and the compact clustering cluster similarity are set to 1.0 and 2.0 respectively. In the testing stage, the trained image encoder is used to extract the feature representations of the query image and the candidate image respectively, and pedestrian re-identification retrieval is completed according to the cross-modal feature similarity. Tables 1 and 2 show the recognition performance achieved by the method of the present invention on multiple publicly available datasets. The experimental results show that the method of the present invention has achieved good recognition performance on multiple publicly available datasets.
[0142] Table 1: Comparison with existing methods on the SYSU-MM01 dataset
[0143]
[0144] In Table 1, the top two best results under the unsupervised setting are marked in bold and underlined respectively.
[0145] Table 2 Comparison with existing methods on the RegDB dataset
[0146]
[0147] In Table 2, the top two best performance results under the unsupervised setting are marked in bold and underlined respectively.
[0148] Table 3 Ablation experiments on SYSU-MM01 and RegDB
[0149]
[0150] As can be seen from Table 3, introducing L hepAfter that, A4 outperformed A2. Specifically, on the SYSU-MM01 dataset, A4 performed better.
[0151] The feature distributions of images A2 and A3 are shown in the appendix. Figure 5 and Figure 6 Different colors represent different identity characteristics. It can be seen that compared with A2, A3 exhibits higher intra-cluster compactness and inter-cluster separability, and the model performance is effectively improved.
[0152] In summary, this invention introduces conventional clusters and compact clusters into the unsupervised visible-infrared pedestrian re-identification process, improves the reliability of cross-modal clustering matching by utilizing a personalized perception matching module, and incorporates personalized supervision signals within clusters into the model training process by utilizing a personalized perception optimization module. This effectively enhances the image encoder's ability to express modality-invariant and identity-related image features.
[0153] It should be understood that any parts not described in detail in this specification belong to the prior art.
[0154] The above description, in conjunction with the accompanying drawings, is merely a specific embodiment and process of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art should understand that various changes, substitutions, or combinations can be made to the above embodiments without departing from the essential content of the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims.
[0155] The embodiments described with reference to the accompanying drawings are exemplary and are only used to explain the invention. They should not be construed as limiting the invention. The specific scope of the embodiments of the invention is not limited thereto. On the contrary, all embodiments of the invention include all variations and modifications that fall within the spirit and scope of the appended claims.
Claims
1. A visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization, characterized in that, Includes the following steps: Step S1: Acquire unlabeled cross-modal pedestrian images and extract image features; Obtain a set of unlabeled visible light images and a set of unlabeled infrared images, and use an image encoder to extract features from the images in the two modes respectively to obtain a set of visible light image features and a set of infrared image features; Step S2: Construct regular clusters and compact clusters; Within each modality, image features are clustered to obtain regular clusters and compact clusters located within the regular clusters. The centers of the regular clusters and compact clusters are calculated respectively. Step S3: Perform homogeneous identity comparison learning based on conventional cluster centers; Based on the regular clustering results within each modality, a homogeneous identity comparison learning process is constructed, enabling the image encoder to obtain the initial identity-related feature extraction capability without human identity labeling. Step S4: Construct a personalized perception matching module; Construct a personalized perception matching module, namely the PAM module; This module is used to perform cross-modal clustering matching between visible light modes and infrared modes, and simultaneously utilizes the common information of regular clusters and the personalized information of compact clusters during the matching process; Step S5: Construct a personalized perception optimization module; A personality perception optimization module, namely the PAO module, is constructed. This module includes homogeneous personality perception loss and heterogeneous personality perception loss, which are used to introduce the personalized supervision signals provided by compact clusters into the feature learning process within the same modality and between different modalities, respectively. Step S6: Jointly optimize the image encoder and complete cross-modal re-identification; The image encoder is optimized based on homogeneous identity comparison loss, heterogeneous identity comparison loss, homogeneous personality perception loss, and heterogeneous personality perception loss, and the optimized image encoder is used to complete the visible light-infrared cross-modal pedestrian re-identification task.
2. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S1 specifically includes the following sub-steps: Step S101: Obtain a set of visible light pedestrian images captured by a visible light camera and a set of infrared pedestrian images captured by an infrared camera. Step S102: Input the visible light image and the infrared image into the image encoder respectively, and extract the corresponding depth feature representations; Step S103: The extracted visible light image features and infrared image features are used as the basis for subsequent intramodal clustering, cross-modal matching and model optimization, respectively.
3. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S2 specifically includes the following sub-steps: Step S201: Within the visible light mode and infrared mode, a clustering algorithm with a first distance threshold is used to cluster the image features to obtain regular clusters; Step S202: Within the visible light mode and the infrared mode, a clustering algorithm with a second distance threshold is used to cluster the image features to obtain compact clusters, wherein the second distance threshold is less than the first distance threshold. Step S203: Calculate the center of the regular cluster based on the image features in the regular cluster; Step S204: Calculate the center of the compact cluster based on the image features in the compact cluster.
4. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S3 specifically includes the following sub-steps: Step S301: Construct the identity comparison loss within the visible light mode based on the conventional cluster centers in the visible light mode; Step S302: Construct the identity comparison loss within the infrared modality based on the conventional cluster centers in the infrared modality; Step S303: Add the identity comparison losses within the two modalities to obtain the total homogeneous identity comparison loss.
5. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S4 specifically includes the following sub-steps: Step S401: Calculate the common cluster similarity between the centers of the common clusters of the visible light mode and the centers of the common clusters of the infrared mode to obtain the common similarity matrix; Step S402: Calculate the compact cluster similarity between the centers of the visible light mode compact clusters and the centers of the infrared mode compact clusters to obtain the compact similarity matrix; Step S403: Introduce the regular similarity matrix and the compact similarity matrix into the inter-modal allocation cost to construct the allocation cost matrix; Step S404: Perform cross-modal clustering matching based on the allocation cost matrix to obtain the matching relationship between visible light conventional clusters and infrared conventional clusters.
6. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S5 specifically includes the following sub-steps: Step S501: Construct a homogeneous individuality perception loss within each modality, so that image features are close to the center of the compact cluster within their respective regular clusters within the same modality; Step S502: Construct a heterogeneous individuality perception loss between the two modalities, so that the image features are close to the center of the compact cluster inside the corresponding cross-modal positive cluster in the cross-modal space; Step S503: Homogeneous personality perception loss and heterogeneous personality perception loss are used together with identity contrast loss for image encoder optimization.
7. The visible light-infrared pedestrian re-identification method based on personalized perception matching and optimization according to claim 1, characterized in that, Step S6 specifically includes the following sub-steps: Step S601: In the intramodal homogeneous learning stage, the image encoder is optimized using homogeneous identity contrast loss and homogeneous personality perception loss. Step S602: In the cross-modal heterogeneous learning stage, the image encoder is jointly optimized using homogeneous identity contrast loss, heterogeneous identity contrast loss, homogeneous personality perception loss and heterogeneous personality perception loss. Step S603: During the training process, personality perception matching and cross-modal heterogeneous learning are performed alternately, so that the image encoder gradually acquires the ability to represent identity features robust to modal differences. Step S604: In the inference phase, the image to be queried and the candidate images are input into the trained image encoder to extract image features, calculate the similarity or distance between cross-modal image features, and output the re-identification ranking results according to the similarity from high to low or the distance from small to large.