Pedestrian re-identification method based on camera perception reciprocity optimization

Abstract

The invention belongs to the field of computer vision, and particularly relates to a pedestrian re-identification method based on camera perception reciprocity optimization. According to the technical scheme, the problem of camera domain offset and cross-modal correspondence in unsupervised visible light-infrared pedestrian re-identification is jointly solved, and single-modal clustering and cross-modal alignment are optimized through double-module design. According to the framework, a camera neighborhood purification CNP module and a reciprocal centroid optimization RCR module work cooperatively. The CNP module effectively captures camera domain differences through a differentiated neighborhood search strategy, and a layered neighborhood selection mechanism significantly improves the single-mode clustering quality, thereby laying a foundation for high-quality pseudo label generation. The RCR module introduces a bidirectional iteration centroid optimization mechanism to realize mutual refinement of visible light and infrared clustering centroids, and the mechanism not only ensures mapping integrity, but also obtains better cross-modal corresponding accuracy through progressive optimization.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a camera-aware reciprocity optimization pedestrian re-identification method. BACKGROUND

[0002] The main goal of pedestrian re-identification (Re-ID) is to match query pedestrian images with gallery images from different cameras. However, traditional pedestrian re-identification is usually based on visible light images, which performs well in well-lit scenes, but in complex environments with weak light, night, or dramatic light changes (such as underground parking lots, night streets, etc.), visible light cameras have difficulty capturing clear pedestrian features, resulting in a significant decline in recognition performance. To improve the applicability of pedestrian re-identification in different lighting conditions, "visible-infrared" pedestrian re-identification is proposed, which allows matching of visible light and infrared pedestrian images in different lighting scenarios. However, the technical solution of supervised learning requires a large amount of cross-modal paired labeled data (i.e., visible light and infrared image pairs of the same pedestrian), and the labeling process requires manual alignment of different "visible-infrared" image pairs, which is costly and prone to noise.

[0003] Therefore, unsupervised "visible-infrared" pedestrian re-identification technology has gradually become a research hotspot. This technology addresses the key issues of traditional methods, such as strong dependence on labeled data and limited scalability, by exploring cross-modal feature expression and matching mechanisms under unsupervised learning, enabling effective pedestrian re-identification without labeling, thereby improving the practicality and generalization ability of the model in real complex scenarios. Existing methods simply cluster samples of a specific modality and use strong association techniques to achieve cross-modal association from instances to clusters or from clusters to clusters. For example, Yang et al. in the paper "Augmented Dual-Contrastive Aggregation Learning for Unsupervised Visible-Infrared Person Re-Identification" proposed an augmented dual-contrastive aggregation learning framework, designed a cross-modal memory aggregation module to filter high-correlation positive samples and aggregate memory features at the cluster level, ensuring that optimization focuses on modality-independent. The model structure is as shown in Figure 1 In addition, Wu et al. in "Unsupervised Visible-Infrared Person Re-Identification via Progressive Graph Matching and Alternate Learning" proposed a progressive graph matching (PGM) method, and the model structure is as shown inFigure 2 The method solves the global mining problem of cross-modal correspondence in the unbalanced clustering scene by modeling the cross-modal correlation mining as a graph matching process and introducing a global matching cost minimization strategy.

[0004] However, the prior art solution has two key technical defects: on the one hand, it fails to solve the camera domain offset problem within the modal, resulting in feature distribution offset of the same identity pedestrian in the single camera and in the cross-camera scene due to the difference in viewing angle; on the other hand, it fails to accurately establish the cross-modal clustering correspondence. Specifically, at the modal level, due to the lack of differentiated feature measurement mechanism, the existing method cannot effectively distinguish between high-reliability positive samples within the camera and complementary auxiliary samples across cameras, resulting in blurred clustering boundaries. At the cross-modal level, the traditional static matching method not only fails to fully mine the complementary information between the visible light and infrared modal, but also lacks an effective iterative optimization mechanism to gradually correct the correspondence. As shown in the figure, this double technical defect leads to two typical phenomena: the intra-modal camera difference causes clustering splitting, the cross-modal difference causes label association ambiguity, and finally the cross-modal pseudo-label correspondence has uncertainty, which significantly reduces the performance of unsupervised “visible light-infrared” pedestrian re-identification. Figure 3

[0005] The existing unsupervised “visible light-infrared” pedestrian re-identification technology only focuses on the modal difference and ignores the camera domain offset, resulting in cross-camera feature drift within the single modal, and the static one-time matching cannot iteratively correct the cross-modal clustering correspondence, resulting in large pseudo-label noise and low accuracy.

[0006] To address the above problems, the present application proposes a technical solution of a camera-aware reciprocal refinement (CARR) framework, which jointly solves the camera domain offset and cross-modal correspondence problems in unsupervised “visible light-infrared” pedestrian re-identification, and optimizes single-modal clustering and cross-modal alignment with a double-module design. The framework works collaboratively with the “camera neighborhood purification” (CNP) module and the “reciprocal centroid optimization” (RCR) module. The CNP module effectively captures the camera domain difference through a differentiated neighborhood search strategy, and the hierarchical neighborhood selection mechanism significantly improves the quality of single-modal clustering, laying the foundation for high-quality pseudo-label generation. The RCR module introduces a bidirectional iterative centroid optimization mechanism to realize mutual refinement of the visible light and infrared clustering centroids. This mechanism not only ensures the mapping integrity, but also obtains a better cross-modal correspondence accuracy through progressive optimization. SUMMARY

[0007] ​The application aims to solve the technical problems in the prior art, and provides a camera-aware reciprocal optimization pedestrian re-identification method, which is suitable for unsupervised "visible light-infrared" pedestrian re-identification tasks. The method can eliminate the single-modal cross-camera feature drift caused by camera domain bias, solve the limitation of static one-time matching, and significantly improve the recognition accuracy by iteratively correcting the cross-modal clustering correspondence.

[0008] The technical scheme of the method is as follows: a camera-aware reciprocal optimization pedestrian re-identification method comprises the following steps: step 1: load the visible light and infrared image data in the data set respectively, the data will be input into the encoder model based on the deep neural network ResNet50, set the Adam optimizer and the learning rate scheduler, and enter step 2.

[0009] Step 2: apply multiple data enhancement methods to the input images to generate diversified training samples, including size adjustment, random cropping, horizontal flipping, channel random erasing and channel exchange enhancement for RGB images, and similar enhancement strategies for IR images plus channel adaptive grayscale enhancement, to simulate the changes in real scenes and improve the generalization ability of the model, and enter step 3.

[0010] Step 3: at the beginning of each epoch, use the pre-trained model to extract the feature vectors of RGB and IR images, combine the real camera label information, and realize camera cooperative neighborhood screening through the CNP module, which can effectively distinguish the same camera internal neighbors and cross-camera neighbors, and purify the noise neighborhood relationship, on this basis, calculate the cross-camera Jaccard rearrangement distance matrix, and use the DBSCAN clustering algorithm to cluster the unlabeled RGB and IR images, and assign pseudo-labels to the clustered images, and enter step 4.

[0011] Step 4: create two memory banks (visible light mode, infrared mode) for feature storage and contrast learning, calculate the feature center points of the RGB and IR clusters, and the RCR module executes the bidirectional iterative consistency alignment algorithm to calculate the bidirectional mapping relationship (i2r, r2i) of IR→RGB and RGB→IR, ensure that each RGB cluster has a corresponding IR cluster matching, realize the unified memory mechanism of cross-modal feature representation, and enter step 5.

[0012] Step 5: create RGB and IR training data sets according to the pseudo-labels, input the cross-modal mapping relationship (i2r, r2i) into the CARR framework, calculate the bidirectional cross-modal contrast loss (visible light to infrared and infrared to visible light mutual mapping), dynamically maintain the cross-modal memory bank features, and update the model parameters using back propagation, and enter step 6.

[0013] Step 6: Update the learning rate after each epoch, evaluate periodically on SYSU-MM01 test set, calculate Rank-n accuracy, mAP indicators, save the best performance model checkpoint, if the specified number of training rounds is reached, end the training process, perform final evaluation, then proceed to Step 7, otherwise continue training and return to Step 3.

[0014] Step 7: Load the best model weight saved during training, perform 10 random trials in full search mode (using all cameras) and indoor search mode (using only indoor cameras) in the test set, use Rank-n accuracy, average precision mean (mAP) and other indicators to measure model effectiveness, output total training time and performance report and end.

[0015] As a preferred, the images in step 1 are all from the visible infrared pedestrian re-identification standard dataset SYSU-MM01 dataset, which contains 491 pedestrian identities of multi-modal images composed of visible light images and infrared images, and contains 6 cameras, among which cam1, cam2, cam4, cam5 are visible light cameras, and cam3, cam6 are infrared cameras, which provides rich camera perspective changes and modal differences for cross-modal pedestrian re-identification research.

[0016] As a preferred, in step 2, the RGB image is enhanced using horizontal flip, random cropping, color jitter, channel random erasing and channel exchange, and the IR image is enhanced using horizontal flip, random cropping, channel random erasing and channel adaptive grayscale enhancement, etc. Specific to the modal enhancement strategy, the generalization ability and robustness of the model are improved by simulating different light conditions, perspective changes and noise interference.

[0017] As a preferred, in step 3, the camera coordination core principle of the CNP module is to first convert the cosine distance to Euclidean distance to construct an initial distance matrix, and then use the real camera label to construct a camera mask to distinguish same-camera and cross-camera sample pairs. This module uses the k-reciprocal neighbor algorithm to preferentially select same-camera nearest neighbors by adding a penalty value to cross-camera samples, and preferentially select cross-camera nearest neighbors by adding a penalty value to same-camera samples, and further purifies the neighbor relationship through a local query expansion mechanism, and finally generates a camera-aware CA-Jaccard distance matrix for subsequent clustering analysis, effectively solving the problem that traditional Jaccard distance cannot handle camera bias, the specific implementation is as follows: First, calculate the single-modal distance matrix, given the L2 normalized feature matrix and where and represent the number of visible light and infrared samples, is the feature dimension. The cosine distance matrix between samples is calculated as follows:

[0018] The inner product of normalized vectors is equivalent to the cosine similarity, while the corresponding distance metric is obtained by a linear transformation of the similarity.

[0019] To distinguish intra-camera and cross-camera sample pairs, a camera mask matrix is constructed. A binary mask matrix is constructed based on the camera ID vectors. Let and denote the camera ID vectors of the visible and infrared samples, respectively, where and are the number of cameras for each modality. The mask matrix is defined as:

[0020] Based on the previously calculated single-modality distance matrix and the camera mask matrix, the camera-aware sample ranking is achieved through a penalty mechanism. The ranking function returns the sample indices sorted in ascending order of distance. For each modality , the following is calculated:

[0021] where the penalty coefficient controls the ranking behavior, by penalizing the selection of intra-camera neighbors for cross-camera samples, and by penalizing the selection of cross-camera neighbors for intra-camera samples.

[0022] Based on the above definitions, camera-aware neighborhood aggregation is performed to calculate the k-reciprocal neighbors of each sample under intra-camera and cross-camera conditions, respectively. For each modality :

[0023] where and denote the neighborhood size for intra-camera and cross-camera scenarios, respectively. This process is consistently handled in both the visible and infrared modalities.

[0024] Based on the camera-aware k-reciprocal neighbors, a similarity weight matrix is constructed by applying SoftMax normalization to the k-reciprocal neighbor set. For each modality :

[0025] This weight calculation method ensures that both the visible and infrared modalities can obtain their respective similarity weight matrices based on their camera-aware neighborhoods.

[0026] In addition, the camera-aware neighborhood expansion further optimizes the clustering quality by hierarchical neighborhood ranking, as shown in Figure 6 For each modality and sample :

[0027]

[0028] where and are modality-specific expansion parameters.

[0029] Secondly, the expanded similarity matrix is converted to the camera-aware Jaccard distance. For each modality :

[0030] This distance measure is independently calculated on the visible light and infrared modalities, effectively capturing the respective camera-aware neighborhood structure characteristics.

[0031] Finally, the camera-aware Jaccard distance matrix is applied to the clustering generation of each modality using a density-based clustering algorithm:

[0032] These camera-aware clustering results will serve as the basis for establishing cross-modality associations in the RCR module, ensuring that single-modality clustering quality can directly improve cross-modality alignment performance.

[0033] As a preferred, in step 4, first calculate the feature center points of RGB and IR clusters based on the clustering results and perform L2 normalization processing, creating two independent memory banks for feature storage and contrast learning of the visible light and infrared modalities. This step uses a bidirectional iterative consistency alignment algorithm to perform initial matching assignment by calculating the similarity matrix between IR features and RGB features, and then performs an iterative optimization process. First, calculate the RGB centroid according to the current assignment relationship, find the IR sample assigned to it for each RGB cluster and calculate the IR centroid, and update the RGB centroid using a weighted fusion strategy. Then perform RGB→IR reverse optimization, recalculate the similarity with the updated RGB centroid and perform reverse assignment, and update the IR centroid using a weighted fusion strategy. Finally, recalculate the similarity matrix and assignment relationship using the updated features, and stop iteration when the assignment result converges. The algorithm constructs bidirectional mapping relationships of IR→RGB and RGB→IR, and forcibly ensures that each RGB cluster has a corresponding IR cluster match, achieving a unified memory mechanism for cross-modality feature representation, effectively solving the problem of inconsistent cluster numbers and feature distribution differences between different modalities.

[0034] The RCR module establishes cross-modal associations between camera perception clusters through a bidirectional iterative consensus alignment mechanism. Given a set of cluster center points from CANP With The module aims to learn the optimal bidirectional mapping relationship to achieve precise alignment of cross-modal semantically similar clusters.

[0035] The algorithm first performs L2 normalization on all cluster centers to ensure the scale invariance of feature computation. This normalization operation effectively eliminates the magnitude differences between different modal features, so that the cross-modal comparison is not affected by the inherent feature scale of each modality:

[0036] This preprocessing step is crucial for maintaining numerical stability, as it ensures that subsequent similarity calculations reflect true semantic associations rather than feature magnitude differences.

[0037] Based on normalized features, the initial cross-modal similarity matrix is first calculated, and the preliminary correspondence is established according to the maximum cosine similarity. This initialization process provides a stable starting benchmark for subsequent iterative optimization:

[0038] The resulting matching vector An initial one-to-many mapping relationship from infrared clusters to visible light clusters is constructed, laying the foundation for subsequent bidirectional optimization.

[0039] The forward optimization phase optimizes the visible light cluster representation by fusing the feature information of the matched infrared clusters. For each visible light cluster, first identify all matched infrared clusters at the current stage, then update through weighted fusion, where the fusion parameter β is used to balance cross-modal information integration and original cluster feature preservation:

[0040]

[0041] To ensure bidirectional consistency, reverse matching calculation is needed. The algorithm calculates the reverse matching pointing to the infrared cluster based on the updated visible light cluster. This reverse matching not only verifies the effectiveness of the forward matching, but also provides guidance for subsequent infrared cluster optimization, thus maintaining the symmetry of the optimization process:

[0042] The reverse matching vector The complementary mapping mechanism ensures that each visible light cluster has a corresponding infrared cluster, thus constructing a symmetric information interaction channel for the subsequent reverse optimization phase.

[0043] After the reverse matching is completed, the reverse optimization phase is carried out, and the algorithm optimizes the infrared clustering by fusing the visible light clustering features. This two-way information exchange mechanism can balance cross-modal feature fusion and avoid the algorithm from falling into a local optimal solution.

[0044]

[0045] The symmetric optimization mechanism forms a mirror relationship with the forward phase, ensuring that both modalities can benefit equally from cross-modal information exchange. By using the fusion weight β uniformly, the algorithm maintains the consistency of two-way optimization, effectively preventing preference bias for a single modality.

[0046] After completing the two-way optimization, the matching is updated and the convergence is determined. The algorithm recalculates the cross-modal matching relationship based on the updated clustering features. This step determines whether the iterative optimization has reached a convergent state by monitoring the stability of the matching:

[0047] After each feature update, normalization is performed to maintain unit norm constraints. When is satisfied, the algorithm is terminated, and the clustering correspondence relationship has reached a stable consensus state.

[0048] After the algorithm converges times, two complementary mappings will be constructed. The infrared-to-visible light mapping records the final matching decision, while the visible light-to-infrared mapping ensures full coverage, ensuring that all clusters participate in cross-modal correspondence:

[0049]

[0050] These two-way mappings establish reliable positive sample pairs across modalities, enabling robust cross-modal contrast learning. The forced matching strategy ensures complete coverage of the entire clustering space.

[0051] As preferred, the CARR framework in step 5 performs cross-modal contrastive learning training. The framework utilizes the bidirectional mapping relationship obtained in step 4 to realize cross-modal label alignment conversion. The core mechanism of the CARR framework includes three levels of loss calculation: first, intra-modal contrastive loss, which is used for same-modal identity feature aggregation learning in the RGB and IR memory banks respectively. Second, cross-modal contrastive loss, which uses the mapping relationship to perform cross-modal contrastive learning between IR features and the RGB memory bank or between RGB features and the IR memory bank. Third, an alternating training strategy is adopted, which uses IR features to contrast the RGB memory bank in odd epochs and uses RGB features to contrast the IR memory bank in even epochs, avoiding training imbalance. The framework combines the intra-modal loss and the weighted cross-modal loss to form the total loss function, which updates the network parameters through backpropagation while dynamically maintaining the memory bank features, realizing stable and efficient cross-modal identity matching learning.

[0052] The CARR framework stores and updates discriminative clustering representations by maintaining two independent clustering centroid memory banks for each modality. Define For L2 normalized features, the memory banks for visible light and infrared modalities are represented as and .

[0053] In the initial stage, the clustering centroids are calculated from the mean of the normalized features within the camera perception clusters generated by the CNP module:

[0054] During training, the momentum update mechanism is used to realize dynamic updating of the memory bank. In the first training cycle, the clustering centroids recalculated are:

[0055] The momentum update formula for the memory bank is:

[0056] where the momentum coefficient This dual-memory bank architecture maintains the cross-modal compatibility of the RCR module while realizing independent learning of single-modal features.

[0057] Finally, the loss function calculation of contrastive learning is realized, and the CARR framework adopts a dual-dimension contrastive learning strategy that integrates single-modal and cross-modal learning. In single-modal learning, each modality enhances the discriminative ability of the same-domain features by optimizing the clustering-level contrastive loss. To achieve batch balancing and feature diversity enhancement, the channel enhancement technique is used for the visible light modality to generate a balanced batch size after processing The single-modal contrastive loss function is defined as:

[0058] wherein represents the enhanced visible light sample, is the temperature coefficient. The robust clustering representation constructed by the single-modal learning mechanism lays a foundation for subsequent cross-modal alignment through the RCR module.

[0059] In the cross-modal alignment stage, the RCR module establishes the reciprocal mapping functions (infrared→visible light) and (visible light→infrared) through the bidirectional iterative consensus with completeness constraints. It should be particularly noted that the cross-modal loss function is only used in the second stage of pseudo-label alignment. The cross-modal contrast loss is defined as:

[0060] To prevent gradient conflicts and ensure training stability, the framework adopts an alternating optimization strategy:

[0061]

[0062] All total loss functions of the complete optimization target:

[0063] Test flow of the present application Step a: Prepare and input to construct a query set and a gallery set, wherein the query set is composed of pedestrian images collected by infrared cameras (cam3, cam6), and the gallery set is composed of pedestrian images collected by visible light cameras. According to different test scenes, the gallery set contains images of four cameras cam1, cam2, cam4 and cam5 in all scene modes, and contains images of only two cameras cam1 and cam2 in indoor scene mode. After completing data preprocessing, proceed to the next step.

[0064] Step b: Use the trained cross-modal pedestrian re-identification model to extract features of all pedestrian images in the query set and the gallery set respectively, and obtain high-dimensional feature vector representation through the backbone network. In order to improve the robustness of the features, the horizontal flip enhancement strategy is adopted, the features of the original image and the flipped image are extracted respectively, then the average is taken, and L2 normalization processing is performed. After the feature extraction is completed, proceed to the similarity calculation stage.

[0065] Step c: The dot product distance calculation method is used for cross-modal similarity measurement, and the similarity matrix is constructed by calculating the inner product operation of the query feature vector and the gallery feature vector. Each element in the similarity matrix represents the matching degree of the corresponding query sample and the gallery sample, and the larger the value is, the higher the similarity is. After completing the similarity calculation between all query images and gallery images, proceed to the result evaluation stage.

[0066] Step d: Sort the matching results of each query sample in descending order according to the similarity matrix, and use Rank-n accuracy and mean average precision (mAP) as evaluation indicators. Rank-n represents the proportion of correct matches in the top n positions of the ranking results. Rank-1, Rank-10, and Rank-20 are calculated. mAP reflects the comprehensive performance of the retrieval system at different recall rates. During evaluation, the standard protocol is followed, and matches between query images and images from the same camera are excluded to ensure fair comparison.

[0067] Step e: Output the complete performance evaluation results, including Rank-1 / 10 / 20 accuracy and mAP values, evaluated in both all-scenario mode and indoor-scenario mode to comprehensively test the performance of the model in different environments. To ensure the statistical significance and reliability of the results, 10 repeated tests are performed and the average values are calculated, and finally a detailed performance analysis report is generated.

[0068] As a preferred, in step b, the feature extraction process adopts a horizontal flip data augmentation strategy, and features are extracted from the original image and the horizontally flipped image respectively. The two sets of feature vectors are averaged and fused, and the fused features are subjected to L2 normalization processing to enhance the robustness and discriminability of the feature representation.

[0069] As a preferred, in step d, the evaluation process strictly follows the standard evaluation protocol for cross-modal pedestrian re-identification, and excludes matching pairs between query images and gallery images from the same camera when calculating matching results, avoiding unfair comparison due to camera bias, and ensuring the objectivity and reliability of the evaluation results.

[0070] As a preferred, in step e, the test evaluation is independently conducted in two different test conditions of all-scenario mode and indoor-scenario mode. By comparing and analyzing the performance in different environments, the adaptability and stability of the model in complex outdoor environments and controlled indoor environments are comprehensively verified, providing reliable performance reference for practical applications.

[0071] As a preferred, the entire test process uses a statistical verification method of multiple repeated experiments. The evaluation results under each test condition are independently tested 10 times, and the mean and standard deviation of each evaluation indicator are calculated to ensure the statistical significance and repeatability of the test results.

[0072] Compared with the prior art, the present application has the following beneficial effects: (1) The application proposes a camera-aware reciprocity optimization pedestrian re-identification method, which is specially used to solve the camera domain offset and cross-modal correspondence inaccuracy problems in the unsupervised "visible light-infrared" pedestrian re-identification, and realizes higher recognition accuracy through cooperative processing of single-modal camera change and cross-modal feature alignment.

[0073] (2) The method of the application takes ResNet50 neural network as a feature extraction backbone network, and constructs a double-module cooperative CARR framework composed of a camera-aware neighborhood purification (CNP) module and a reciprocity centroid optimization (RCR) module. The CNP module, as a basic component, explicitly distinguishes the sample relationship within the same camera and between the cross-camera through a hierarchical neighbor selection strategy, and uses different k value parameters to process the reliable neighbors within the same camera and the complementary neighbors between the cross-camera, effectively alleviating the fragmentation problem of identity features caused by changes in view angle and illumination.

[0074] (3) The CNP module innovatively adopts a camera-aware penalty mechanism, which distinguishes the same camera and cross-camera sample pairs by constructing a camera mask matrix and applies different distance sorting strategies. This mechanism uses the k-reciprocity nearest neighbor algorithm to establish a reliable neighborhood relationship, ensuring that only high-confidence matches are retained in the purified neighbor set, thereby constructing a more robust camera-aware Jaccard distance metric, laying a foundation for high-quality cluster pseudo-label generation.

[0075] (4) The RCR module is based on the high-quality clustering results output by the CNP, and replaces the traditional static matching strategy through a bidirectional iterative centroid optimization mechanism. This module realizes the iterative convergence process of forward optimization (infrared information guiding visible light centroid adjustment) and reverse optimization (visible light information guiding infrared centroid adjustment), balances cross-modal information integration and original feature preservation through a weighted fusion strategy, while ensuring mapping integrity, and establishes a dynamic optimized cross-modal feature mapping relationship.

[0076] (5) In the full-scene search mode of the SYSU-MM01 standard dataset, the method of the application achieves a Rank-1 accuracy of 55.37% and an average precision (mAP) of 52.63%, and in the indoor search mode, it achieves 62.70% and 68.73% respectively. Compared with existing unsupervised "visible light-infrared" pedestrian re-identification methods, through the synergistic effect of camera awareness and reciprocity optimization, this framework significantly improves the cross-modal matching accuracy and feature discrimination ability, providing an effective technical solution for cross-modal pedestrian identification in complex monitoring scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0077] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation of the application.

[0078] Figure 1 Technical flow chart proposed for the prior art "Augmented Dual-Contrastive Aggregation Learning for Unsupervised Visible-Infrared Person Re-Identification"; Figure 2 Technical flow chart proposed for the prior art "Unsupervised Visible-Infrared Person Re-Identification via Progressive Graph Matching and Alternate Learning"; Figure 3 For the prior art, the clustering is split due to the difference between the intra-modal camera, and the cross-modal difference between the visible light and infrared modalities causes the label association to be ambiguous. Figure 4 For the network framework of the method of the present application; Figure 5 For the training flow chart of the network of the present application; Figure 6 For the camera-aware neighborhood expansion step in the CNP module of the present application. DETAILED DESCRIPTION

[0079] The present application will be described in detail below with reference to the accompanying drawings to further explain and illustrate the present application, so that those skilled in the art can have a deeper understanding of the present application and can implement it. However, the following examples are only used to explain the present application, and are not limiting.

[0080] The present application proposes a camera-aware mutual optimization unsupervised "visible light-infrared" pedestrian re-identification method, which uses a deep learning network framework as shown in Figure 4 .

[0081] The present application specially designs a training strategy, and the training flow chart of the proposed network is as shown in Figure 5 . The training process adopts a joint training method, and the entire training process is as follows: Step 1: Load the visible light and infrared image data in the data set respectively, these data will be input into the encoder model based on the deep neural network ResNet50, set the Adam optimizer and the learning rate scheduler, and go to step 2.

[0082] Step 2: Apply multiple data augmentation methods to the input images to generate diverse training samples, including resizing, random cropping, horizontal flipping, channel random erasing, and channel swapping augmentation for RGB images, and similar augmentation strategies plus channel adaptive grayscale augmentation for IR images to simulate real-world variations and improve the model's generalization ability, leading to Step 3.

[0083] Step 3: At the beginning of each epoch, use the pre-trained model to extract feature vectors for RGB and IR images, combine with the true camera label information, and implement camera collaborative neighborhood screening through the CNP module, which can effectively distinguish between intra-camera neighbors and cross-camera neighbors, purifying noisy neighborhood relationships. On this basis, calculate the cross-camera Jaccard rearrangement distance matrix, use the DBSCAN clustering algorithm to cluster the unlabeled RGB and IR images, and assign pseudo-labels to the clustered images, leading to Step 4.

[0084] Step 4: Create two memory banks (visible light modality, infrared modality) for feature storage and contrastive learning, calculate the feature center points of RGB and IR clusters, and the RCR module executes a bidirectional iterative consistency alignment algorithm to calculate the bidirectional mapping relationships (i2r, r2i) of IR→RGB and RGB→IR, ensuring that each RGB cluster has a corresponding IR cluster match, implementing a unified memory mechanism for cross-modal feature representation, leading to Step 5.

[0085] Step 5: Create RGB and IR training datasets based on pseudo-labels, input cross-modal mapping relationships (i2r, r2i) into the CARR framework, calculate bidirectional cross-modal contrastive loss (visible light to infrared and infrared to visible light mutual mapping), dynamically maintain cross-modal memory bank features, update model parameters using backpropagation, and proceed to Step 6.

[0086] Step 6: Update the learning rate after each epoch, regularly evaluate on the SYSU-MM01 test set, calculate Rank-n accuracy and mAP indicators, save the best performance model checkpoint, and if the specified number of training rounds is reached, end the training process and perform final evaluation, then proceed to Step 7, otherwise continue training and return to Step 3.

[0087] Step 7: Load the best model weights saved during training, perform 10 random trials in full search mode (using all cameras) and indoor search mode (using only indoor cameras) on the test set, use Rank-n accuracy, average precision (mAP), and other indicators to measure model effectiveness, output total training time and performance report, and end.

[0088] The images in step 1 are all from the visible light infrared pedestrian re-identification standard dataset SYSU-MM01 dataset, which contains 491 pedestrian identities multi-modal images composed of visible light images and infrared images, and contains 6 cameras, among which cam1, cam2, cam4, cam5 are visible light cameras, and cam3, cam6 are infrared cameras, which provides rich camera view changes and modal differences for cross-modal pedestrian re-identification research.

[0089] In step 2, the RGB image is enhanced using horizontal flip, random cropping, color jitter, channel random erasing and channel exchange, and the IR image is enhanced using horizontal flip, random cropping, channel random erasing and channel adaptive grayscale enhancement, etc. Specific to the modal enhancement strategy, the generalization ability and robustness of the model are improved by simulating different light conditions, view changes and noise interference.

[0090] In step 3, the camera coordination core principle of the CNP module is to first convert the cosine distance to Euclidean distance to construct the initial distance matrix, and then use the real camera label to construct the camera mask to distinguish the same camera and cross-camera sample pairs. This module is based on the k-reciprocal neighbor algorithm, which adds a penalty value to the cross-camera samples to preferentially select the same camera nearest neighbor, and adds a penalty value to the same camera samples to preferentially select the cross-camera nearest neighbor. Further, through the local query expansion mechanism, the neighbor relationship is purified, and finally a camera-aware CA-Jaccard distance matrix is generated for subsequent clustering analysis, effectively solving the problem that the traditional Jaccard distance cannot handle camera bias. The specific implementation is as follows: First, calculate the single-modal distance matrix, given the L2 normalized feature matrix And , where And represent the number of visible light and infrared samples, is the feature dimension. The cosine distance matrix between samples is calculated as follows:

[0091] The inner product of the normalized vector is equivalent to the cosine similarity, and the corresponding distance metric is obtained by linear transformation of the similarity.

[0092] To distinguish intra-camera and cross-camera sample pairs, a camera mask matrix needs to be constructed. Based on the camera ID vector, a binary mask matrix is constructed. Let And represent the camera ID vectors of visible light and infrared samples, respectively, where And are the number of cameras in each modality. The mask matrix is defined as:

[0093] Based on the previously computed single-modality distance matrix and camera mask matrix, a sample ranking is achieved by a penalty mechanism. The ranking function returns the sample indices sorted in ascending order of distance. For each modality , the following is computed:

[0094] where the penalty coefficient controls the ranking behavior, by penalizing the preference of intra-camera neighbors over inter-camera samples, and by penalizing the preference of inter-camera neighbors over intra-camera samples.

[0095] Based on the above definition, camera-aware neighborhood aggregation is performed to compute the k-reciprocal neighbors of each sample under intra-camera and inter-camera conditions, respectively. For each modality :

[0096] where and denote the neighborhood size under intra-camera and inter-camera scenarios, respectively, which are treated consistently in both visible and infrared modalities.

[0097] Further based on the camera-aware k-reciprocal neighbors, a similarity weight matrix is constructed by applying SoftMax normalization to the k-reciprocal neighbor set. For each modality :

[0098] This weight computation method ensures that both visible and infrared modalities can obtain their respective similarity weight matrices based on their camera-aware neighborhoods.

[0099] In addition, camera-aware neighborhood expansion further optimizes clustering quality by hierarchical neighborhood ranking, as shown in Figure 6 . For each modality and sample :

[0100]

[0101] where and are modality-specific expansion parameters.

[0102] Secondly, the expanded similarity matrix is converted to a camera-aware Jaccard distance. For each modality :

[0103] The distance metric is independently calculated on the visible light and infrared modalities, respectively, effectively capturing the neighborhood structure features perceived by each camera.

[0104] Finally, the camera-perceived Jaccard distance matrix is applied to the clustering generation of each modality using a density-based clustering algorithm:

[0105] These camera-perceived clustering results will serve as the basis for establishing cross-modal associations in the RCR module, ensuring that the quality of single-modality clustering directly improves cross-modal alignment performance.

[0106] In step 4, first, the feature center points of RGB and IR clusters are calculated based on the clustering results and L2 normalization is performed, creating two independent memory banks for feature storage and contrastive learning in the visible light and infrared modalities. This step uses a bidirectional iterative consensus alignment algorithm to perform initial matching assignment by calculating the similarity matrix between IR features and RGB features, and then performs an iterative optimization process. First, calculate the RGB centroid based on the current assignment relationship, find the IR sample assigned to each RGB cluster, and calculate the IR centroid. The RGB centroid is updated using a weighted fusion strategy. Then, perform RGB→IR reverse optimization, recalculate the similarity with the updated RGB centroid and IR features, and perform reverse assignment. Similarly, update the IR centroid using a weighted fusion strategy. Finally, recalculate the similarity matrix and assignment relationship using the updated features, and stop iteration when the assignment result converges. The algorithm constructs bidirectional mapping relationships for IR→RGB and RGB→IR, and forces each RGB cluster to have a corresponding IR cluster match, achieving a unified memory mechanism for cross-modal feature representation, effectively solving the problem of inconsistent cluster numbers and feature distribution differences between different modalities.

[0107] The RCR module establishes cross-modal associations between camera-perceived clusters through a bidirectional iterative consensus alignment mechanism. Given the set of cluster center points from CANP and This module aims to learn the optimal bidirectional mapping relationship to achieve precise alignment of cross-modal semantically similar clusters.

[0108] The algorithm first performs L2 normalization on all cluster centers to ensure the scale invariance of feature calculation. This normalization operation effectively eliminates the magnitude difference between different modalities, so that the cross-modal comparison is not affected by the inherent feature scale of each modality:

[0109] This preprocessing step is crucial for maintaining numerical stability, which ensures that the subsequent similarity computation reflects the true semantic association rather than the magnitude difference of features.

[0110] Based on the normalized features, the initial cross-modal similarity matrix is first computed, and the preliminary correspondence is established according to the maximum cosine similarity. This initialization process provides a stable starting benchmark for subsequent iterative optimization:

[0111] The resulting matching vector An initial one-to-many mapping relationship from infrared clusters to visible light clusters is constructed, laying the foundation for subsequent bidirectional optimization.

[0112] In the forward optimization phase, the feature information of matched infrared clusters is fused to optimize the representation of visible light clusters. For each visible light cluster, all matched infrared clusters at the current stage are first identified, and then updated through weighted fusion, where the fusion parameter β is used to balance the integration of cross-modal information and the preservation of original cluster features:

[0113]

[0114] To ensure bidirectional consistency, reverse matching calculation is needed. The algorithm calculates the reverse matching pointing to infrared clusters based on the updated visible light clusters. This reverse matching not only verifies the effectiveness of the forward matching, but also provides guidance for subsequent optimization of infrared clusters, thereby maintaining the symmetry of the optimization process:

[0115] The resulting reverse matching vector The complementary mapping mechanism ensures that each visible light cluster has a corresponding infrared cluster, thereby constructing a symmetric information exchange channel for the subsequent reverse optimization phase.

[0116] After completing the reverse matching, the reverse optimization phase is performed. The algorithm optimizes the infrared clusters by fusing the features of visible light clusters, which balances the cross-modal feature fusion and avoids falling into a local optimal solution:

[0117]

[0118] This symmetric optimization mechanism forms a mirror relationship with the forward phase, ensuring that both modalities can benefit equally from cross-modal information exchange. By using the fusion weight β uniformly, the algorithm maintains the consistency of bidirectional optimization, effectively preventing preference bias towards a single modality.

[0119] After the bi-directional optimization is completed, the matching update and convergence judgment are re-matched, and the algorithm re-calculates the cross-modal matching relationship based on the updated clustering features. This step determines whether the iterative optimization has reached a convergent state by monitoring the matching stability:

[0120] After each feature update, re-normalization is performed to maintain the unit norm constraint. When is satisfied, the algorithm is terminated, and the clustering correspondence relationship has reached a stable consensus state.

[0121] After the algorithm converges times, two complementary mappings are constructed. The infrared-to-visible light mapping records the final matching decision, while the visible-to-infrared mapping ensures full coverage, ensuring that all clusters participate in cross-modal correspondence:

[0122]

[0123] These bi-directional mappings achieve robust cross-modal contrast learning by establishing reliable positive sample pairs across modalities. The forced matching strategy ensures complete coverage of the entire clustering space.

[0124] In step 5, the CARR framework performs cross-modal contrast learning training. This framework utilizes the bi-directional mapping relationships obtained in step 4 to achieve cross-modal label alignment conversion. The core mechanism of the CARR framework includes three levels of loss calculation: first, intra-modal contrast loss, which learns the same-modal identity feature aggregation in the RGB and IR memory banks. Second, cross-modal contrast loss, which uses the mapping relationship to perform cross-modal contrast learning between IR features and RGB memory banks or RGB features and IR memory banks. Third, an alternating training strategy is adopted, using IR features to contrast with RGB memory banks in odd epochs and using RGB features to contrast with IR memory banks in even epochs, avoiding training imbalance. The framework combines intra-modal loss and weighted cross-modal loss to form the total loss function, updates network parameters through backpropagation, and dynamically maintains memory bank features, achieving stable and efficient cross-modal identity matching learning.

[0125] The CARR framework stores and updates discriminative clustering representations by maintaining two independent clustering centroid memory banks for each modality. Define as L2 normalized features, then the memory banks for visible light and infrared modalities are represented as and .

[0126] In the initial stage, the clustering centroids are calculated from the mean of the normalized features within the camera perception clusters generated by the CNP module:

[0127] The momentum update mechanism is adopted in the training process to realize dynamic update of the memory bank. The clustering center recalculated in the training period is:

[0128] The momentum update formula of the memory bank is:

[0129] The momentum coefficient is The dual-memory bank architecture realizes independent learning of single-modal features while maintaining the cross-modal compatibility of the RCR module.

[0130] Finally, the loss function calculation of the contrastive learning is realized, and the CARR framework adopts a dual-dimension contrastive learning strategy that integrates single-modal and cross-modal learning. In single-modal learning, each modality optimizes the clustering-level contrastive loss to enhance the discriminability of the same-domain features. To realize batch balancing and feature diversity enhancement, the channel enhancement technology is used for the visible light modality to generate a balanced batch size Processing generates a balanced batch size After that, the single-modal contrastive loss function is defined as:

[0131] Wherein represents the enhanced visible light sample, is a temperature coefficient. The robust clustering representation constructed by the single-modal learning mechanism lays a foundation for subsequent cross-modal alignment through the RCR module.

[0132] In the cross-modal alignment stage, the RCR module establishes the reciprocal mapping functions (infrared to visible light) and (visible light to infrared) through bidirectional iterative consensus with completeness constraints. It should be particularly noted that the cross-modal loss function is only used in the second stage of pseudo-label alignment. The cross-modal contrastive loss is defined as:

[0133] To prevent gradient conflicts and ensure training stability, the alternating optimization strategy is adopted in the framework:

[0134]

[0135] All total loss functions of the complete optimization target are:

[0136] The test flow of the present application Step a: Prepare and input the query set and gallery set, where the query set consists of pedestrian images captured by infrared cameras (cam3, cam6), and the gallery set consists of pedestrian images captured by visible light cameras. According to different test scenarios, the gallery set contains images from cam1, cam2, cam4, and cam5 in all scene modes, and only contains images from cam1 and cam2 in indoor scene mode. After completing data preprocessing, proceed to the next step.

[0137] Step b: Use the trained cross-modal pedestrian re-identification model to extract features from all pedestrian images in the query set and gallery set, and obtain high-dimensional feature vector representations through the backbone network. To improve feature robustness, use horizontal flip augmentation strategy to extract features from original images and flipped images, then take the average and perform L2 normalization. After feature extraction, proceed to the similarity calculation stage.

[0138] Step c: Cross-modal similarity measurement uses dot product distance calculation method, and constructs a similarity matrix by calculating the inner product of query feature vectors and gallery feature vectors. Each element in the similarity matrix represents the matching degree of the corresponding query sample and gallery sample, and the larger the value, the higher the similarity. After completing the similarity calculation between all query images and gallery images, proceed to the result evaluation stage.

[0139] Step d: According to the similarity matrix, arrange the matching results of each query sample in descending order, and use Rank-n accuracy and mean average precision (mAP) as evaluation indicators. Rank-n represents the proportion of correct matches in the top n results, and mAP reflects the comprehensive performance of the retrieval system at different recall rates. During evaluation, follow the standard protocol and exclude the matching between query images and gallery images from the same camera to ensure fair comparison.

[0140] Step e: Output the complete performance evaluation results, including Rank-1 / 10 / 20 accuracy and mAP values, and evaluate them in all scene modes and indoor scene modes to comprehensively test the performance of the model in different environments. To ensure the statistical significance and reliability of the results, perform 10 repeated tests and calculate the average value, and finally generate a detailed performance analysis report.

[0141] In step b, the feature extraction process uses a horizontal flip data augmentation strategy to extract features from original images and horizontally flipped images, averages the two sets of feature vectors, and performs L2 normalization on the fused features to enhance the robustness and discriminability of the feature representation.

[0142] In step d, the evaluation process strictly follows the standard evaluation protocol for cross-modal pedestrian re-identification, excluding matching pairs between the query image and gallery images from the same camera when calculating the matching results, avoiding unfair comparisons due to camera bias, ensuring the objectivity and reliability of the evaluation results.

[0143] In step e, the test evaluation is independently conducted under two different test conditions: all scene mode and indoor scene mode. By comparing the performance in different environments, the adaptability and stability of the model in complex outdoor and controlled indoor environments are verified, providing reliable performance reference for practical applications.

[0144] The entire test process uses a statistical verification method of multiple repeated experiments. The evaluation results under each test condition are tested 10 times independently, and the mean and standard deviation of each evaluation index are calculated to ensure the statistical significance and repeatability of the test results.

[0145] Example 1: This example is based on the widely used SYSU-MM01 cross-modal pedestrian re-identification benchmark public dataset. This dataset is specifically designed to address the technical challenges of cross-modal matching of “visible light-infrared” images in real monitoring environments, with high practicality and representativeness. The dataset uses a six-camera deployment architecture, including four visible light cameras (cam1, cam2, cam4, cam5) and two infrared cameras (cam3, cam6). The training set covers 395 different identities of pedestrian samples, including 22,258 visible light images and 11,909 infrared images, forming a rich cross-modal training sample library.

[0146] To comprehensively evaluate the performance of the model in different application scenarios, the dataset provides two standard test configurations: all-search mode uses all visible light cameras as the gallery set, simulating large-scale retrieval tasks in open environments. Indoor search mode (indoor-search) only uses indoor visible light cameras (cam1, cam2) to build the gallery set, specifically evaluating the recognition ability in controlled environments. This multi-mode evaluation strategy ensures the comprehensiveness and reliability of model performance evaluation.

[0147] The application adopts cumulative matching characteristic curve (CMC) and average precision mean (mAP) as core performance measurement standards. The CMC curve evaluates the overall performance of the retrieval system by calculating the correct matching probability of different ranking positions, and selects three key nodes of Rank-1, Rank-10 and Rank-20 for quantitative analysis, wherein Rank-1 reflects the first hit rate, and Rank-10 and Rank-20 reflect the recall ability of the system. The mAP index measures the comprehensive retrieval accuracy of the system by averaging the average precision (AP) of all query samples, and can comprehensively reflect the performance stability of the model under different recall rates.

[0148] The present research is implemented based on the PyTorch framework, and is completed on an NVIDIA RTX4090 (24GB memory) graphics card. ResNet50 is used as a feature extractor and ImageNet pre-trained weights are loaded, and the AGW benchmark configuration is strictly followed. In the test stage, the cosine similarity is calculated by extracting features through the GeM pooling layer, the batch size is set to 64 during training, and the input image is uniformly adjusted to a resolution of 288x144. We use data enhancement techniques such as horizontal flipping, random cropping and random erasing, and apply channel enhancement to the visible light modality to alleviate cross-modal differences.

[0149] In terms of training strategy, 4 pedestrian identities are sampled from each modality training set per batch, and DBSCAN algorithm is used to generate pseudo labels. The Adam optimizer is used for model optimization, and the training process includes two stages of 50 rounds each (totaling 100 rounds), wherein the CNP module is activated throughout the process, and the RCR module is only enabled in the second stage. The initial learning rate is set to and a step decay strategy is used, reducing to 1 / 10 of the previous value every 20 training rounds. The momentum parameter of the memory bank is , the camera internal and external neighborhood parameters are set to , and , , and the centroid fusion parameter is .

[0150] In the SYSU-MM01 dataset, the recognition method of the application is evaluated. The performance comparison results of each method on the SYSU-MM01 dataset are shown in Table 1.

[0151] Table 1 Comparison results of the method of the application and other methods on the SYSU-MM01 dataset

[0152] From Table 1, it can be observed that in the previous method, the present application achieves the optimal performance on the SYSU-MM01 dataset. In the All-Search mode, the Rank-1 accuracy reaches 55.37%, and the mAP reaches 52.63%. In the Indoor-Search mode, the Rank-1 accuracy is improved to 62.70%, and the mAP is as high as 68.73%, fully verifying the stability and effectiveness of the framework in different test scenarios. Compared with the advanced unsupervised "visible light-infrared" pedestrian re-identification learning methods SCA and CCLNet in the table, the present application improves the accuracy of Rank-n and mAP indicators on SYSU-MM01.

[0153] In summary, the camera-aware reciprocal optimization unsupervised "visible light-infrared" pedestrian re-identification method proposed in the present application better learns the features of visible light and infrared images when processing pedestrian re-identification tasks. These performance improvements fully demonstrate the synergistic effect of the camera-aware neighborhood purification and reciprocal centroid optimization mechanism, effectively solving the technical bottlenecks of traditional methods in handling camera domain shift and cross-modal correspondence.

[0154] Embodiment 2: This embodiment will introduce a typical application scenario of the present application in the field of child safety protection.

[0155] A large theme park had a child lost incident during the weekend peak period. The parents found that a 5-year-old child had lost in the crowd, and the park security department only obtained a child photo provided by the parents. Due to the large area of the park, the large number of tourists, and the distribution of hundreds of different types of monitoring equipment (including outdoor visible light cameras and night infrared cameras), the traditional manual search method faces great challenges.

[0156] The traditional method has three limitations. First, it takes several hours or even longer for security personnel to view the monitoring screen one by one, and the safety of the lost child cannot be delayed, so the time window is extremely valuable. Second, the park monitoring equipment is complex, including cameras of different brands and under different lighting conditions, and the traditional recognition system significantly reduces the accuracy when processing the modal differences between visible light and infrared images. Finally, children are low in height, and the imaging difference under different monitoring angles is large, further increasing the identification difficulty.

[0157] The technical solution proposed by the present application has the following specific process for finding a lost child: Step one: pedestrian detection automatically detects and extracts all pedestrian images from the real-time video stream of each monitoring node in the park, establishing a dynamic pedestrian image library to lay the data foundation for subsequent matching.

[0158] Step two: Camera-aware neighborhood optimization intelligently identifies samples from the same camera and different cameras through the CNP module, constructs a camera mask matrix to distinguish intra-camera and inter-camera sample pairs, adopts a double-path neighbor selection strategy to process reliable intra-camera neighbors and complementary inter-camera neighbors respectively, and filters out high-confidence positive sample pairs through a camera-aware penalty mechanism and a reciprocal nearest neighbor algorithm.

[0159] Step three: Based on the camera-aware Jaccard distance matrix constructed by the CNP module, the DBSCAN clustering algorithm is used to automatically group the visible light and infrared images respectively, and a unique pseudo-label is assigned to each cluster. Images of the same identity under different modalities and different cameras are automatically identified and grouped into corresponding clusters, and a high-quality multi-modal training sample set is constructed.

[0160] Step four: Cross-modal feature alignment constructs a camera-aware similarity matrix, and the RCR module is used to establish an accurate correspondence between visible light and infrared images, and a bidirectional iterative optimization mechanism is used to enhance the cross-modal recognition capability of the system.

[0161] Step five: The photo of the lost child provided by the parent is taken as the query image, and deep feature matching is performed with all pedestrian images in the system database. The system quickly identifies the child's appearance records in different areas of the park and at different time periods, including images captured by infrared cameras near the playground at night.

[0162] Step six: Based on the identification results, the moving track of the lost child is reconstructed, and the system finds that the child last appeared in the children's playground area on the east side of the park. The security personnel immediately go to the area, and under the continuous tracking and guidance of the system, they successfully find the lost child playing near the carousel in only 15 minutes, and safely return him to the parent's side.

[0163] This embodiment demonstrates the practical application value of the present application in child safety protection. Through unsupervised learning mechanism and cross-modal recognition technology, the system realizes the full automation process from data collection, model training to target positioning, shortens the traditional manual search time from several hours to tens of minutes, significantly improves the timeliness and success rate of lost child rescue, and provides important technical support for smart park construction and public safety protection.

[0164] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application, and is not intended to limit the scope of the present application. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present application shall fall within the scope of the present application.

Claims

1. A pedestrian re-identification method of camera-aware reciprocity optimization, characterized in that, Comprising the following steps: Step 1: Load the visible light and infrared image data in the dataset respectively, which will be input into the encoder model based on the deep neural network ResNet50, set the Adam optimizer and learning rate scheduler, and go to step 2; Step 2: Apply multiple data augmentation methods to the input images to generate diverse training samples, including size adjustment, random cropping, horizontal flipping, channel random erasing, and channel exchange enhancement for RGB images, and similar enhancement strategies plus channel adaptive grayscale enhancement for IR images to simulate changes in real-world scenarios and improve the model's generalization ability, go to step 3; Step 3: At the beginning of each epoch, use the pre-trained model to extract feature vectors of RGB and IR images, combine with real camera label information, and realize camera collaborative neighborhood screening through the CNP module, which can effectively distinguish intra-camera neighbors and cross-camera neighbors, and purify noisy neighborhood relationships, on this basis, calculate the cross-camera Jaccard rearrangement distance matrix, use the DBSCAN clustering algorithm to cluster the unlabeled RGB and IR images, and assign pseudo-labels to the clustered images, go to step 4; Step 4: Create two memory banks including visible light and infrared modalities for feature storage and contrastive learning, calculate the feature center points of RGB and IR clusters, and the RCR module executes the bidirectional iterative consistency alignment algorithm to calculate the bidirectional mapping relationship (i2r, r2i) of IR→RGB and RGB→IR, ensuring that each RGB cluster has a corresponding IR cluster match, realizing a unified memory mechanism for cross-modal feature representation, go to step 5; Step 5: Create RGB and IR training datasets based on pseudo-labels, input cross-modal mapping relationships (i2r, r2i) into the CARR framework, calculate bidirectional cross-modal contrastive loss, dynamically maintain cross-modal memory bank features, and update model parameters using backpropagation, go to step 6; Step 6: Update the learning rate after each epoch, regularly evaluate on the SYSU-MM01 test set, calculate Rank-n accuracy and mAP indicators, save the best performance model checkpoint, if the specified number of training rounds is reached, end the training process, and perform final evaluation, then go to step 7, otherwise continue training and return to step 3; Step 7: Load the best model weight saved during training, perform 10 random trials in full search mode and indoor search mode on the test set, use Rank-n accuracy and average precision mAP indicators to measure model effectiveness, output total training time and performance report, and end.

2. The method of claim 1, wherein, The images in step 1 are all from the visible-infrared pedestrian re-identification standard dataset SYSU-MM01 dataset, which contains 491 multi-modal images of pedestrian identities, composed of visible light images and infrared images, and contains 6 cameras, of which cam1, cam2, cam4, cam5 are visible light cameras, and cam3, cam6 are infrared cameras, which provides rich camera view changes and modal differences for cross-modal pedestrian re-identification research.

3. The method of claim 2, wherein, In step 2, the RGB images are enhanced using horizontal flipping, random cropping, color jittering, channel random erasing and channel swapping, and the IR images are enhanced using horizontal flipping, random cropping, channel random erasing and channel adaptive grayscale enhancement, and other modal-specific enhancement strategies, to simulate different lighting conditions, view changes and noise interference to improve the generalization ability and robustness of the model.

4. The method of claim 3, wherein, In step 3, the camera coordination core principle of the CNP module is to first convert the cosine distance to Euclidean distance to construct an initial distance matrix, and then use the real camera label to construct a camera mask to distinguish same-camera and cross-camera sample pairs; this module uses the k-reciprocal neighbor algorithm, adds a penalty value to the cross-camera samples to preferentially select the same-camera nearest neighbor, and adds a penalty value to the same-camera samples to preferentially select the cross-camera nearest neighbor, and uses a local query expansion mechanism to purify the neighbor relationship, and finally generates a camera-aware CA-Jaccard distance matrix for subsequent clustering analysis, the specific implementation is as follows: First, the single-modality distance matrix is calculated, given the L2-normalized feature matrix and where and denote the number of visible and infrared samples, respectively, is the feature dimension; the cosine distance matrix between samples is calculated as follows: The inner product of the normalized vector is equivalent to the cosine similarity, and the corresponding distance metric is obtained by linear transformation of the similarity; To distinguish between intra-camera and cross-camera sample pairs, a camera mask matrix needs to be established; Construct a binary mask matrix based on the camera ID vectors; let and denote the camera ID vectors for the visible and infrared samples, respectively, where and are the number of cameras for each modality; the mask matrix is defined as: Based on the previously computed single modality distance matrix and the camera mask matrix, a sample ranking is achieved by a penalty mechanism; the ranking function Returning the sample indices sorted in ascending order by distance; for each modality , is computed as follows: wherein the penalty coefficient controlling the ordering behavior, by penalizing the selection of intra-camera neighbors in favor of cross-camera neighbors, and by penalizing the selection of cross-camera neighbors in favor of intra-camera neighbors. Based on the above definitions, camera-aware neighborhood aggregation is performed, computing the k-reciprocal nearest neighbors of each sample, respectively, within-camera and cross-camera conditions ; for each modality : wherein with denote the neighborhood scale for intra- and cross-camera situations, respectively, which the process treats consistently in both visible and infrared modalities; Further based on the camera-aware k-reciprocal nearest neighbors, a similarity weight matrix is constructed by applying SoftMax normalization to the set of k-reciprocal nearest neighbors; for each modality : This weight calculation method ensures that both visible light and infrared modalities can obtain the corresponding similarity weight matrix based on their respective camera-aware neighborhoods; Furthermore the camera-aware neighborhood extension further optimizes the clustering quality by a hierarchical neighborhood ordering for each modality and samples : wherein with is a modal specific extension parameter; Secondly, the extended similarity matrix is converted into the camera- aware Jaccard distance; for each modality : This distance metric is independently calculated on the visible light and infrared modalities, effectively capturing the respective camera-aware neighborhood structure features; Finally, the density-based clustering algorithm is used to apply the camera-aware Jaccard distance matrix to the clustering of each modality to generate: These camera-aware clustering results will serve as the basis for establishing cross-modal associations in the RCR module, ensuring that the quality of single-modal clustering can directly improve the performance of cross-modal alignment.

5. The method of claim 4, wherein, In step 4, the feature centers of RGB and IR clusters are first calculated based on the clustering results and L2 normalization is performed, creating two independent memory banks for feature storage and contrastive learning in visible light and infrared modalities respectively. The algorithm uses a bidirectional iterative consistency alignment algorithm to perform initial matching assignment by calculating the similarity matrix between IR and RGB features, and then performs an iterative optimization process. First, the RGB centroids are calculated based on the current assignment relationship, and for each RGB cluster, the IR samples assigned to it are found and the IR centroids are calculated. The RGB centroids are updated using a weighted fusion strategy. Then, the RGB→IR reverse optimization is performed, and the similarity between the updated RGB centroids and the IR features is recalculated and assigned in reverse. The IR centroids are also updated using a weighted fusion strategy. Finally, the similarity matrix and assignment relationship are recalculated using the updated features, and the iteration is stopped when the assignment result converges. The algorithm constructs bidirectional mapping between IR→RGB and RGB→IR, and forces to ensure that each RGB cluster has a corresponding IR cluster matching, realizing a unified memory mechanism for cross-modal feature representation. The RCR module establishes cross-modal association between camera perception clusters through a bidirectional iterative consensus alignment mechanism; given a set of cluster center points from the CANP With This module aims to learn the optimal bidirectional mapping relationship to achieve precise alignment of cross-modal semantic similar clusters; The algorithm first performs L2 normalization on all cluster centers to ensure the scale invariance of feature calculation. This normalization operation can effectively eliminate the magnitude difference between different modal features, so that the cross-modal comparison is not affected by the inherent feature scale of each modality: This preprocessing step is crucial for maintaining numerical stability, as it ensures that the subsequent similarity calculation reflects the true semantic association rather than the feature magnitude difference; Based on the normalized features, the initial cross-modal similarity matrix is first calculated, and the preliminary correspondence is established based on the maximum cosine similarity. This initialization process provides a stable starting benchmark for subsequent iterative optimization: The resulting matching vector An initial one-to-many mapping relationship from infrared clusters to visible light clusters is constructed, laying a foundation for subsequent bidirectional optimization. The forward optimization phase optimizes the visible light cluster representation by fusing the feature information of the matched infrared clusters. For each visible light cluster, the matching infrared clusters at the current stage are first identified, and then updated through weighted fusion, where the fusion parameter β is used to balance the integration of cross-modal information and the preservation of original cluster features: To ensure bidirectional consistency, reverse matching calculation is needed. The algorithm calculates the reverse matching of infrared clusters based on the updated visible light clusters. This reverse matching not only verifies the effectiveness of the forward matching, but also provides guidance for the subsequent optimization of infrared clusters, thereby maintaining the symmetry of the optimization process: Reverse matching vector The established complementary mapping mechanism ensures that each visible light cluster has a corresponding infrared cluster, thereby constructing a symmetric information interaction channel for the subsequent reverse optimization phase; After completing the reverse matching, the reverse optimization phase is performed. The algorithm optimizes the infrared clusters symmetrically by fusing the visible light cluster features. This bidirectional information exchange mechanism can balance cross-modal feature fusion and avoid falling into local optimal solutions: This symmetric optimization mechanism forms a mirror relationship with the forward phase, ensuring that both modalities benefit equally from cross-modal information exchange. By using the same fusion weight β, the algorithm maintains the consistency of bidirectional optimization, effectively preventing bias towards a single modality; After completing bidirectional optimization, the matching is updated and convergence is determined. The algorithm recalculates the cross-modal matching relationship based on the updated cluster features. This step monitors the stability of the matching to determine whether the iterative optimization has reached convergence: The normalization is performed after each feature update to maintain the unit norm constraint, and the algorithm terminates when the clustering correspondence has reached a stable consensus state. The algorithm iterates After the sub-convergence, two complementary mappings are constructed, the infrared-to-visible mapping records the final matching decisions, while the visible-to-infrared mapping ensures full coverage, guaranteeing that all clusters participate in cross-modal correspondence: These two-way mappings achieve robust cross-modal contrastive learning by establishing reliable positive sample pairs across modalities, where the forced matching strategy ensures complete coverage of the entire clustering space.

6. The method of claim 5, wherein, The CARR framework in step 5 performs cross-modal contrastive learning training; the framework utilizes the two-way mapping relationship obtained in step 4 to achieve cross-modal label alignment conversion; the core mechanism of the CARR framework includes three levels of loss calculation: first, intra-modal contrastive loss, which learns the same modal identity feature aggregation in the RGB and IR memory banks; Second, cross-modal contrastive loss, which uses the mapping relationship to perform cross-modal contrastive learning between IR features and RGB memory banks or RGB features and IR memory banks; third, an alternating training strategy is used, which uses IR features to contrast with RGB memory banks at odd epochs and uses RGB features to contrast with IR memory banks at even epochs to avoid training imbalance; the framework combines the intra-modal loss and the weighted cross-modal loss to form the total loss function, which updates the network parameters through backpropagation while dynamically maintaining the memory bank features, achieving stable and efficient cross-modal identity matching learning; The CARR framework stores and updates discriminative cluster representations by maintaining a dual-modality independent cluster centroid memory bank; define For L2-normalized features, the memory banks for visible and infrared modalities are denoted as and respectively; In the initial stage, the clustering centroids are calculated from the normalized feature means within the camera perception clusters generated by the CNP module: The momentum updating mechanism is adopted in the training process to realize dynamic updating of the memory library. The recalculated clustering center in the training period is: The memory bank momentum update formula is: where the momentum coefficient The dual-memory library architecture realizes independent learning of single-modal features while maintaining cross-modal compatibility of the RCR module. Finally, the loss function calculation of contrastive learning is realized. The CARR framework adopts a dual-dimensional contrastive learning strategy that integrates single-modal and cross-modal learning. In single-modal learning, each modality enhances the discriminability of intra-domain features by optimizing the clustering-level contrastive loss. To achieve batch balancing and feature diversity enhancement, the visible light modality uses channel enhancement technology to generate balanced batch size Processing to generate balanced batch size After that, the single-modal contrastive loss function is defined as: wherein represents the enhanced visible light sample, is the temperature coefficient; the robust clustering representation constructed by the single-modal learning mechanism lays a foundation for subsequent cross-modal alignment through the RCR module; In the cross-modal alignment stage, the RCR module establishes the mutual inverse mapping functions through bidirectional iterative consensus with completeness constraints (infrared→visible light) and (visible light→infrared); it should be particularly pointed out that the cross-modal loss function is only used in the second stage of pseudo-label alignment; the cross-modal contrast loss is defined as: To prevent gradient conflicts and ensure training stability, the framework uses an alternating optimization strategy: The complete optimization target is all the total loss functions: 。 7. The method of claim 6, wherein, In step 7, the test specifically includes: Step a: Prepare and input to build a query set and a gallery set, where the query set is composed of pedestrian images captured by infrared cameras cam3 and cam6, and the gallery set is composed of pedestrian images captured by visible light cameras; according to different test scenarios, the gallery set contains images from cameras cam1, cam2, cam4, and cam5 in all scene modes, and only contains images from cameras cam1 and cam2 in indoor scene mode; after data preprocessing, proceed to the next step; Step b: Use the trained cross-modal pedestrian re-identification model to extract features from all pedestrian images in the query set and the gallery set, and obtain high-dimensional feature vector representations through the backbone network; to improve feature robustness, use horizontal flip augmentation strategy to extract features from original images and flipped images, then take the average and perform L2 normalization; after feature extraction, proceed to the similarity calculation stage; Step c: Cross-modal similarity measurement uses the dot product distance calculation method, which constructs a similarity matrix by calculating the inner product of query feature vectors and gallery feature vectors; each element in the similarity matrix represents the matching degree of the corresponding query sample and gallery sample, and the larger the value, the higher the similarity; after completing the similarity calculation between all query images and gallery images, proceed to the result evaluation stage; Step d: Rank-n accuracy and mean average precision (mAP) are used as evaluation indicators to rank the matching results of each query sample in descending order according to the similarity matrix. Rank-n represents the proportion of correct matches in the top n results. Rank-1, Rank-10, and Rank-20 are calculated. mAP reflects the comprehensive performance of the retrieval system at different recall rates. During evaluation, the standard protocol is followed, and matches between query images and images from the same camera are excluded to ensure fair comparison. Step e: The complete performance evaluation results, including Rank-1 / 10 / 20 accuracy and mAP values, are output. The evaluation is conducted in both all-scene mode and indoor scene mode to comprehensively test the performance of the model in different environments. To ensure statistical significance and reliability of the results, 10 repeated tests are performed, and the average values are calculated to generate a detailed performance analysis report.

8. The method of claim 7, wherein, In step b, the feature extraction process uses a horizontal flip data augmentation strategy. The original image and the horizontally flipped image are respectively subjected to feature extraction. The two sets of feature vectors are averaged and fused, and the fused features are subjected to L2 normalization processing to enhance the robustness and discriminability of the feature representation.

9. The method of claim 8, wherein, In step d, the evaluation process strictly follows the standard evaluation protocol for cross-modal pedestrian re-identification. When calculating the matching results, the matching pairs between the query image and the gallery image from the same camera are excluded to avoid unfair comparison due to camera bias, ensuring the objectivity and credibility of the evaluation results.

10. The method of claim 9, wherein, In step e, the test evaluation is independently conducted in two different test conditions: all-scene mode and indoor scene mode. By comparing and analyzing the performance in different environments, the adaptability and stability of the model in complex outdoor environments and controlled indoor environments are comprehensively verified, providing reliable performance reference for practical applications. The entire test process uses a statistical verification method with multiple repeated experiments. The evaluation results under each test condition are independently tested 10 times, and the mean and standard deviation of each evaluation indicator are calculated to ensure the statistical significance and repeatability of the test results.

Images