Unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and edge noise learning

Abstract

The application discloses an unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and edge noise learning, comprising the following steps: collecting visible light images and infrared images of pedestrians, and extracting visible light feature vectors and infrared feature vectors; for each visible light feature vector, searching for the optimal matching infrared feature vector in the infrared feature space, constructing a unified graph and performing feature propagation to obtain an enhanced feature vector; then, clustering is performed to divide noise nodes and non-noise nodes; boundary collaborative features are constructed between noise nodes of different modalities in the same cluster; noise nodes are screened based on a dynamic threshold, and weighted aggregation is performed to form noise collaborative features; a truncated Gaussian distribution is constructed to simulate the joining and exiting of pedestrians, all pedestrians in the same cluster are regarded as a pseudo-cluster group, and a pseudo-cluster group center is obtained; and a loss function is constructed and trained. The application focuses on explicitly mining and coordinating boundaries and noises on the basis of existing pseudo-labels to enhance robustness.

Description

Technical Field

[0001] This invention relates to the interdisciplinary application of artificial intelligence, pattern recognition, and intelligent security monitoring, specifically to an unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning. Background Technology

[0002] Person re-identification (ReID) aims to match pedestrians from different camera perspectives and is a key technology in the fields of intelligent security and video surveillance. In recent years, with the increasing demand for all-weather intelligent monitoring, visible-infrared re-identification (VI-ReID) has received widespread attention. Due to the high cost of cross-modal data annotation, unsupervised visible-infrared re-identification (USL-VI-ReID) has become a more practical research direction.

[0003] Existing research mainly generates pseudo-labels through clustering and uses contrastive learning and other methods for cross-modal feature alignment. For example, existing techniques employ progressive graph matching and alternating contrastive learning frameworks to address the problem of imbalanced cluster numbers in cross-modal clustering, or use bidirectional optimal transport label allocation and neighbor-consistent label optimization strategies.

[0004] Despite the significant progress made by the aforementioned methods, existing models generally suffer from insufficient utilization of difficult samples (such as boundary samples and noisy samples). Specifically, in the process of generating pseudo-labels based on clustering, low-confidence boundary samples and noisy samples considered outliers are often simply discarded or ignored, failing to effectively participate in model training. This approach limits the model's ability to fit complex cross-modal distributions, resulting in considerable room for improvement in the model's robustness and generalization ability in real-world scenarios with significant modal differences and uneven sample quality. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing models in utilizing difficult samples (such as noisy samples). This invention provides an unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and edge-noise learning, which explicitly mines and reconciles boundary and noisy samples to enhance model robustness. It aims to extract effective information from low-confidence samples without discarding any samples, through algorithmic design.

[0006] To achieve the above functions, this invention designs an unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning. The method executes the following steps S1-S6 to construct an unsupervised cross-modal pedestrian re-identification model, and then executes step S7 to complete the pedestrian re-identification:

[0007] Step S1: Based on cameras deployed at multiple locations, collect visible light and infrared images of each pedestrian to form visible light and infrared datasets. Use an encoder to extract visible light and infrared feature vectors respectively.

[0008] Step S2: Based on the visible light dataset and infrared dataset, as well as the visible light feature vector and infrared feature vector, for each visible light feature vector, search for similar candidate feature vectors in the infrared feature space formed by the infrared feature vectors, and further find the best matching infrared feature vector among the candidate feature vectors to construct a unified graph.

[0009] Step S3: Using the visible light feature vector and the best-matched infrared feature vector as nodes in the unified graph, perform graph feature propagation on the unified graph to obtain the enhanced feature vector; then, use HDBSCAN hierarchical density clustering to cluster the enhanced feature vector, dividing it into noisy nodes and non-noisy nodes; for non-noisy nodes, further divide them into high-confidence nodes, boundary nodes, and approximate noisy nodes.

[0010] Step S4: Construct boundary collaborative features based on distance between noisy nodes of different modalities in the same cluster;

[0011] Step S5: Filter noisy nodes based on dynamic thresholds, and aggregate multiple selected noisy nodes by weighting to form noise collaborative features;

[0012] Step S6: Construct a truncated Gaussian distribution to simulate the entry and exit of pedestrians. Based on the truncated Gaussian distribution, construct a pedestrian retention mask, treat all pedestrians in the same cluster as a pseudo-cluster, and obtain the center of the pseudo-cluster.

[0013] Step S7: Construct dual contrastive learning loss and modality-invariant contrastive learning loss, and construct boundary coordination loss and noise perception loss based on boundary collaborative features and noise collaborative features; further construct the complete loss to complete the training of the unsupervised cross-modal pedestrian re-identification model, and apply the unsupervised cross-modal pedestrian re-identification model to complete pedestrian re-identification.

[0014] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0015] 1. This invention does not directly discard low-confidence boundary samples and noise samples generated during clustering. Instead, it explicitly mines and coordinates these difficult samples by designing a boundary-aware sample collaboration mechanism and a noise-aware progressive learning strategy. Specifically, by constructing boundary collaboration features to bring cross-modal boundary samples closer together, it alleviates the offset of decision boundaries between modalities. At the same time, by using a dynamically decaying threshold function, only reliable noise samples are introduced in the early stage of model training, and more challenging noise samples are gradually included in the later stage. This effectively avoids the instability of the model in the early stage and significantly improves the model's ability to distinguish difficult samples and its overall robustness.

[0016] 2. This invention alleviates the problem of independent clustering of different modes by constructing cross-modal soft connections and a unified graph structure to pre-establish preliminary associations between modes before feature propagation. Simultaneously, it introduces a cross-modal density alignment mechanism, which effectively corrects the inconsistency in the distribution density of visible light and infrared features through local density estimation and modal weighting functions. This encourages the model to learn mode-invariant feature representations, thereby improving the accuracy and stability of cross-modal feature alignment.

[0017] 3. This invention introduces a pseudo-clustering population learning method based on a truncated Gaussian distribution in the later stages of training. This method treats the same cluster as a pseudo-population and generates diverse sub-cluster population centers as reliable positive samples by simulating the random retention and exit of population members. This not only enhances the stability and diversity of positive sample supervision signals but also improves the model's tolerance to clustering errors and missing samples, further enhancing the model's generalization ability. Attached Figure Description

[0018] Figure 1 This is a flowchart of an unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side noise learning provided according to an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0020] The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning provided in this embodiment of the invention refers to... Figure 1 Perform the following steps S1-S6 to construct an unsupervised cross-modal pedestrian re-identification model, and then perform step S7 to complete the pedestrian re-identification:

[0021] Step S1: Based on cameras deployed at multiple locations, collect visible light and infrared images of each pedestrian to form visible light and infrared datasets. Use an encoder to extract visible light and infrared feature vectors respectively.

[0022] The specific method for step S1 is as follows:

[0023] For multiple cameras at different locations with varying resolutions and modalities, capturing a large number of pedestrian images from multiple locations, an unsupervised cross-modal pedestrian re-identification model needs to identify the same person from different images. Based on the visible light dataset DV and the infrared dataset DR, an unlabeled visible light-infrared pedestrian re-identification dataset is constructed. ,in Indicates inclusion An unlabeled visible light dataset of pedestrian images, This represents the i-th pedestrian image in the unlabeled visible light dataset; Indicates inclusion An unlabeled infrared dataset of pedestrian images, This represents the i-th pedestrian image in the unlabeled infrared dataset;

[0024] For the unsupervised visible-infrared pedestrian re-identification task, the goal is to train a robust model from the visible-infrared pedestrian re-identification dataset. The samples are projected into the embedding space. The visible light feature vectors are extracted using an encoder. ,in Represents the visible light feature vector of the i-th pedestrian image in the unlabeled visible light dataset; and the infrared feature vector. ,in Represents the infrared feature vector of the i-th pedestrian image in the unlabeled infrared dataset; Indicates the dimension of the feature vector.

[0025] In this embodiment, a dual-stream feature extractor (using ResNet50 as the backbone) is employed to encode visible light and infrared images separately. The dual-stream feature extractor shares deep network parameters to reduce modal differences and achieve cross-modal feature information transfer, while retaining independent shallow input branches to adapt to the input characteristics of different modalities. Specifically, for a ResNet50 architecture, the input layer, the first convolutional layer, the first batch normalization (BN) layer, and the first ReLU layer are independent layers, designed to adapt to the input characteristics of different modalities. The second convolutional layer and all subsequent residual blocks and fully connected layers are shared layers, designed to reduce modal differences, share deep semantic information, and reduce the number of parameters. For an input image x, the features extracted by the network are represented as: .in, Let θ represent the backbone network and θ represent the learnable parameters.

[0026] Step S2: Based on the visible light dataset and infrared dataset, as well as the visible light feature vector and infrared feature vector, for each visible light feature vector, search for similar candidate feature vectors in the infrared feature space formed by the infrared feature vectors, and further find the best matching infrared feature vector among the candidate feature vectors to construct a unified graph.

[0027] The specific method for step S2 is as follows:

[0028] To avoid the visible light mode and infrared mode forming independent clusters during subsequent clustering, cross-modal soft connections are first constructed to pre-establish cross-modal identity alignment relationships. For the visible light mode and infrared mode, k-nearest neighbor graphs are constructed based on the K-nearest neighbor method. In this embodiment, the number of KNN nearest neighbors is set to 10. The k-nearest neighbor graph is represented as a set of points. , ;in, Point set representing visible light modes, The point set represents the infrared mode, and the nodes in the point set correspond to the visible light feature vector or the infrared feature vector extracted by the encoder, respectively. The set of nodes representing visible light modes. This represents the set of edges inside a visible light mode; The set of nodes representing infrared modes, Represents the set of edges inside an infrared mode;

[0029] In the edge set, the weights of the edges are calculated as follows:

[0030] ;

[0031] Where i represents the index of the current center node; j represents the index of the neighbor node of the i-th node; This represents the weight of the edge between the i-th node and the j-th node. , It is the set of neighboring nodes of the i-th node; Temperature coefficient; Let represent the feature vector corresponding to the i-th node. This represents the feature vector corresponding to the j-th node; Let represent the feature vector corresponding to the k-th node in the set of neighbor nodes of the i-th node; exp represents the exponential function with the real number e as the base.

[0032] The purpose of this step is to construct local structural relationships by utilizing the reliable similarity between nodes of the same modality, thereby maintaining the feature discrimination power within the modality and preventing the decrease in intramodal distinguishability due to information confusion during subsequent cross-modal fusion.

[0033] For each visible light feature vector, graph-level matching is used to retrieve its Top-K most similar candidate feature vectors in the infrared feature space. In this embodiment, the number of cross-modal candidate matches is set to 5, and the structural similarity weight and matching threshold are both set to 0.5. This step does not directly use nearest neighbors because there will be a large number of false matches.

[0034] The graph-level matching method is as follows:

[0035] Define a local subgraph, for the i-th node, ,in, This indicates that the K-nearest neighbor method is used for calculation. Let represent a local subgraph centered at the i-th node; the matching score is defined as follows:

[0036] ;

[0037] in, This represents the matching score between the i-th node and the j-th node. This represents a local subgraph centered at the j-th node. Indicating structural similarity, , This represents the number of nodes in the set of neighboring nodes of the i-th node. For dynamic weighting factors; Represents the set of neighboring nodes of the i-th node. Any neighboring node in the list, This represents the set of neighboring nodes of the j-th node. Any neighboring node in the list; Represents a node The corresponding feature vector, Represents a node The corresponding feature vector;

[0038] The optimal match is then sought. Ideally, a matching matrix would be constructed, but this method is computationally complex in practice. To simplify the code and minimize the impact on model performance, bidirectional matching is used for a simplified implementation. The optimal matching infrared feature vector is found among the candidate feature vectors, cross-modal edges are generated, and a unified graph is constructed, as shown in the following equation:

[0039] ;

[0040] in, Indicates a cross-modal edge. Represents a unified diagram; The set of nodes representing visible light modes. This represents the set of edges inside a visible light mode; The set of nodes representing infrared modes, This represents the set of edges inside the infrared mode.

[0041] Step S3: Using the visible light feature vector and the best-matched infrared feature vector as nodes in the unified graph, perform graph feature propagation on the unified graph to obtain the enhanced feature vector; then, use HDBSCAN hierarchical density clustering to cluster the enhanced feature vector, dividing it into noisy nodes and non-noisy nodes; for non-noisy nodes, further divide them into high-confidence nodes, boundary nodes, and approximate noisy nodes.

[0042] The specific steps of step S3 are as follows:

[0043] Step S3.1: Graph feature propagation is performed using a residual connection structure, retaining a portion of the original feature vectors. Weights are assigned to the visible light feature vectors and infrared feature vectors respectively, aiming to prevent excessive smoothing or loss of feature information during multi-layer propagation, as shown in the following formula:

[0044] ;

[0045] in, This represents the enhanced feature vector corresponding to node i after graph feature propagation; Indicates weight; Let represent the feature vector corresponding to the i-th node. This represents the feature vector corresponding to the j-th node; It is the set of neighboring nodes of the i-th node; This represents the weight of the edge between the i-th node and the j-th node; where neighbors include kNN of the same modality and cross-modal matching edges.

[0046] Constructing an enhanced feature vector set For enhancing the set of feature vectors The HDBSCAN clustering algorithm, based on hierarchical density clustering, is used. HDBSCAN can automatically discover clusters of different densities; in this example, the minimum cluster size is set to 4. The HDBSCAN algorithm outputs the cluster label of the i-th node. The confidence level of the i-th node belonging to its cluster. Cluster labels , For the number of clusters, This indicates that the i-th node is identified as a noise node. Other values ​​indicate that the i-th node is determined to be a non-noise node; It reflects the confidence that a node belongs to its cluster; the higher the value, the higher the confidence.

[0047] Step S3.2: Regarding confidence level Preset thresholds T1 and T2 are used to further classify non-noise nodes:

[0048] like If the i-th node is a high-confidence node, then the i-th node is a high-confidence node.

[0049] like If the i-th node is a boundary node, then the i-th node is a boundary node.

[0050] like If the i-th node is not initially classified as noise, its low confidence level means that introducing it in the early stages of the model could lead to negative bias. Therefore, it is initially treated as noise. As the model trains and stabilizes, the threshold T1 is dynamically adjusted to gradually introduce approximate noise nodes into the model for learning, thereby improving the model's ability to distinguish difficult samples. In this example, T1 is initially set to 0.3 and T2 to 0.6.

[0051] Step S4: Construct boundary collaborative features based on distance between noisy nodes of different modalities in the same cluster;

[0052] The specific method for step S4 is as follows:

[0053] Filter out noisy nodes The samples are used as cooperative noise nodes, where It is a threshold that changes dynamically with the number of training rounds t;

[0054] To address the inconsistency in feature distribution density between visible light and infrared light, for the feature vector corresponding to the i-th node... Introducing local density estimation, the local density of the i-th node... The calculation is as follows:

[0055] ;

[0056] in, It is the feature distance between the i-th node and the j-th node. It is the modal label (visible light mode or infrared mode) of the i-th node. It is the modal label of the j-th node; It is a modal weighting function; Indicates the total number of nodes; Indicates bandwidth or scale parameters;

[0057] The modal weighting function is as follows:

[0058] ;

[0059] Here, η is a weighting factor less than 1, used to reduce the contribution of another modal node when calculating density, thereby encouraging the model to learn modality-invariant feature structures and align density distributions across modalities;

[0060] For the feature vector corresponding to a high confidence node ,by As an anchor point, in the set B of boundary nodes within the same cluster but different modalities, find the boundary node most similar to the anchor point:

[0061] ;

[0062] in, Represents the set of boundary nodes. This represents the feature vector corresponding to the boundary node. This represents the feature vector corresponding to the boundary node most similar to the anchor point;

[0063] Based on the feature vector corresponding to the boundary node most similar to the anchor point The boundary collaborative features are constructed using linear interpolation as follows:

[0064] ;

[0065] in, This represents the boundary collaborative feature, which serves as a smoothed version of the anchor point in the feature space and is used for subsequent positive sample contrast learning to bring the cross-modal boundary nodes closer together. The coefficient used to control the interpolation ratio.

[0066] Step S5: Filter noisy nodes based on dynamic thresholds, and weight and aggregate multiple selected noisy nodes to form noise collaborative features;

[0067] The specific method for step S5 is as follows:

[0068] To utilize reliable noisy nodes, a dynamically decaying threshold is set to control their participation. This threshold dynamically changes with the training epoch t. As shown in the following formula:

[0069] ;

[0070] in, It is the initial threshold. It refers to the total number of training rounds. For the current training round, It is the attenuation coefficient;

[0071] For each noise node, calculate the similarity with the anchor point. Only nodes with a similarity higher than the current threshold are considered noise nodes. Only noisy nodes will be selected; in the example, when the total number of training rounds is 50, the attenuation coefficient is set to 1.

[0072] The selected noise nodes are weighted and aggregated to form the following noise collaborative features:

[0073] ;

[0074] in, Indicates the cooperative characteristics of noise. Let be the feature vector corresponding to the i-th node. Here, the weight corresponds to the i-th node, and the node here is the selected noise node;

[0075] The weight corresponding to the i-th node The calculation is as follows, determined by the outlier score; the lower the outlier score (the higher the confidence level), the greater the node weight:

[0076] ;

[0077] in, The sharpening factor is set to 2 in this example to enhance the weight of high-confidence nodes. Let represent the outlier score of the i-th node; Let represent the outlier score of the j-th node. This mechanism allows the model to progressively learn from easier noisy nodes and gradually expand to more difficult nodes. This ensures that when the model is unstable, it does not learn high-difficulty nodes, but learns high-difficulty noisy nodes later when it is stable to improve the model's discriminative ability.

[0078] The above method is used throughout all 100 training rounds. In the last 50 rounds of training, a new pseudo-positive sample module representing the cluster centers is introduced, thereby increasing the number of pseudo-positive samples while improving the reliability of positive samples.

[0079] Step S6: Construct a truncated Gaussian distribution to simulate the entry and exit of pedestrians. Based on the truncated Gaussian distribution, construct a pedestrian retention mask, treat all pedestrians in the same cluster as a pseudo-cluster, and obtain the center of the pseudo-cluster.

[0080] The specific method for step S6 is as follows:

[0081] In the last 50 training rounds, the center nodes of existing clusters were selected for training. Drawing inspiration from the idea of ​​randomly transforming the group topology in pedestrian re-identification to improve model recognition capabilities, a truncated Gaussian distribution was used to simulate pedestrian entry and exit. All nodes in the same cluster were treated as a pseudo-cluster, and the number of nodes was considered a truncated Gaussian distribution rather than a fixed value. The truncated Gaussian distribution is constructed based on prior knowledge of group stability and randomness. The formula for calculating the parameters of the truncated Gaussian distribution is as follows:

[0082] ;

[0083] in, This indicates the maximum permitted percentage of pedestrians to leave. It represents the probability that the group is in a preset stable state, and is a hyperparameter; This represents the inverse function of the Gaussian error function; and Let represent the mean and standard deviation of the truncated Gaussian distribution, respectively;

[0084] Based on the truncated Gaussian distribution, a formula for generating a pedestrian retention mask is constructed. For each pedestrian in the same category, it determines whether to retain them, as shown in the following formula:

[0085] ;

[0086] in, Reserve a mask for pedestrians. It follows the Bernoulli distribution. This represents the probability of a pedestrian leaving; within the same pseudo-cluster, all nodes share the same probability of a pedestrian leaving. express Follow the mean Standard deviation is Random variables sampled from a normal distribution;

[0087] Based on the pedestrian-preserving mask, a pseudo-cluster group is obtained:

[0088] ;

[0089] in, This represents the k-th pseudo-cluster group; This represents the enhanced feature vector corresponding to node i after graph feature propagation; Represents the cluster label of the i-th node; The number of clusters; pseudo-clustering groups are designed to simulate clustering errors and missing nodes, improving the model's robustness to incomplete structures.

[0090] Calculate the center of each pseudo-cluster:

[0091] ;

[0092] in, This represents the center of the k-th pseudo-cluster group; This represents the confidence level that the i-th node belongs to its cluster;

[0093] Finally, the constructed pseudo-clustering group center As a positive sample representation, it is introduced into the training process and together with the original nodes, it forms a positive sample set, thereby enhancing the stability and diversity of the positive sample supervision signal.

[0094] Step S7: Construct dual contrastive learning loss and modality-invariant contrastive learning loss, and construct boundary coordination loss and noise perception loss based on boundary collaborative features and noise collaborative features; further construct the complete loss to complete the training of the unsupervised cross-modal pedestrian re-identification model, and apply the unsupervised cross-modal pedestrian re-identification model to complete pedestrian re-identification.

[0095] The specific method for step S7 is as follows:

[0096] Constructing a dual-contrast learning loss Based on ClusterNCE loss, contrastive learning is performed on the visible light mode and the infrared mode respectively, aiming to learn a compact intra-mode representation, as follows:

[0097] ;

[0098] in, This represents the visible light query feature vector of the i-th node after cluster center alignment or normalization. This represents the infrared query feature vector of the i-th node; Indicates a positive sample. This represents the k-th feature vector in the memory library; Indicates the temperature coefficient; It is the size of the memory library for the visible light modes; This refers to the memory library size for the infrared mode. It is the k-th feature vector in the visible light modal memory library; It is the kth feature vector in the infrared modality memory library; These are positive samples in the infrared modality memory library; Indicates the total number of nodes;

[0099] Construct modality-invariant contrastive learning loss As shown in the following formula:

[0100] ;

[0101] in, Let represent the arbitrary modal query feature vector of the i-th node. This represents a positive sample in a cross-modal shared memory library. This represents the k-th eigenvector in the cross-modal shared memory library. Indicates the size of the cross-modal shared memory library;

[0102] Constructing boundary coordination loss The aim is to construct boundary collaborative features and then integrate them into the original loss function, thereby bringing the cross-modal boundary nodes closer together and mitigating the decision boundary offset between modalities, as shown in the following formula:

[0103] ;

[0104] in, Represents the set of difficult samples; This represents the feature vector corresponding to the high-confidence node; Represents boundary collaborative features;

[0105] Constructing noise-sensing loss The aim is to construct noisy collaborative features and then incorporate them into the original loss function, thereby gradually utilizing reliable noise to improve robustness, as shown in the following formula:

[0106] ;

[0107] in, Indicates the cooperative characteristics of noise;

[0108] Construct the complete loss As shown in the following formula:

[0109] ;

[0110] in, , , They are respectively , , In the example, the weights are adjusted by setting... It is 0.5. It is 0.3. It is 0.2.

[0111] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. An unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning, characterized in that, Perform the following steps S1-S6 to construct an unsupervised cross-modal pedestrian re-identification model, and then perform step S7 to complete the pedestrian re-identification: Step S1: Based on cameras deployed at multiple locations, collect visible light and infrared images of each pedestrian to form visible light and infrared datasets. Use an encoder to extract visible light and infrared feature vectors respectively. Step S2: Based on the visible light dataset and infrared dataset, as well as the visible light feature vector and infrared feature vector, for each visible light feature vector, search for similar candidate feature vectors in the infrared feature space formed by the infrared feature vectors, and further find the best matching infrared feature vector among the candidate feature vectors to construct a unified graph. Step S3: Using the visible light feature vector and the best-matched infrared feature vector as nodes in the unified graph, perform graph feature propagation on the unified graph to obtain the enhanced feature vector; then, use HDBSCAN hierarchical density clustering to cluster the enhanced feature vector, dividing it into noisy nodes and non-noisy nodes; for non-noisy nodes, further divide them into high-confidence nodes, boundary nodes, and approximate noisy nodes. Step S4: Construct boundary collaborative features based on distance between noisy nodes of different modalities in the same cluster; Step S5: Filter noisy nodes based on dynamic thresholds, and weight and aggregate multiple selected noisy nodes to form noise collaborative features; Step S6: Construct a truncated Gaussian distribution to simulate the entry and exit of pedestrians. Based on the truncated Gaussian distribution, construct a pedestrian retention mask, treat all pedestrians in the same cluster as a pseudo-cluster, and obtain the center of the pseudo-cluster. Step S7: Construct dual contrastive learning loss and modality-invariant contrastive learning loss, and construct boundary coordination loss and noise perception loss based on boundary collaborative features and noise collaborative features; further construct the complete loss to complete the training of the unsupervised cross-modal pedestrian re-identification model, and apply the unsupervised cross-modal pedestrian re-identification model to complete pedestrian re-identification.

2. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 1, characterized in that, The specific method for step S1 is as follows: Based on the visible light dataset DV and the infrared dataset DR, an unlabeled visible light-infrared pedestrian re-identification dataset was constructed. ,in Indicates inclusion An unlabeled visible light dataset of pedestrian images, This represents the i-th pedestrian image in the unlabeled visible light dataset; Indicates inclusion An unlabeled infrared dataset of pedestrian images, This represents the i-th pedestrian image in the unlabeled infrared dataset; The encoder is used to extract the visible light feature vectors respectively. ,in Represents the visible light feature vector of the i-th pedestrian image in the unlabeled visible light dataset; and Infrared feature vector ,in Represents the infrared feature vector of the i-th pedestrian image in the unlabeled infrared dataset; Indicates the dimension of the feature vector.

3. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 2, characterized in that, The specific method for step S2 is as follows: For the visible light mode and the infrared mode, k-nearest neighbor graphs are constructed based on the k-nearest neighbor method, and the k-nearest neighbor graphs are represented as point sets. , ;in, Point set representing visible light modes, The point set represents the infrared mode, and the nodes in the point set correspond to the visible light feature vector or the infrared feature vector extracted by the encoder, respectively. The set of nodes representing visible light modes. This represents the set of edges inside a visible light mode; The set of nodes representing infrared modes, Represents the set of edges inside an infrared mode; In the edge set, the weights of the edges are calculated as follows: ； Where i represents the index of the current center node; j represents the index of the neighbor node of the i-th node; This represents the weight of the edge between the i-th node and the j-th node. , It is the set of neighboring nodes of the i-th node; Temperature coefficient; Let represent the feature vector corresponding to the i-th node. This represents the feature vector corresponding to the j-th node. Let represent the feature vector corresponding to the k-th node in the set of neighbor nodes of the i-th node; exp represents the exponential function with the real number e as the base. For each visible light feature vector, graph-level matching is used to retrieve its Top-K most similar candidate feature vectors in the infrared feature space. The graph-level matching method is as follows: Define a local subgraph, for the i-th node, ,in, This indicates that the K-nearest neighbor method is used for calculation. Let represent a local subgraph centered at the i-th node; the matching score is defined as follows: ； in, This represents the matching score between the i-th node and the j-th node. This represents a local subgraph centered at the j-th node. Indicating structural similarity, , For dynamic weighting factors; Represents the set of neighboring nodes of the i-th node. Any neighboring node in the list, This represents the set of neighboring nodes of the j-th node. Any neighboring node in the list; Represents a node The corresponding feature vector, Represents a node The corresponding feature vector; This represents the number of nodes in the set of neighboring nodes of the i-th node; Using bidirectional matching, the optimal matching infrared feature vector is found among the candidate feature vectors, cross-modal edges are generated, and a unified graph is constructed, as shown in the following formula: ； in, Indicates a cross-modal edge. Represents a unified diagram; The set of nodes representing visible light modes. This represents the set of edges inside a visible light mode; The set of nodes representing infrared modes, This represents the set of edges inside the infrared mode.

4. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 3, characterized in that, The specific steps of step S3 are as follows: Step S3.1: Assign weights to the visible light feature vector and the infrared feature vector respectively, as shown in the following formula: ； in, This represents the enhanced feature vector corresponding to node i after graph feature propagation; Indicates weight; Let represent the feature vector corresponding to the i-th node. This represents the feature vector corresponding to the j-th node; It is the set of neighboring nodes of the i-th node; This represents the weight of the edge between the i-th node and the j-th node; Constructing an enhanced feature vector set For enhancing the set of feature vectors The HDBSCAN clustering algorithm, based on hierarchical density clustering, is used to perform clustering. The HDBSCAN algorithm outputs the cluster label of the i-th node. The confidence level of the i-th node belonging to its cluster. Cluster labels , For the number of clusters, This indicates that the i-th node is identified as a noise node. Other values ​​indicate that the i-th node is determined to be a non-noise node; Step S3.2: Regarding confidence level Preset thresholds T1 and T2 are used to further classify non-noise nodes: like If the i-th node is a high-confidence node, then the i-th node is a high-confidence node. like If the i-th node is a boundary node, then the i-th node is a boundary node. like If the i-th node is an approximate noise node, then the i-th node is an approximate noise node. The value of threshold T1 is dynamically adjusted.

5. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 4, characterized in that, The specific method for step S4 is as follows: Filter out noisy nodes The samples are used as cooperative noise nodes, where It is a threshold that changes dynamically with the number of training rounds t; For the feature vector corresponding to the i-th node Introducing local density estimation, the local density of the i-th node... The calculation is as follows: ； in, It is the feature distance between the i-th node and the j-th node. It is the modal label of the i-th node. It is the modal label of the j-th node; It is a modal weighting function; Indicates the total number of nodes; Indicates bandwidth or scale parameters; The modal weighting function is as follows: ； Where η is a weighting factor less than 1; For the feature vector corresponding to a high confidence node ,by As an anchor point, in the set B of boundary nodes within the same cluster but different modalities, find the boundary node most similar to the anchor point: ； in, Represents the set of boundary nodes. This represents the feature vector corresponding to the boundary node. This represents the feature vector corresponding to the boundary node most similar to the anchor point; Based on the feature vector corresponding to the boundary node most similar to the anchor point The boundary collaborative features are constructed using linear interpolation as follows: ； in, Representing boundary collaborative features, The coefficient used to control the interpolation ratio.

6. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 5, characterized in that, The specific method for step S5 is as follows: Set a threshold that changes dynamically with the number of training epochs t. As shown in the following formula: ； in, It is the initial threshold. It refers to the total number of training rounds. For the current training round, It is the attenuation coefficient; For each noise node, calculate the similarity with the anchor point. Only nodes with a similarity higher than the current threshold are considered noise nodes. Only noise nodes will be selected; The selected noise nodes are weighted and aggregated to form the following noise collaborative features: ； in, Indicates the cooperative characteristics of noise. Let be the feature vector corresponding to the i-th node. Let i be the weight corresponding to the i-th node; The weight corresponding to the i-th node The calculation is as follows: ； in, This is the sharpening factor. Let represent the outlier score of the i-th node; Let represent the outlier score of the j-th node.

7. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 6, characterized in that, The specific method for step S6 is as follows: A truncated Gaussian distribution is used to simulate the entry and exit of pedestrians. The formula for solving the parameters of the truncated Gaussian distribution is as follows: ； in, This indicates the maximum permitted percentage of pedestrians to leave. This represents the probability that the group is in a predetermined stable state. This represents the inverse function of the Gaussian error function; and Let represent the mean and standard deviation of the truncated Gaussian distribution, respectively; Based on the truncated Gaussian distribution, a formula for generating a pedestrian retention mask is constructed. For each pedestrian in the same category, it determines whether to retain them, as shown in the following formula: ； in, Reserve a mask for pedestrians. It follows the Bernoulli distribution. This represents the probability that a pedestrian leaves. express Follow the mean Standard deviation is Random variables sampled from a normal distribution; Based on the pedestrian-preserving mask, a pseudo-cluster group is obtained: ； in, This represents the k-th pseudo-cluster group; This represents the enhanced feature vector corresponding to node i after graph feature propagation; Represents the cluster label of the i-th node; The number of clusters; Calculate the center of each pseudo-cluster: ； in, This represents the center of the k-th pseudo-cluster group; This represents the confidence level that the i-th node belongs to its cluster; Finally, the constructed pseudo-clustering group center It is introduced into the training process as a positive sample representation.

8. The unsupervised cross-modal pedestrian re-identification method based on hierarchical clustering and side-noise learning according to claim 7, characterized in that, The specific method for step S7 is as follows: Constructing a dual-contrast learning loss as follows: ； in, This represents the visible light query feature vector of the i-th node after cluster center alignment or normalization. This represents the infrared query feature vector of the i-th node; Indicates a positive sample. This represents the k-th feature vector in the memory library; Indicates the temperature coefficient; It is the memory size of the visible light mode; This refers to the memory library size for the infrared mode. It is the k-th feature vector in the visible light modal memory library; It is the kth feature vector in the infrared modality memory library; These are positive samples in the infrared modality memory library; Indicates the total number of nodes; Construct modality-invariant contrastive learning loss As shown in the following formula: ； in, Let represent the arbitrary modal query feature vector of the i-th node. This represents a positive sample in a cross-modal shared memory library. This represents the k-th eigenvector in the cross-modal shared memory library. Indicates the size of the cross-modal shared memory library; Constructing boundary coordination loss As shown in the following formula: ； in, Represents the set of difficult samples; This represents the feature vector corresponding to the high-confidence node; Represents boundary collaborative features; Constructing noise-sensing loss As shown in the following formula: ； in, Indicates the cooperative characteristics of noise; Construct the complete loss As shown in the following formula: ； in, , , They are respectively , , Adjusting the weights.

Images