Text-to-image pedestrian re-identification uncertainty guidance collaborative learning method
By extracting features using the CLIP model and combining it with a Gaussian mixture model and uncertainty-guided loss, the overfitting problem caused by noise association is solved, achieving more accurate dataset classification and improved robustness in noisy environments.
Patent Information
- Application Number
- CN202511882774.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-05-15
AI Technical Summary
Existing text-to-image person re-identification methods suffer from overfitting and reduced performance when faced with noisy associations. Furthermore, existing methods ignore the uncertainty of high-loss sample pairs and fail to fully explore potential performance.
The CLIP model is used to extract global and local features of images and text. A two-component Gaussian mixture model is used to model the sample-level loss distribution. Adaptive alignment is performed through uncertainty-guided loss and reliable boundary loss. Dynamic pseudo-label correction is used to process noisy data.
It achieves more accurate dataset classification, reduces the risk of incorrect supervision, reduces mismatches and misidentifications, and improves the robustness and accuracy of the model in noisy environments.
Smart Images
Figure CN122045987A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text-to-image pedestrian re-identification technology, specifically to an uncertainty-guided collaborative learning method for text-to-image pedestrian re-identification. Background Technology
[0002] Text-to-Image Person Re-identification (TI-ReID) aims to retrieve target individuals from a pedestrian image database based on a provided text description. This technology has significant application value in intelligent surveillance, public safety, and other fields. Compared to image-based person re-identification, text-based queries offer greater flexibility. Furthermore, TI-ReID has proven particularly important and effective when relevant images of the target person are unavailable for retrieval. Given its practical application value, TI-ReID has received increasing attention in recent years. However, modal differences and redundant appearance attributes make accurate alignment of cross-modal representations difficult, making TI-ReID a highly challenging task.
[0003] To address the aforementioned challenges, existing methods primarily employ a strategy combining global and local matching to align text and visual representations. For example, methods such as [examples omitted] combine global matching with contrastive learning for image-text alignment, but lack focus on fine-grained feature discrimination. In contrast, local matching techniques, such as achieving local interactive alignment through horizontal image segmentation, have been explored. Recent research has begun to utilize pre-trained models such as BERT, ViT, CLIP, and ALBEF to learn richer feature representations, enabling both coarse-grained global alignment and finer-grained local interactions, thereby improving cross-modal matching performance.
[0004] While the aforementioned methods have made significant progress, they all rely on a key assumption: that all training image-text pairs are correctly aligned. In practical applications, due to uncontrollable factors such as pedestrian pose, camera angle, and lighting conditions, manually labeled text descriptions often contain a certain proportion of inaccuracies or errors. This noise can lead to overfitting during model training, significantly reducing the performance of existing text-to-image person re-identification (TI-ReID) methods. Therefore, we focus on a specific type of noise—noisy correlation. Noisy correlation refers to the semantic mismatch between an image and its paired text; that is, image-text pairs originally labeled as positive samples actually have semantic mismatches or inconsistencies. Unlike traditional category-level label noise, this problem is difficult to directly correct using standard label noise processing methods.
[0005] To address these challenges, RDE employs a Gaussian Mixture Model (GMM) combined with the Expectation-Maximization (EM) algorithm. It divides the training samples into "clean" and "noisy" subsets based on a preset loss threshold, then discards the noisy samples. However, this coarse-grained binary classification ignores the inherent uncertainty in high-loss sample pairs, resulting in the potential performance improvements not being fully explored.
[0006] Therefore, a new solution is needed to address the above problems. Summary of the Invention
[0007] The purpose of this invention is to provide an uncertainty-guided collaborative learning method for text-to-image pedestrian re-identification, in order to solve the technical problems mentioned in the background art.
[0008] To achieve the above objectives, the present invention provides the following technical solution: an uncertainty-guided collaborative learning method for text-to-image person re-identification, comprising at least the following steps:
[0009] S1: Feature extraction and similarity calculation. The encoder architecture of the CLIP model is used to extract global and local features of the input image and text respectively, and calculate global and local similarity.
[0010] S2: Precise correspondence identification. A two-component Gaussian mixture model is used to model the sample-level loss distribution obtained from global and local similarity measurements, thereby achieving precise dataset partitioning. That is, based on the Gaussian mixture model, the global and local features of the samples are jointly analyzed, and the dataset is divided into clean dataset, uncertain dataset, and noisy dataset.
[0011] S3: Uncertainty-guided alignment. For uncertain datasets, sample uncertainty is modeled through a dual-channel evidence flow, and adaptive alignment is achieved by combining uncertainty-guided loss and reliable boundary loss.
[0012] S4: Dynamic pseudo-label correction. For noisy datasets, a dynamic pseudo-label generation method is proposed to alleviate the problem of excessive memory of noise in the model.
[0013] S5: Model training and inference. The model is trained using a composite objective function. Different optimization strategies are adopted for clean data, uncertain data and noisy data partitions. In the inference stage, global and local similarities are aggregated to obtain the final retrieval results.
[0014] Furthermore, S1 includes at least the following steps:
[0015] First, the input image is used to extract features using the CLIP image encoder to obtain the representation sequence. ;
[0016] The representation sequence includes positional encoding information and special CLS markers. The encoded CLS tags are used as a global image representation. ;
[0017] Accordingly, the text description is processed by the CLIP text encoder based on an improved Transformer architecture, employing a byte-pair encoding algorithm containing 49,152 words to generate a representation sequence. ;
[0018] The end of the sequence As a global text feature ;
[0019] Local representations are obtained from the labeled output of CLIP, including visual sequences. With text sequence ;
[0020] These labeled sequences are processed by the Temporal Enhancement (TSE) module of RDE to generate optimized local image and local text feature representations. and This is used for subsequent matching and alignment tasks;
[0021] Based on the global and local feature representations, the global and local similarities between the transformed representations are calculated as follows: .
[0022] Furthermore, S2 includes at least the following steps:
[0023] Let the set of training sample pairs be represented as ,in, Indicates the first One image, Indicates the first A text or label, This is the total number of training samples;
[0024] Given a cross-modal model The loss for each sample is defined as:
[0025]
[0026] in, This represents the TAL loss function proposed in RDE. Specifically, it applies the triple alignment loss to both global and local feature representations. For global features, the loss is calculated as follows:
[0027]
[0028] in, and These represent the weighted average similarity scores of positive sample pairs in the image-to-text and text-to-image directions, respectively. Representing an image and text Global feature similarity between them; Indicates temperature parameter; Represents the probability weights of the categories; This represents a constant value used to control the threshold for loss.
[0029] For local features, the loss function is calculated as follows:
[0030]
[0031] in, Representing an image and text Local feature similarity between them;
[0032] Define hinge addition operator ax(x,0), the exponential function And record the small batch size as , It refers to the batch size;
[0033] Pair matching tags {0,1} is used to represent an image. With text Whether they constitute a semantic matching pair, where =1 indicates a positive sample pair. =0 represents a negative sample pair; based on this, the weights are defined. To reverse the matching relationship;
[0034] =
[0035] in, / These are the normalized weights for positive sample pairs; similarly, the weights in the text-to-image direction can be defined. ;
[0036] These metrics enhance highly similar positive sample pairs through temperature-scaled softmax weighting, thus providing a comprehensive evaluation index of cross-modal alignment quality;
[0037] By fitting the sample loss with a Gaussian mixture model, two Gaussian distributions with low mean and high mean, respectively, are obtained.
[0038] Based on the principles of statistical learning, low-mean distributions are associated with clean data clusters, and high-mean distributions are associated with noisy data clusters.
[0039] The GMM parameters are optimized using the expectation-maximization algorithm, and for each sample, its posterior membership probability is calculated using Bayes' theorem.
[0040]
[0041] in, It is a sample Belongs to components Posterior probability; latent variable , representing a clean sample ( or noise sample ( ); Indicates components The prior probability; Given components Lower sample The likelihood is the probability distribution of a sample under that component. Indicates sample The marginal probability;
[0042] Subsequently, the posterior probability sequence {p(k=0| ) By setting a partition threshold Version 0.5 implements the partitioning of the dataset into clean and noisy subsets:
[0043]
[0044]
[0045] By combining the results of global and local partitioning, more reliable and accurate noise sample pair identification can be obtained, providing a foundation for subsequent uncertainty perception processing;
[0046] make and These represent the clean set and the noisy set obtained from the partitioning, respectively.
[0047] For global features and local features, the partitioning results are obtained respectively using formula (6). and ;
[0048] It should be noted that both partitions cover the complete dataset, i.e., satisfying... ;
[0049] To obtain the final optimized partition, the intersection of the global and local feature classification results is taken as the clean dataset, i.e. The remaining data was divided into a noisy dataset and an uncertain dataset, i.e. and ;
[0050] Therefore, the final dataset is divided into clean datasets. Uncertainty dataset and noise dataset .
[0051] Furthermore, S3 includes at least the following steps:
[0052] To effectively handle uncertain subsets For difficult sample pairs, this paper proposes an uncertainty modeling framework based on subjective logic and Dirichlet distribution. It captures complementary features of matching confidence through a dual-channel evidence stream. Specifically:
[0053] Regarding the first Text and the first Global feature similarity score between images (As defined in formula (1)), evidence is extracted using two different activation functions:
[0054]
[0055]
[0056] Among them, temperature coefficient =0.1 is used to control based on Sensitivity of evidence extraction using the hyperbolic tangent function; It is a nonlinear function;
[0057] For each stream of evidence Calculate the Dirichlet distribution parameters respectively:
[0058]
[0059] in, Indicates the s-th evidence stream. The first text description and the first Dirichlet concentration parameters for candidate images; Representing the The text in the first... The total intensity (sum of concentration parameters) of each evidence stream; based on this, The expected matching probability is represented, while the uncertainty is quantified by the following formula:
[0060]
[0061] in, Indicates the total number of candidate images;
[0062] Subsequently, the uncertainties of the two evidence streams were calculated. The arithmetic mean of the , yielding the . The overall uncertainty of the text description:
[0063]
[0064] The aggregation uncertainty By fusing dual-stream evidence features, a comprehensive measure of matching reliability is provided for each image-text pair;
[0065] For each evidence stream, the loss function integrates three complementary optimization objectives;
[0066] Among them, the expected alignment item Used to ensure consistency between the predicted probability distribution and the true annotation:
[0067]
[0068] This uncertainty calibration item The uncertainty estimation is regularized using the inherent variance property of the Dirichlet distribution:
[0069]
[0070] This uncertainty calibration term utilizes the inherent variance property of the Dirichlet distribution to apply regularization constraints to the uncertainty estimate; the distribution regularization term... Then, parameter stability is maintained through KL divergence:
[0071]
[0072]
[0073] in, It is a weighting factor; Indicates KL divergence; Indicates the Dirichlet distribution; Let N be a vector of length N, consisting of all 1s, representing a uniformly distributed concentration parameter. It represents the Hadamardi (or Hadama) stack;
[0074] Indicates the first The first in the evidence stream The Dirichlet concentration parameter corresponding to each text sample, and each component of it... Characterizing the first The text and the first The strength of evidence for each image match;
[0075] The composite loss function for each evidence stream integrates three components:
[0076]
[0077] An adaptive weighting strategy based on uncertainty is used to fuse the two evidence streams;
[0078] The sample is categorized according to overall uncertainty. Obtain the corresponding rank by sorting in descending order. By constructing a weight function , here ∈ (0, 1];
[0079] The final definition of the uncertainty-guided weighted loss function is:
[0080]
[0081] in, and They represent and The evidence flow is a composite loss as defined by formula (16);
[0082] To mitigate the negative impact of inaccurate soft margins, a reliable boundary loss function is designed. This function fully utilizes the modeled uncertainties, and the formula for calculating the reliable boundary is defined as follows:
[0083]
[0084] in, [0,1] represents the overall uncertainty of the i-th image-text pair; The scaling factor is used to control the sensitivity to changes in uncertainty; Represents the baseline marginal value; parameter As an uncertainty threshold to distinguish between pairs of low and high uncertainties;
[0085] Integrating global similarity into the reliable boundary loss function, the final definition is as follows:
[0086]
[0087] in, This represents the global similarity between matched image-text pairs (as defined in Equation (1)); This represents the similarity function used for mining the most difficult negative samples; and They represent the most difficult negative samples, =max(0,x) is the hinge loss function. This formula enhances the separation boundary for high-confidence matches through an adaptive mechanism, while relaxing the constraints on low-confidence samples, thereby effectively improving the robustness of the model in noisy environments.
[0088] Furthermore, S4 includes at least the following steps:
[0089] For belonging to the noise set For sample pairs, the original unreliable labels are replaced by processing the similarity scores between text-image pairs. The specific matching probability calculation is as follows:
[0090]
[0091]
[0092] in, and Representation based on global similarity matrix The calculated bidirectional matching probability; It is obtained by applying the softmax function to the row vectors of the matrix, i.e., the direction from image to text. It is then calculated by applying the softmax function to the column vectors of the matrix, i.e., the direction from text to image;
[0093] The specific formula for updating pseudo-tags is as follows:
[0094]
[0095] Among them, momentum coefficient Set to 0.4; Indicates the first Estimated labels in each training cycle;
[0096]
[0097] By calculating the arithmetic mean of the probabilities in the two directions, It provides a symmetric and robust confidence measure for the quality of correspondences, and this estimated probability is further applied to the pseudo-label update rule based on momentum smoothing;
[0098] Accordingly, dynamically updated pseudo-labels are used to label the noise set, while maintaining the original labels of the clean set and the uncertain set. The specific definitions are as follows: .
[0099] Furthermore, S5 includes at least the following steps:
[0100] For clean subsets The goal is to achieve cross-modal matching by bringing positive sample pairs closer and negative sample pairs further apart in the embedding space. To ensure full alignment, the triple alignment loss is applied to the global similarity simultaneously. Local similarity Their definitions are shown in formulas (1) and (2) respectively;
[0101] The loss function for subsets is defined as follows:
[0102]
[0103] For uncertain subsets The sample pairs in the dataset mitigate the impact of potential noise annotations through an uncertainty perception mechanism.
[0104] Two complementary strategies are adopted:
[0105] Uncertainty-driven loss The sample weighting mechanism, and the reliable boundary loss Achieved adaptive boundary adjustment;
[0106] Both components use global feature similarity. As the basis for uncertainty estimation and boundary adjustment;
[0107] The overall loss function for uncertain subsets is defined as follows:
[0108]
[0109] For noise subset The sample pairs in the dataset use dynamically corrected pseudo-labels. To mitigate the impact of incorrect monitoring signals, the loss function is defined as follows:
[0110]
[0111] in, Represents the cross-entropy loss function; These are the corrected pseudo-labels obtained through a dynamic update strategy;
[0112] The overall objective function is composed of a weighted sum of three loss components:
[0113]
[0114] in, It is a weighting factor used for balancing. The contribution of the loss;
[0115] In the joint inference phase, the final image-text similarity is obtained by aggregating the global embedding score and the local embedding score using an arithmetic mean method, i.e., satisfying... = ( .
[0116] Compared with the prior art, the beneficial effects of the present invention are:
[0117] 1. This invention proposes a precise correspondence recognition module that combines global and local features to achieve more accurate dataset classification, effectively reducing the risk of incorrect supervision and minimizing mismatches and misidentifications. It achieves more accurate data classification through a multi-level sample partitioning strategy, effectively reducing the risk of incorrect supervision and minimizing mismatches and misidentifications.
[0118] 2. This invention proposes an uncertainty-guided alignment module, which models the uncertainty of samples, estimates the uncertainty to enhance the contribution of positive sample pairs and suppress the negative impact of mismatched pairs. By estimating the uncertainty of samples, the processing mechanism for unreliable matching pairs is optimized, thereby enhancing the contribution of positive sample pairs while suppressing the negative impact of mismatched pairs.
[0119] 3. This invention proposes a dynamic pseudo-label correction module, which dynamically updates pseudo-labels for noisy subsets to improve model robustness. Combined with a selective supervision strategy, dynamic pseudo-label updates are implemented for noisy data subsets, significantly improving model robustness. Attached Figure Description
[0120] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0121] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation
[0122] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0123] Please see Figure 1 An uncertainty-guided collaborative learning method for text-to-image person re-identification includes at least the following steps:
[0124] S1: Feature extraction and similarity calculation. The encoder architecture of the CLIP model is used to extract global and local features of the input image and text respectively, and calculate global and local similarity.
[0125] S2: Precise correspondence identification. A two-component Gaussian mixture model is used to model the sample-level loss distribution obtained from global and local similarity measurements, thereby achieving precise dataset partitioning. That is, based on the Gaussian mixture model, the global and local features of the samples are jointly analyzed, and the dataset is divided into clean dataset, uncertain dataset, and noisy dataset.
[0126] S3: Uncertainty-guided alignment. For uncertain datasets, sample uncertainty is modeled through a dual-channel evidence flow, and adaptive alignment is achieved by combining uncertainty-guided loss and reliable boundary loss.
[0127] S4: Dynamic pseudo-label correction. For noisy datasets, a dynamic pseudo-label generation method is proposed to alleviate the problem of excessive memory of noise in the model.
[0128] S5: Model training and inference. The model is trained using a composite objective function. Different optimization strategies are adopted for clean data, uncertain data and noisy data partitions. In the inference stage, global and local similarities are aggregated to obtain the final retrieval results.
[0129] S1 includes at least the following steps:
[0130] First, the input image is used to extract features using the CLIP image encoder to obtain the representation sequence. ;
[0131] The representation sequence includes positional encoding information and special CLS markers. The encoded CLS tags are used as a global image representation. ;
[0132] Accordingly, the text description is processed by the CLIP text encoder based on an improved Transformer architecture, employing a byte-pair encoding algorithm containing 49,152 words to generate a representation sequence. ;
[0133] The end of the sequence As a global text feature ;
[0134] Local representations are obtained from the labeled output of CLIP, including visual sequences. With text sequence ;
[0135] These labeled sequences are processed by the Temporal Enhancement (TSE) module of RDE to generate optimized local image and local text feature representations. and This is used for subsequent matching and alignment tasks;
[0136] Based on the global and local feature representations, the global and local similarities between the transformed representations are calculated as follows: .
[0137] S2 includes at least the following steps:
[0138] Let the set of training sample pairs be represented as ,in, Indicates the first One image, Indicates the first A text or label, This is the total number of training samples;
[0139] Given a cross-modal model The loss for each sample is defined as:
[0140]
[0141] in, This represents the TAL loss function proposed in RDE. Specifically, it applies the triple alignment loss to both global and local feature representations. For global features, the loss is calculated as follows:
[0142]
[0143] in, and These represent the weighted average similarity scores of positive sample pairs in the image-to-text and text-to-image directions, respectively. Representing an image and text Global feature similarity between them; Indicates temperature parameter; Represents the probability weights of the categories; This represents a constant value used to control the threshold for loss.
[0144] For local features, the loss function is calculated as follows:
[0145]
[0146] in, Representing an image and text Local feature similarity between them;
[0147] Define hinge addition operator ax(x,0), the exponential function And record the small batch size as , It refers to the batch size;
[0148] Pair matching tags {0,1} is used to represent an image. With text Whether they constitute a semantic matching pair, where =1 indicates a positive sample pair. =0 represents a negative sample pair; based on this, the weights are defined. To reverse the matching relationship, boundary parameters Set to 0.1, temperature parameter Set to 0.015;
[0149] =
[0150] in, / These are the normalized weights for positive sample pairs; similarly, the weights in the text-to-image direction can be defined. ;
[0151] These metrics enhance highly similar positive sample pairs through temperature-scaled softmax weighting, thus providing a comprehensive evaluation index of cross-modal alignment quality;
[0152] By fitting the sample loss with a Gaussian mixture model, two Gaussian distributions with low mean and high mean, respectively, are obtained.
[0153] Based on the principles of statistical learning, low-mean distributions are associated with clean data clusters, and high-mean distributions are associated with noisy data clusters.
[0154] The GMM parameters are optimized using the expectation-maximization algorithm, and for each sample, its posterior membership probability is calculated using Bayes' theorem.
[0155]
[0156] in, It is a sample Belongs to components Posterior probability; latent variable , representing a clean sample ( or noise sample ( ); Indicates components The prior probability; Given components Lower sample The likelihood is the probability distribution of a sample under that component. Indicates sample The marginal probability;
[0157] Subsequently, the posterior probability sequence {p(k=0| ) By setting a partition threshold Version 0.5 implements the partitioning of the dataset into clean and noisy subsets:
[0158]
[0159]
[0160] By combining the results of global and local partitioning, more reliable and accurate noise sample pair identification can be obtained, providing a foundation for subsequent uncertainty perception processing;
[0161] make and These represent the clean set and the noisy set obtained from the partitioning, respectively.
[0162] For global features and local features, the partitioning results are obtained respectively using formula (6). and ;
[0163] It should be noted that both partitions cover the complete dataset, i.e., satisfying... ;
[0164] To obtain the final optimized partition, the intersection of the global and local feature classification results is taken as the clean dataset, i.e. The remaining data was divided into a noisy dataset and an uncertain dataset, i.e. and ;
[0165] Therefore, the final dataset is divided into clean datasets. Uncertainty dataset and noise dataset .
[0166] S3 includes at least the following steps:
[0167] To effectively handle uncertain subsets For difficult sample pairs, this paper proposes an uncertainty modeling framework based on subjective logic and Dirichlet distribution. It captures complementary features of matching confidence through a dual-channel evidence stream. Specifically:
[0168] Regarding the first Text and the first Global feature similarity score between images (As defined in formula (1)), evidence is extracted using two different activation functions:
[0169]
[0170]
[0171] Among them, temperature coefficient =0.1 is used to control based on Sensitivity of evidence extraction using the hyperbolic tangent function; It is a nonlinear function;
[0172] For each stream of evidence Calculate the Dirichlet distribution parameters respectively:
[0173]
[0174] in, Indicates the s-th evidence stream. The first text description and the first Dirichlet concentration parameters for candidate images; Representing the The text in the first... The total intensity (sum of concentration parameters) of each evidence stream; based on this, The expected matching probability is represented, while the uncertainty is quantified by the following formula:
[0175]
[0176] in, Indicates the total number of candidate images;
[0177] Subsequently, the uncertainties of the two evidence streams were calculated. The arithmetic mean of the , yielding the . The overall uncertainty of the text description:
[0178]
[0179] The aggregation uncertainty By fusing dual-stream evidence features, a comprehensive measure of matching reliability is provided for each image-text pair;
[0180] For each evidence stream, the loss function integrates three complementary optimization objectives;
[0181] Among them, the expected alignment item Used to ensure consistency between the predicted probability distribution and the true annotation:
[0182]
[0183] This uncertainty calibration item The uncertainty estimation is regularized using the inherent variance property of the Dirichlet distribution:
[0184]
[0185] This uncertainty calibration term utilizes the inherent variance property of the Dirichlet distribution to apply regularization constraints to the uncertainty estimate; the distribution regularization term... Then, parameter stability is maintained through KL divergence:
[0186]
[0187]
[0188] in, It is a weighting factor; Indicates KL divergence; Indicates the Dirichlet distribution; Let N be a vector of length N, consisting of all 1s, representing a uniformly distributed concentration parameter. It represents the Hadamardi (or Hadama) stack;
[0189] This formula effectively prevents the accumulation of false evidence by selectively regularizing the distribution of non-target categories to a uniform distribution.
[0190] Indicates the first The first in the evidence stream The Dirichlet concentration parameter corresponding to each text sample, and each component of it... Characterizing the first The text and the first The strength of evidence for each image match;
[0191] The composite loss function for each evidence stream integrates three components:
[0192]
[0193] An adaptive weighting strategy based on uncertainty is used to fuse the two evidence streams;
[0194] The sample is categorized according to overall uncertainty. Obtain the corresponding rank by sorting in descending order. By constructing a weight function , here ∈ (0, 1];
[0195] The final definition of the uncertainty-guided weighted loss function is:
[0196]
[0197] in, and They represent and The evidence flow is a composite loss as defined by formula (16);
[0198] This optimization framework improves model performance through a triple collaborative mechanism: leveraging... This accurately aligns the predicted results with the actual annotations, through... The term maintains a well-calibrated uncertainty estimate and utilizes This effectively prevents parameter degradation. This weighting mechanism, which incorporates uncertainty awareness, allows the model to focus on learning high-confidence matching samples while adaptively handling challenging difficult samples, thus significantly enhancing the model's robustness in cross-modal retrieval tasks.
[0199] Currently, contrastive loss functions have achieved significant results in bringing positive sample pairs closer together and pushing negative sample pairs further apart; however, their performance often degrades significantly in noisy data environments. On the other hand, soft-margin strategies rely solely on the estimated soft-correspondence labels, making them extremely sensitive to estimation accuracy, which can easily lead to poor model performance, especially in high-noise scenarios. To mitigate the negative impact of inaccurate soft margins, a reliable boundary loss function is designed. This function fully utilizes the modeled uncertainties, aiming to provide a more reliable and accurate soft-margin correction mechanism. The formula for calculating the reliable boundary is defined as follows:
[0200]
[0201] in, [0,1] represents the overall uncertainty of the i-th image-text pair; The scaling factor is used to control the sensitivity to changes in uncertainty; Represents the baseline marginal value; parameter As an uncertainty threshold to distinguish between pairs of low and high uncertainties;
[0202] Integrating global similarity into the reliable boundary loss function, the final definition is as follows:
[0203]
[0204] This formula ensures that when uncertainty is... equal to threshold At that time, the adjusted marginal value Exactly equal to the baseline value For those with low uncertainty ( For high-confidence sample pairs, the marginal value will be expanded based on the baseline value to improve the feature discrimination ability; conversely, for samples with high uncertainty (…), the marginal value will be expanded based on the baseline value to improve the feature discrimination ability. For low-confidence sample pairs, the marginal value will be appropriately reduced to mitigate the potential negative impact of mismatches. Furthermore, the global similarity is integrated into the reliable boundary loss function, ultimately defined as follows:
[0205]
[0206] in, This represents the global similarity between matched image-text pairs (as defined in Equation (1)); This represents the similarity function used for mining the most difficult negative samples; and They represent the most difficult negative samples, =max(0,x) is the hinge loss function. This formula enhances the separation boundary for high-confidence matches through an adaptive mechanism, while relaxing the constraints on low-confidence samples, thereby effectively improving the robustness of the model in noisy environments.
[0207] The uncertainty-guided alignment framework proposed by S3 integrates two complementary optimization objectives: uncertainty-guided loss. With reliable boundary loss This framework is achieved through... Establishing a Dirichlet distribution parameterization Dir under the subjective logic paradigm To model prediction uncertainty, this loss function aligns the predicted probability distribution with the true label while regularizing the prediction variance and applying a sample weighting mechanism based on uncertainty, thereby effectively improving model calibration and reliability. Meanwhile, By fully utilizing these uncertainty estimates to adaptively adjust the decision boundary, the boundary margin is expanded for high-reliability sample pairs, and the constraints are relaxed for uncertain sample pairs, thereby effectively mitigating the impact of noise correspondences. This framework... Obtain a well-calibrated confidence estimate, while utilizing By optimizing boundary discrimination capabilities, this joint optimization strategy simultaneously enhances the model's alignment robustness and feature discriminativeness in noisy environments.
[0208] S4 includes at least the following steps:
[0209] For belonging to the noise set For sample pairs, the original unreliable labels are replaced by processing the similarity scores between text-image pairs. The specific matching probability calculation is as follows:
[0210]
[0211]
[0212] in, and Representation based on global similarity matrix The calculated bidirectional matching probability; It is obtained by applying the softmax function to the row vectors of the matrix, i.e., the direction from image to text. It is then calculated by applying the softmax function to the column vectors of the matrix, i.e., the direction from text to image;
[0213] The specific formula for updating pseudo-tags is as follows:
[0214]
[0215] Among them, momentum coefficient Set to 0.4; Indicates the first Estimated labels in each training cycle;
[0216]
[0217] By calculating the arithmetic mean of the probabilities in the two directions, It provides a symmetric and robust confidence measure for the quality of correspondences, and this estimated probability is further applied to the pseudo-label update rule based on momentum smoothing;
[0218] Accordingly, dynamically updated pseudo-labels are used to label the noise set, while maintaining the original labels of the clean set and the uncertain set. The specific definitions are as follows: .
[0219] S5 includes at least the following steps:
[0220] For clean subsets The goal is to achieve cross-modal matching by bringing positive sample pairs closer and negative sample pairs further apart in the embedding space. To ensure full alignment, the triple alignment loss is applied to the global similarity simultaneously. Local similarity Their definitions are shown in formulas (1) and (2) respectively;
[0221] The loss function for subsets is defined as follows:
[0222]
[0223] For uncertain subsets The sample pairs in the dataset mitigate the impact of potential noise annotations through an uncertainty perception mechanism.
[0224] Two complementary strategies are adopted:
[0225] Uncertainty-driven loss The sample weighting mechanism, and the reliable boundary loss Achieved adaptive boundary adjustment;
[0226] Both components use global feature similarity. As the basis for uncertainty estimation and boundary adjustment;
[0227] The overall loss function for uncertain subsets is defined as follows:
[0228]
[0229] For noise subset The sample pairs in the dataset use dynamically corrected pseudo-labels. To mitigate the impact of incorrect monitoring signals, the loss function is defined as follows:
[0230]
[0231] in, Represents the cross-entropy loss function; These are the corrected pseudo-labels obtained through a dynamic update strategy;
[0232] The overall objective function is composed of a weighted sum of three loss components:
[0233]
[0234] in, It is a weighting factor used for balancing. The contribution of the loss;
[0235] In the joint inference phase, the final image-text similarity is obtained by aggregating the global embedding score and the local embedding score using an arithmetic mean method, i.e., satisfying... = ( .
[0236] In summary:
[0237] For input image-text pairs, a pre-trained encoder based on a contrastive language-image pre-trained model is first used to extract image and text features respectively. The encoded features are then input into a Precise Correspondence Identification (PCI) module. This module uses a Gaussian mixture model to jointly analyze the global and local features of the samples, dividing the data into three semantically defined subsets: a clean subset, an uncertain subset, and a noisy subset. Each subset employs a differentiated processing strategy: the clean subset directly performs cross-modal matching learning through triplet alignment loss; the uncertain subset is processed by an Uncertainty Guided Alignment (UGA) module, which achieves adaptive weight allocation and dynamic interval adjustment through sample-level confidence estimation; and the noisy subset undergoes progressive label optimization through a Dynamic Pseudo-Label Correction (DPR) module, with high-confidence corrected samples re-injected into the training process. Through the synergistic optimization of these three modules, the system effectively suppresses noise correspondence interference and significantly improves the robustness of cross-modal retrieval. This framework enhances the system's adaptability to noisy data while maintaining the model's discriminative power.
[0238] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. An uncertainty-guided collaborative learning method for text-to-image person re-identification, characterized in that: At least the following steps are included: S1: Feature extraction and similarity calculation. The encoder architecture of the CLIP model is used to extract global and local features of the input image and text respectively, and calculate global and local similarity. S2: Precise correspondence identification. A two-component Gaussian mixture model is used to model the sample-level loss distribution obtained from global and local similarity measurements, thereby achieving precise dataset partitioning. That is, based on the Gaussian mixture model, the global and local features of the samples are jointly analyzed, and the dataset is divided into clean dataset, uncertain dataset, and noisy dataset. S3: Uncertainty-guided alignment. For uncertain datasets, sample uncertainty is modeled through a dual-channel evidence flow, and adaptive alignment is achieved by combining uncertainty-guided loss and reliable boundary loss. S4: Dynamic pseudo-label correction. For noisy datasets, a dynamic pseudo-label generation method is proposed to alleviate the problem of excessive memory of noise in the model. S5: Model training and inference. The model is trained using a composite objective function. Different optimization strategies are adopted for clean data, uncertain data and noisy data partitions. In the inference stage, global and local similarities are aggregated to obtain the final retrieval results.
2. The uncertainty-guided collaborative learning method for text-to-image person re-identification according to claim 1, characterized in that: S1 includes at least the following steps: First, the input image is used to extract features using the CLIP image encoder to obtain the representation sequence. ; The representation sequence includes positional encoding information and special CLS markers. The encoded CLS tags are used as a global image representation. ; Accordingly, the text description is processed by the CLIP text encoder based on an improved Transformer architecture, employing a byte-pair encoding algorithm containing 49,152 words to generate a representation sequence. ; The end of the sequence As a global text feature ; Local representations are obtained from the labeled output of CLIP, including visual sequences. With text sequence ; These labeled sequences are processed by the temporal enhancement module of RDE to generate optimized local image and local text feature representations. and This is used for subsequent matching and alignment tasks; Based on the global and local feature representations, the global and local similarities between the transformed representations are calculated as follows: .
3. The uncertainty-guided collaborative learning method for text-to-image person re-identification according to claim 2, characterized in that: S2 includes at least the following steps: Let the set of training sample pairs be represented as ,in, Indicates the first One image, Indicates the first A text or label, This is the total number of training samples; Given a cross-modal model The loss for each sample is defined as: in, This represents the TAL loss function proposed in RDE. Specifically, it applies the triple alignment loss to both global and local feature representations. For global features, the loss is calculated as follows: in, and These represent the weighted average similarity scores of positive sample pairs in the image-to-text and text-to-image directions, respectively. Representing an image and text Global feature similarity between them; Indicates temperature parameter; Represents the probability weights of the categories; This represents a constant value used to control the threshold for loss. For local features, the loss function is calculated as follows: in, Representing an image and text Local feature similarity between them; Define hinge addition operator ax(x,0), the exponential function And record the small batch size as , It refers to the batch size; Pair matching tags {0,1} is used to represent an image. With text Whether they constitute a semantic matching pair, where =1 indicates a positive sample pair. =0 represents a negative sample pair; based on this, the weights are defined. To reverse the matching relationship, boundary parameters Set to 0.1, temperature parameter Set to 0.015; = in, / These are the normalized weights for positive sample pairs; similarly, the text-to-image direction can be defined. ; These metrics enhance highly similar positive sample pairs through temperature-scaled softmax weighting, thus providing a comprehensive evaluation index of cross-modal alignment quality; By fitting the sample loss with a Gaussian mixture model, two Gaussian distributions with low mean and high mean, respectively, are obtained. Based on the principles of statistical learning, low-mean distributions are associated with clean data clusters, and high-mean distributions are associated with noisy data clusters. The GMM parameters are optimized using the expectation-maximization algorithm, and for each sample, its posterior membership probability is calculated using Bayes' theorem. in, It is a sample Belongs to components Posterior probability; latent variable , representing a clean sample or a noisy sample; Indicates components The prior probability; Given components Lower sample The likelihood is the probability distribution of a sample under that component. Indicates sample The marginal probability; Subsequently, the posterior probability sequence {p(k=0| ) By setting a partition threshold Version 0.5 implements the partitioning of the dataset into clean and noisy subsets: By combining the results of global and local partitioning, more reliable and accurate noise sample pair identification can be obtained, providing a foundation for subsequent uncertainty perception processing; make and These represent the clean set and the noisy set obtained from the partitioning, respectively. For global features and local features, the partitioning results are obtained respectively using formula (6). and ; It should be noted that both partitions cover the complete dataset, i.e., satisfying... ; To obtain the final optimized partition, the intersection of the global and local feature classification results is taken as the clean dataset, i.e. The remaining data was divided into a noisy dataset and an uncertain dataset, i.e. and ; Therefore, the final dataset is divided into clean datasets. Uncertainty dataset and noise dataset .
4. The uncertainty-guided collaborative learning method for text-to-image person re-identification according to claim 3, characterized in that: The S3 includes at least the following steps: To effectively handle uncertain subsets For difficult sample pairs, this paper proposes an uncertainty modeling framework based on subjective logic and Dirichlet distribution. It captures complementary features of matching confidence through a dual-channel evidence stream. Specifically: Regarding the first Text and the first Global feature similarity score between images Evidence was extracted using two different activation functions: Among them, temperature coefficient =0.1 is used to control based on Sensitivity of evidence extraction using the hyperbolic tangent function; It is a nonlinear function; For each stream of evidence Calculate the Dirichlet distribution parameters respectively: in, Indicates the s-th evidence stream. The first text description and the first Dirichlet concentration parameters for candidate images; Representing the The text in the first... The total strength of the evidence flow; based on this... The expected matching probability is represented, while the uncertainty is quantified by the following formula: in, Indicates the total number of candidate images; Subsequently, the uncertainties of the two evidence streams were calculated. The arithmetic mean of the , yielding the . The overall uncertainty of the text description: The aggregation uncertainty By fusing dual-stream evidence features, a comprehensive measure of matching reliability is provided for each image-text pair; For each evidence stream, the loss function integrates three complementary optimization objectives; Among them, the expected alignment item Used to ensure consistency between the predicted probability distribution and the true annotation: This uncertainty calibration item The uncertainty estimation is regularized using the inherent variance property of the Dirichlet distribution: This uncertainty calibration term utilizes the inherent variance property of the Dirichlet distribution to apply regularization constraints to the uncertainty estimate; the distribution regularization term... Then, parameter stability is maintained through KL divergence: in, It is a weighting factor; Indicates KL divergence; Indicates the Dirichlet distribution; Let N be a vector of length N, consisting of all 1s, representing a uniformly distributed concentration parameter. It represents the Hadamardi (or Hadama) stack; Indicates the first The first in the evidence stream The Dirichlet concentration parameter corresponding to each text sample, and each component of it... Characterizing the first The text and the first The strength of evidence for each image match; The composite loss function for each evidence stream integrates three components: An adaptive weighting strategy based on uncertainty is used to fuse the two evidence streams; The sample is categorized according to overall uncertainty. Obtain the corresponding rank by sorting in descending order. By constructing a weight function , here ∈ (0, 1]; The final definition of the uncertainty-guided weighted loss function is: in, and They represent and The evidence flow is a composite loss as defined by formula (16); To mitigate the negative impact of inaccurate soft margins, a reliable boundary loss function is designed. This function fully utilizes the modeled uncertainties, and the formula for calculating the reliable boundary is defined as follows: in, [0,1] represents the overall uncertainty of the i-th image-text pair; The scaling factor is used to control the sensitivity to changes in uncertainty; Represents the baseline marginal value; parameter As an uncertainty threshold to distinguish between pairs of low and high uncertainties; Integrating global similarity into the reliable boundary loss function, the final definition is as follows: in, This represents the global similarity between matched image-text pairs; This represents the similarity function used for mining the most difficult negative samples; and They represent the most difficult negative samples, =max(0,x) is the hinge loss function. This formula enhances the separation boundary for high-confidence matches through an adaptive mechanism, while relaxing the constraints on low-confidence samples, thereby effectively improving the robustness of the model in noisy environments.
5. The uncertainty-guided collaborative learning method for text-to-image person re-identification according to claim 4, characterized in that: The S4 includes at least the following steps: For belonging to the noise set For sample pairs, the original unreliable labels are replaced by processing the similarity scores between text-image pairs. The specific matching probability calculation is as follows: in, and Representation based on global similarity matrix The calculated bidirectional matching probability; It is obtained by applying the softmax function to the row vectors of the matrix, i.e., the direction from image to text. It is then calculated by applying the softmax function to the column vectors of the matrix, i.e., the direction from text to image; The specific formula for updating pseudo-tags is as follows: Among them, momentum coefficient Set to 0.4; Indicates the first Estimated labels in each training cycle; By calculating the arithmetic mean of the probabilities in the two directions, It provides a symmetric and robust confidence measure for the quality of correspondences, and this estimated probability is further applied to the pseudo-label update rule based on momentum smoothing; Accordingly, dynamically updated pseudo-labels are used to label the noise set, while maintaining the original labels of the clean set and the uncertain set. The specific definitions are as follows: .
6. The uncertainty-guided collaborative learning method for text-to-image person re-identification according to claim 5, characterized in that: The S5 includes at least the following steps: For clean subsets The goal is to achieve cross-modal matching by bringing positive sample pairs closer and negative sample pairs further apart in the embedding space. To ensure full alignment, the triple alignment loss is applied to the global similarity simultaneously. Local similarity Their definitions are shown in formulas (1) and (2) respectively; The loss function for subsets is defined as follows: For uncertain subsets The sample pairs in the dataset mitigate the impact of potential noise annotations through an uncertainty perception mechanism. Two complementary strategies are adopted: Uncertainty-driven loss The sample weighting mechanism, and the reliable boundary loss Achieved adaptive boundary adjustment; Both components use global feature similarity. As the basis for uncertainty estimation and boundary adjustment; The overall loss function for uncertain subsets is defined as follows: For noise subset The sample pairs in the dataset use dynamically corrected pseudo-labels. To mitigate the impact of incorrect monitoring signals, the loss function is defined as follows: in, Represents the cross-entropy loss function; These are the corrected pseudo-labels obtained through a dynamic update strategy; The overall objective function is composed of a weighted sum of three loss components: in, It is a weighting factor used for balancing. The contribution of the loss; In the joint inference phase, the final image-text similarity is obtained by aggregating the global embedding score and the local embedding score using an arithmetic mean method, i.e., satisfying... = ( .