Weak supervision infrared visible light pedestrian re-identification method based on heterogeneous expert joint learning

Abstract

The invention relates to a weak supervision infrared visible light pedestrian re-identification method based on heterogeneous expert joint learning. Belongs to the technical field of image recognition. The method comprises the following steps: training by using an infrared visible light pedestrian re-identification data set with a single-modal label to complete heterogeneous modal expert construction; cross-modal identity prediction is carried out through heterogeneous modal experts, and cross-modal relationship establishment is completed by using a relationship establishment mechanism; samples are dynamically selected to train infrared and visible light encoders, a shared classifier and heterogeneous modal experts according to different confidence degrees of the pseudo labels, mutual collaboration and promotion of all components are achieved, heterogeneous expert joint learning is completed, and pedestrian re-identification is conducted through the optimized infrared and visible light encoders; according to the method, the dependence on cross-modal labeling is eliminated, the accuracy of cross-modal retrieval is effectively improved, and a brand new solution is provided for an infrared visible light pedestrian re-recognition task.

Description

Technical Field

[0001] This invention relates to a weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning, belonging to the field of image recognition technology. Background Technology

[0002] Person re-identification (ReID), a key technology in multi-camera collaborative perception and intelligent security systems, aims to identify target individuals with similar behavioral characteristics across camera perspectives. In well-lit environments, visible light cameras can capture rich color and texture information, leading to significant progress in traditional single-modal ReID methods on multiple public datasets. However, in uncontrollable scenarios such as nighttime or low-light conditions, the quality of visible light images degrades significantly, severely limiting their applicability in all-weather surveillance systems. Infrared imaging, as a perception method unaffected by changes in lighting, can stably capture the contours and thermal radiation features of pedestrians in dark environments, thus being widely used in tasks such as nighttime surveillance and border security. This has given rise to the emerging research direction of Visible-Infrared Person Re-Identification (VIReID), which aims to achieve identity matching between infrared and visible light images with significant modal differences, thereby enabling all-weather, multimodal pedestrian tracking.

[0003] However, current mainstream VIReID methods generally rely on complete cross-modal identity annotation, meaning that each pair of infrared and visible light images needs to be manually labeled with their corresponding identities. This premise faces serious challenges in practical applications. Firstly, constructing VIReID datasets presents two major difficulties: firstly, infrared and visible light cameras typically operate asynchronously, resulting in a lack of real-world time synchronization and viewpoint pairing between acquired images; secondly, due to differences in imaging mechanisms, images from different modalities exhibit significant differences in color, texture, and even contour features, making accurate manual annotation difficult. Furthermore, although some methods have conducted VIReID research in unsupervised settings, these methods typically rely on feature clustering to generate pseudo-identity labels. The introduction of noise can easily weaken the model's discriminative performance, severely impacting the final retrieval results.

[0004] To reduce reliance on cross-modal labels, this invention proposes a weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning, targeting a weakly supervised setting that is closer to real-world application scenarios. It uses only easily obtainable single-modal labels for training, and relies on experts to predict cross-modal relationships. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning, which effectively solves the problem of missing cross-modal annotations. While significantly reducing annotation costs, it achieves effective modeling of cross-modal feature consistency.

[0006] The technical solution of this invention is: a weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning, the method comprising the following steps:

[0007] Step 1, Heterogeneous Modal Expert Construction: Input pedestrian images in infrared and visible light modalities, train the infrared and visible light encoders and modal classifiers, and thus construct the two modal classifiers into heterogeneous modal experts with intramodal identity discrimination capabilities;

[0008] Step 2, cross-modal relationship establishment: Using the above heterogeneous modal experts, the identities of all samples in another modality are predicted to obtain the identity prediction results of cross-modal samples. An identity correspondence matrix is ​​generated based on the prediction results, and the relationships are fused based on the consistency and conflict of the prediction relationships, and divided into cross-modal pseudo-labels with multiple confidence levels.

[0009] Step 3, Heterogeneous Modality Expert Joint Learning: Based on the different pseudo-label correspondences established in Step 2, samples are dynamically selected, and multiple loss functions are jointly constrained for the infrared and visible light encoders, heterogeneous modality experts, and shared classifiers. The shared classifier guides the encoder to learn modality-invariant feature extraction capabilities to achieve collaborative optimization of different components; the optimized infrared and visible light encoders are used for pedestrian re-identification.

[0010] Furthermore, under the weak supervision setting, all input images only possess intra-modal identity labels, and there is no cross-modal corresponding label information, defined as:

[0011]

[0012] in, Let t be the mode t, where t is an image sample under visible light v or infrared r. For its corresponding label, n t Let be the total number of samples for mode t.

[0013] Furthermore, in step 1, the optimization objectives of the training include single-modal identity loss and weighted regularized triplet loss, specifically:

[0014] The formula for the single-modal identity loss function is expressed as follows:

[0015]

[0016] Among them, f i tIndicates the input image The features extracted by the encoder, C t (f i t C is a classifier expert for mode t. t The classification results are given; under the constraint of the in-modal identity loss function, the classifier expert obtains the modality-specific identity discrimination ability;

[0017] The formula for the weighted regularized triplet loss function is expressed as follows:

[0018]

[0019] in, and These respectively represent the data in a batch of input data. The corresponding positive and negative sample sets, that is, the sets of samples with the same and different identities, where j and k represent the indices of the positive and negative samples, respectively. and n represents the Euclidean distance between two positive sample features. t Let t be the total number of samples in modality t; the role of this loss function is to bring positive samples closer together while widening the distance between negative samples in the feature space, thus guiding the encoder to obtain discriminative pedestrian features.

[0020] Furthermore, in step 2, heterogeneous modal experts predict the identities of all samples from another modality, and integrate the prediction results from different experts using a relationship-establishing mechanism. This process includes:

[0021] First, visible light expert C v And infrared expert C r Identity prediction is performed on all images of another modality to obtain the identity prediction score S of the visible light expert for the infrared sample. r→v Infrared experts predict the identity score S of visible light samples v→r To characterize cross-modal identity matching results;

[0022] Then, the statistics of S r→v and S v→r The identity correspondence in the matrix is ​​analyzed. Identity pairs with corresponding relationships are sorted in descending order of frequency of occurrence. Conflicting correspondences are removed to obtain the cross-modal identity correspondence matrix M belonging to the two experts. r→v M v →r The element represents the relationship between different modal identities, where 1 indicates a correspondence and 0 indicates no correspondence.

[0023] Finally, three different cross-modal relationships are established, including:

[0024] A consensus among experts:

[0025]

[0026] Relationships unique to a single expert:

[0027]

[0028] Among them, C t Represents the total number of categories for mode t, 1 {0} This is an indicator function that outputs 1 when the input is 0, and 0 otherwise.

[0029] The contradictory relationship among experts:

[0030] M w =M v→r +(M r→v ) T -2M c -M s .

[0031] Further, step 3 includes:

[0032] Step 3.1: Use the cross-modal identity relationships with different confidence levels obtained in Step 2 as pseudo-labels to guide cross-modal learning, enabling it to learn the representational ability of modality-invariant features. The process includes:

[0033] For those belonging to M c and M s For samples with high-confidence identity correspondence, cross-modal identity loss and weighted regularized triplet loss are used to constrain the shared classifier and encoder; except that the single-modal label is replaced with a cross-modal pseudo-label, the expression of the loss function formula is consistent with the training loss function in step 1.

[0034] For those belonging to M w For samples containing noisy identity mappings, a weakly supervised loss function is used to constrain the model to avoid erroneous supervisory information from the noise affecting the model. Its formula is expressed as follows:

[0035]

[0036] Where K = {k|M w (j,k)=1}, where j is a sample Intramodal identity labels, ∈=10 -10 Its function is to prevent numerical overflow. Indicates the classification result C c (f i v The probability value of the l-th position in ), m il It is a vector m iThe l-th element, This indicates that the identity exists in M ​​among a batch of samples. w The total number of samples in the sample;

[0037] Step 3.2: During training, continuously optimize the cross-modal feature discrimination capability using expert consistency loss. The process is defined as follows:

[0038] Before each round of training in step 3, a feature prototype belonging to each identity is constructed for each modality. It is initialized using the feature mean of its identity; the feature prototype is updated using momentum update, with the following formula:

[0039]

[0040] Where λ is the momentum update factor to prevent drastic fluctuations in the feature. This represents the mean of all features belonging to identity i;

[0041] The expert consistency loss function is used to encourage experts to make consistent predictions for samples with the same identity but different modalities.

[0042] For visible light experts, the formula is expressed as follows:

[0043]

[0044] Where, n c This indicates that the identity exists in M ​​among a batch of samples. c The total number of samples in C v This represents the total number of pedestrian identities in the visible light mode v. and The classification results of visible light experts for the i-th visible light sample and for the infrared prototype with the same identity as the sample are respectively given. This indicates that in expert decision-making, information entropy H reflects the confidence level of the expert's decision; a smaller information entropy indicates a greater certainty in the expert's decision. The expert consensus loss function weighted by information entropy is expressed as follows:

[0045]

[0046] in, H(·) represents the expert consensus loss corresponding to infrared modality r, and H(·) represents the information entropy of the classification result. Under the action of this loss, the weights can be adaptively adjusted to control the constraint intensity. When the decision made by the experts is unreliable, the constraint intensity is increased. In this step, the single-modal identity loss function of step 1 is also used to constrain the classifier.

[0047] The present invention also provides a weakly supervised infrared and visible light pedestrian re-identification system based on heterogeneous expert joint learning, the system comprising: a module for executing the aforementioned weakly supervised infrared and visible light pedestrian re-identification method based on heterogeneous expert joint learning.

[0048] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning.

[0049] The beneficial effects of this invention are:

[0050] 1. This invention proposes a modality expert construction method, aiming to train two modality-specific classifiers (infrared and visible light) separately without cross-modal labels, enabling them to predict cross-modal identities. Each expert is trained independently using in-modality identity labels, achieving good single-modal discrimination performance while laying the foundation for subsequent cross-modal identity relationship construction. This method effectively avoids the reliance on manually labeled cross-modal identity pairs, significantly reducing the human and time costs during training.

[0051] 2. This invention designs a relationship establishment module that integrates the classification prediction results of two modality experts on samples. By evaluating the consistency between the prediction results, it generates a subset of pseudo-labels with different confidence levels and uses them to dynamically select training samples, thereby providing a stable and reliable supervision signal.

[0052] 3. This invention proposes a heterogeneous expert joint learning framework. Guided by a constructed pseudo-cross-modal identity relationship, the encoder learns modality-invariant and discriminative feature representations. Simultaneously, an expert consensus loss function based on information entropy weighting is introduced to continuously optimize the discriminative and collaborative abilities of the two experts. These two components work synergistically during training, complementing each other and jointly enhancing the model's cross-modal retrieval performance under weak supervision. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall process framework of the method of the present invention;

[0054] Figure 2 A structural diagram of the heterogeneous expert construction method of the present invention is provided.

[0055] Figure 3 A module structure diagram is established to illustrate the relationships within the method of this invention;

[0056] Figure 4 This is a structural diagram of the heterogeneous expert collaborative learning method of the present invention;

[0057] Figure 5This is a graph showing the change in pseudo-label accuracy during the training process of the method of this invention;

[0058] Figure 6 This is a diagram showing the pedestrian retrieval results of the method of the present invention. Detailed Implementation

[0059] Example 1: As Figures 1-6 As shown, a weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning includes the following steps:

[0060] Step 1, Heterogeneous Modal Expert Construction: Input pedestrian images in infrared and visible light modalities, train the infrared and visible light encoders and modal classifiers, and thus construct the two modal classifiers into heterogeneous modal experts with intramodal identity discrimination capabilities;

[0061] Step 2, cross-modal relationship establishment: Using the above heterogeneous modal experts, the identities of all samples in another modality are predicted to obtain the identity prediction results of cross-modal samples. An identity correspondence matrix is ​​generated based on the prediction results, and the relationships are fused based on the consistency and conflict of the prediction relationships, and divided into cross-modal pseudo-labels with multiple confidence levels.

[0062] Step 3, Heterogeneous Modality Expert Joint Learning: Based on the different pseudo-label correspondences established in Step 2, samples are dynamically selected, and multiple loss functions are jointly constrained for the infrared and visible light encoders, heterogeneous modality experts, and shared classifiers. The shared classifier guides the encoder to learn modality-invariant feature extraction capabilities to achieve collaborative optimization of different components; the optimized infrared and visible light encoders are used for pedestrian re-identification.

[0063] Furthermore, under the weak supervision setting, all input images only possess intra-modal identity labels, and there is no cross-modal corresponding label information, defined as:

[0064]

[0065] in, Let t be the mode t, where t is an image sample under visible light v or infrared r. For its corresponding label, n t Let be the total number of samples for mode t.

[0066] Furthermore, in step 1, the optimization objectives of the training include single-modal identity loss and weighted regularized triplet loss, specifically:

[0067] The formula for the single-modal identity loss function is expressed as follows:

[0068]

[0069] Among them, fi t Indicates the input image The features extracted by the encoder, C t (f i t C is a classifier expert for mode t. t The classification results are given; under the constraint of the in-modal identity loss function, the classifier expert obtains the modality-specific identity discrimination ability;

[0070] The formula for the weighted regularized triplet loss function is expressed as follows:

[0071]

[0072] in, and These respectively represent the data in a batch of input data. The corresponding positive and negative sample sets, that is, the sets of samples with the same and different identities, where j and k represent the indices of the positive and negative samples, respectively. and n represents the Euclidean distance between two positive sample features. t Let t be the total number of samples in modality t; the role of this loss function is to bring positive samples closer together while widening the distance between negative samples in the feature space, thus guiding the encoder to obtain discriminative pedestrian features.

[0073] Furthermore, in step 2, heterogeneous modal experts predict the identities of all samples from another modality, and integrate the prediction results from different experts using a relationship-establishing mechanism. This process includes:

[0074] First, visible light expert C v And infrared expert C r Identity prediction is performed on all images of another modality to obtain the identity prediction score S of the visible light expert for the infrared sample. r→v Infrared experts predict the identity score S of visible light samples v→r To characterize cross-modal identity matching results;

[0075] Then, the statistics of S r→v and S v→r The identity correspondence in the matrix is ​​analyzed. Identity pairs with corresponding relationships are sorted in descending order of frequency of occurrence. Conflicting correspondences are removed to obtain the cross-modal identity correspondence matrix M belonging to the two experts. r→v M v →r The element represents the relationship between different modal identities, where 1 indicates a correspondence and 0 indicates no correspondence.

[0076] Finally, three different cross-modal relationships are established, including:

[0077] A consensus among experts:

[0078]

[0079] Relationships unique to a single expert:

[0080]

[0081] Among them, C t Represents the total number of categories for mode t, 1 {0} This is an indicator function that outputs 1 when the input is 0, and 0 otherwise.

[0082] The contradictory relationship among experts:

[0083] M w =M v→r +(M r→v ) T -2M c -M s .

[0084] Further, step 3 includes:

[0085] Step 3.1: Use the cross-modal identity relationships with different confidence levels obtained in Step 2 as pseudo-labels to guide cross-modal learning, enabling it to learn the representational ability of modality-invariant features. The process includes:

[0086] For those belonging to M c and M s For samples with high-confidence identity correspondence, cross-modal identity loss and weighted regularized triplet loss are used to constrain the shared classifier and encoder; except that the single-modal label is replaced with a cross-modal pseudo-label, the expression of the loss function formula is consistent with the training loss function in step 1.

[0087] For those belonging to M w For samples containing noisy identity mappings, a weakly supervised loss function is used to constrain the model to avoid erroneous supervisory information from the noise affecting the model. Its formula is expressed as follows:

[0088]

[0089] Where K = {k|M w (j,k)=1}, where j is a sample Intramodal identity labels, ∈=10 -10 Its function is to prevent numerical overflow. Indicates the classification result C c (f i v The probability value of the l-th position in ), mil It is a vector m i The l-th element, This indicates that the identity exists in M ​​among a batch of samples. w The total number of samples in the model; the weakly supervised loss function aims to avoid misclassification by the classifier, make full use of the noisy supervision signal, and avoid noise affecting model performance.

[0090] Step 3.2: During training, continuously optimize the cross-modal feature discrimination capability using expert consistency loss. The process is defined as follows:

[0091] Before each round of training in step 3, a feature prototype belonging to each identity is constructed for each modality. It is initialized using the feature mean of its identity; the feature prototype is updated using momentum update, with the following formula:

[0092]

[0093] Where λ is the momentum update factor to prevent drastic fluctuations in the feature. This represents the mean of all features belonging to identity i;

[0094] The expert consistency loss function is used to encourage experts to make consistent predictions for samples with the same identity but different modalities.

[0095] For visible light experts, the formula is expressed as follows:

[0096]

[0097] Where, n c This indicates that the identity exists in M ​​among a batch of samples. c The total number of samples in C v This represents the total number of pedestrian identities in the visible light mode v. and The classification results of visible light experts for the i-th visible light sample and for the infrared prototype with the same identity as the sample are respectively given. This indicates that in expert decision-making, information entropy H reflects the confidence level of the expert's decision; a smaller information entropy indicates a greater certainty in the expert's decision. The expert consensus loss function weighted by information entropy is expressed as follows:

[0098]

[0099] in, H(·) represents the expert consensus loss corresponding to infrared mode r, and H(·) represents the information entropy of the classification result. Under the action of this loss, the weights can be adaptively adjusted to control the constraint intensity. When the decision made by the expert is unreliable, the constraint intensity is increased. In order to enable the expert to continuously adapt to the mode-invariant features output by the encoder, in this step, the single-modal identity loss function of step 1 is also used to constrain the classifier.

[0100] The present invention also provides a weakly supervised infrared and visible light pedestrian re-identification system based on heterogeneous expert joint learning, the system comprising: a module for executing the aforementioned weakly supervised infrared and visible light pedestrian re-identification method based on heterogeneous expert joint learning.

[0101] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning.

[0102] To verify the effectiveness of the method of this invention, experiments were conducted on two datasets: SYSU-MM01 and LLCM. SYSU-MM01 is the first large-scale publicly available visible-infrared person re-identification (VIReID) dataset. Its training set contains 395 cross-modal identity labels, with a total of 22,258 visible light images and 11,909 infrared images collected, covering diverse indoor and outdoor shooting scenarios, and is widely used. LLCM, as the most challenging emerging dataset in the current VI-ReID field, has a training set containing 713 identity-labeled samples, including 16,946 visible light images and 13,975 infrared images, all collected from real-world scenes with complex lighting interference and background changes, more closely reflecting the difficulty requirements of actual deployment scenarios. To fully evaluate the retrieval capability of the method of this invention, the Rank accuracy in the Cumulative MatchCharacteristic (CMC) curve is used as a refined indicator of retrieval performance, while the mean Average Precision (mAP) is introduced as a core evaluation indicator of comprehensive performance. The higher the value of the above indicators, the better the accuracy and coverage of the model in actual retrieval, and the stronger the cross-modal recognition capability.

[0103] Table 1 compares the performance with other methods on the SYSU-MM01 dataset.

[0104]

[0105]

[0106] Table 2 compares the performance with other methods on the LLCM dataset.

[0107]

[0108] Comparison on the SYSU-MM01 dataset: As shown in Table 1, the method of this invention achieved a Rank-1 accuracy of 70.4% and an mAP of 66.6% on the SYSU-MM01 dataset. Although this method does not rely on cross-modal manual labels, its performance even surpasses some supervised methods, fully demonstrating the effectiveness of expert collaborative reasoning and pseudo-labeling mechanisms. When comparing with existing unsupervised and semi-supervised methods, this invention still demonstrates a leading advantage: compared with the mainstream semi-supervised method DPIS, it leads by 12.0% and 11.0% in Rank-1 accuracy and mAP, respectively; and compared with the representative unsupervised method GUR, it achieves performance improvements of 6.9% and 5.0%, respectively.

[0109] Comparison on the LLCM dataset: As shown in Table 2, the proposed method still achieves satisfactory performance on the challenging LLCM dataset. In the visible to infrared retrieval mode, the Rank-1 accuracy reaches 55.3%, and the mAP reaches 58.7%, very close to the performance of partially fully supervised methods. In comparison with other unsupervised and semi-supervised models, the proposed method continues to perform excellently: compared to the semi-supervised method OTLA, it achieves a lead of 11.1% and 10.5% in Rank-1 and mAP metrics, respectively; compared to the unsupervised method PGM, the proposed method improves by 10.4% and 9.7%, respectively. These results demonstrate that the proposed method can still achieve leading performance by relying on expert reasoning and relation learning without the involvement of manual cross-modal labeling.

[0110] like Figure 5 As shown, with the increase in the number of training rounds, the cross-modal identity relationships established by the method of this invention are continuously optimized, and the matching accuracy of pseudo-labels steadily increases, significantly outperforming the baseline model. In the mid-to-late training stages, the model can predict more than 90% of the correct cross-modal identity pairs, demonstrating its powerful ability in constructing high-confidence pseudo-labels. This result further illustrates that the method of this invention effectively eliminates the dependence on cross-modal manual labels and can complete the construction of accurate cross-modal identity correspondences with the assistance of expert judgment and relationship fusion mechanisms.

[0111] like Figure 6 As shown, green borders indicate successfully retrieved samples, while red borders indicate incorrectly identified samples. Compared to traditional baseline methods, this invention significantly improves the quality of retrieval results. Baseline methods still suffer from a high probability of misclassification when dealing with complex, cross-modal, difficult samples (such as those with occlusion, illumination variations, and background interference), while the method proposed in this invention significantly improves the model's ability to discriminate difficult samples.

[0112] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning, characterized in that: The method includes the following steps: Step 1, Heterogeneous Modal Expert Construction: Input pedestrian images in infrared and visible light modalities, train the infrared and visible light encoders and modal classifiers, and thus construct the two modal classifiers into heterogeneous modal experts with intramodal identity discrimination capabilities; Step 2, cross-modal relationship establishment: Using the above heterogeneous modal experts, the identities of all samples in another modality are predicted to obtain the identity prediction results of cross-modal samples. An identity correspondence matrix is ​​generated based on the prediction results, and the relationships are fused based on the consistency and conflict of the prediction relationships, and divided into cross-modal pseudo-labels with multiple confidence levels. Step 3, Heterogeneous Modality Expert Joint Learning: Based on the different pseudo-label correspondences established in Step 2, samples are dynamically selected, and multiple loss functions are jointly constrained for the infrared and visible light encoders, heterogeneous modality experts, and shared classifiers. The shared classifier guides the encoder to learn modality-invariant feature extraction capabilities to achieve collaborative optimization of different components; the optimized infrared and visible light encoders are used for pedestrian re-identification.

2. The weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning according to claim 1, characterized in that: Under weak supervision, all input images only have intra-modal identity labels and no cross-modal label information, defined as: in, Let t be the mode t, where t is an image sample under visible light v or infrared r. For its corresponding label, n t Let be the total number of samples for mode t.

3. The weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning according to claim 1, characterized in that: In step 1, the optimization objectives of the training include single-modal identity loss and weighted regularized triplet loss, specifically: The formula for the single-modal identity loss function is expressed as follows: Among them, f i t Indicates the input image The features extracted by the encoder, C t (f i t C is a classifier expert for mode t. t The classification results are given; under the constraint of the in-modal identity loss function, the classifier expert obtains the modality-specific identity discrimination ability; The formula for the weighted regularized triplet loss function is expressed as follows: in, and These respectively represent the data in a batch of input data. The corresponding positive and negative sample sets, that is, the sets of samples with the same and different identities, where j and k represent the indices of the positive and negative samples, respectively. and n represents the Euclidean distance between two positive sample features. t Let t be the total number of samples in modality t; the role of this loss function is to bring positive samples closer together while widening the distance between negative samples in the feature space, thus guiding the encoder to obtain discriminative pedestrian features.

4. The weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning according to claim 1, characterized in that: In step 2, heterogeneous modal experts predict the identities of all samples from another modality, and integrate the prediction results from different experts using a relationship-establishing mechanism. The process includes: First, visible light expert C v And infrared expert C r Identity prediction is performed on all images of another modality to obtain the identity prediction score S of the visible light expert for the infrared sample. r→v Infrared experts predict the identity score S of visible light samples v→r To characterize cross-modal identity matching results; Then, the statistics of S r→v and S v→r The identity correspondence in the matrix is ​​analyzed. Identity pairs with corresponding relationships are sorted in descending order of frequency of occurrence. Conflicting correspondences are removed to obtain the cross-modal identity correspondence matrix M belonging to the two experts. r→v M v→r The element represents the relationship between different modal identities, where 1 indicates a correspondence and 0 indicates no correspondence. Finally, three different cross-modal relationships are established, including: A consensus among experts: Relationships unique to a single expert: A(i,j)=r i ·c j , Among them, C t Represents the total number of categories for mode t, 1 {0} This is an indicator function that outputs 1 when the input is 0, and 0 otherwise. The contradictory relationship among experts: M w =M v→r +(M r→v ) T -2M c -M s 。 5. The weakly supervised infrared-visible light pedestrian re-identification method based on heterogeneous expert joint learning according to claim 1, characterized in that: Step 3 includes: Step 3.1: Use the cross-modal identity relationships with different confidence levels obtained in Step 2 as pseudo-labels to guide cross-modal learning, enabling it to learn the representational ability of modality-invariant features. The process includes: For those belonging to M c and M s For samples with high-confidence identity correspondence, cross-modal identity loss and weighted regularized triplet loss are used to constrain the shared classifier and encoder; except that the single-modal label is replaced with a cross-modal pseudo-label, the expression of the loss function formula is consistent with the training loss function in step 1. For those belonging to M w For samples containing noisy identity mappings, a weakly supervised loss function is used to constrain the model to avoid erroneous supervisory information from the noise affecting the model. Its formula is expressed as follows: Where K = {k|M w (j,k)=1}, where j is a sample Intramodal identity labels, ∈=10 -10 Its function is to prevent numerical overflow. Indicates the classification result C c (f i v The probability value of the l-th position in ), m il It is a vector m i The l-th element, This indicates that the identity exists in M ​​among a batch of samples. w The total number of samples in the sample; Step 3.2: During training, continuously optimize the cross-modal feature discrimination capability using expert consistency loss. The process is defined as follows: Before each round of training in step 3, a feature prototype belonging to each identity is constructed for each modality. It is initialized using the feature mean of its identity; the feature prototype is updated using momentum update, with the following formula: Where λ is the momentum update factor to prevent drastic fluctuations in the feature. This represents the mean of all features belonging to identity i; The expert consistency loss function is used to encourage experts to make consistent predictions for samples with the same identity but different modalities. For visible light experts, the formula is expressed as follows: Where, n c This indicates that the identity exists in M ​​among a batch of samples. c The total number of samples in C v This represents the total number of pedestrian identities in the visible light mode v. and The classification results of visible light experts for the i-th visible light sample and for the infrared prototype with the same identity as the sample are respectively given. This indicates that in expert decision-making, information entropy H reflects the confidence level of the expert's decision; a smaller information entropy indicates a greater certainty in the expert's decision. The expert consensus loss function weighted by information entropy is expressed as follows: in, H(·) represents the expert consensus loss corresponding to infrared modality r, and H(·) represents the information entropy of the classification result. Under the action of this loss, the weights can be adaptively adjusted to control the constraint intensity. When the decision made by the experts is unreliable, the constraint intensity is increased. In this step, the single-modal identity loss function of step 1 is also used to constrain the classifier.

6. A weakly supervised infrared-visible light pedestrian re-identification system based on heterogeneous expert joint learning, characterized in that, The system includes a module for performing the weakly supervised infrared-visible pedestrian re-identification method based on heterogeneous expert joint learning as described in any one of claims 1 to 5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the weakly supervised infrared and visible light pedestrian re-identification method based on heterogeneous expert joint learning as described in any one of claims 1 to 5.

Images