A cross-modal re-identification method based on feature separation and causal comparison loss

Abstract

The application provides a cross-modal re-identification method based on feature separation and causal comparison loss. First, a feature extraction network of Resnet50 embedded with a CNS non-local neural network is used to extract global features of pedestrians, the network can not only enhance the extraction ability of the CNN for global semantic information, but also establish the connection between two pixels with a certain distance on the image. Second, the extracted features are separated by using a feature separation module to obtain two types of features of strong and weak identity correlation, then a double causal comparison loss is used as a constraint to guide the removal of identity irrelevant information from the strong identity correlation features, so that the expression ability of the strong identity correlation features is enhanced, and finally, the cross-modal pedestrian re-identification precision is improved by combining identity loss and weighted regularization triplet loss for joint training. The application is mainly applied to a video monitoring intelligent analysis application system, and has broad application prospects in the fields of image retrieval, pedestrian re-identification intelligent security and the like.

Description

Technical Field

[0001] This invention relates to a cross-modal re-identification method based on feature separation and causal comparison loss, and to a novel visible and infrared cross-modal person re-identification framework designed with feature separation and dual causal constraint loss networks. It relates to the cross-modal pedestrian re-identification problem in the field of intelligent video surveillance and belongs to the field of computer vision and intelligent information processing. Background Technology

[0002] Person re-identification (ReID) is a subtask of image retrieval, aiming to match the same person between non-overlapping cameras. With the increasing prevalence of surveillance systems in real life, person recognition has become increasingly important. Most existing methods focus on visible light RGB images. However, in poor lighting conditions or at night, visible light cameras cannot capture effective appearance information, limiting the applicability of ReID in practical surveillance applications. In recent years, with the increasing demands of practical applications, research on visible and infrared cross-modality person re-identification (VI-ReID) has grown significantly. Since visible light images and infrared images belong to different data modalities, the process of retrieving visible light images from infrared images is often referred to as a cross-modal re-identification method. Besides facing intramodal variations such as changes in lighting, occlusion, and viewing angle, VI-ReID also faces new challenges arising from modal differences. VI-ReID is a very challenging task. First, light and infrared images differ significantly in composition. Light images have three channels containing color information, while infrared images have only one channel containing infrared information, thus they can be considered heterogeneous data. Second, from an imaging principle perspective, the wavelength ranges of light and infrared images are different, so color information alone is insufficient for effective classification and matching. Furthermore, pedestrian images in a single modality also exhibit intra-class differences due to variations in pose, lighting, and camera perspective.

[0003] For cross-modal ReID tasks, domain-specific network model construction and the extraction and matching of common features between the two modalities are crucial. Most existing VI-ReID algorithms focus only on learning common features across different modalities, neglecting the detailed representations specific to each modality. In recent years, many methods for extracting local details have emerged. Some methods use pedestrian image slicing to extract local information; others use keypoint skeleton methods to extract human body information; and still others use semantic segmentation. However, when pedestrian images are misaligned, image segmentation significantly reduces performance. Furthermore, keypoint skeleton extraction and semantic segmentation methods do not utilize end-to-end training; they require pre-trained models for keypoint skeleton extraction and semantic segmentation, necessitating two stages, which is cumbersome and inefficient. Unlike these methods, this invention directly extracts identity-related features from different modalities and filters out identity-irrelevant interference, ultimately allowing the network to learn more effective, domain-invariant feature representations. Compared to previous methods, this invention offers superior performance and efficiency.

[0004] As is well known, deep learning-based image retrieval methods typically use CNN models to extract features, and then use distance metrics to compare the similarity between feature vectors, selecting the image with the most similar features as the retrieval result. Some works also use multi-scale or different convolutional channels to capture pixel-level / small region-level attention. However, in cross-modal recognition, the optimization process is unstable due to significant cross-modal differences and noise. To enhance the ability of CNNs to extract global semantic information and better establish the connection between two pixels at a certain distance in an image, this invention proposes a CNS non-local neural network based on non-local neural networks. The CNS non-local neural network can be easily embedded into any layer of ResNet50.

[0005] Traditional VI-ReID algorithms typically extract global feature representations of pedestrians in visible light and infrared light separately, then project these features into a common space using a similarity metric objective function. This results in similar pairs of infrared and visible light pedestrians being close in spatial distance, indicating high similarity. However, this matching method only roughly measures the global similarity of visible light and infrared pedestrians, without specifically considering which local aspects of the pedestrians are similar. Furthermore, most methods do not account for background noise interference, leading to often low recognition accuracy. Considering the differences between different modal domains, we utilize FSM for style normalization to eliminate style differences. This invention uses the FSM module to separate strongly correlated and weakly correlated features from different modalities, effectively mitigating the impact of modal differences and enabling the learning of more effective feature representations. Many VI-ReID methods first use a dual-path convolutional network to extract features from RGB and infrared images separately; then input the features from both modalities into a network with shared parameters; finally, guide training using cross-entropy loss and triplet loss. If only identity loss is used to constrain the model, it can only improve inter-class differences, not solve the inter-class differences between different modalities. Current methods often employ identity loss and triplet loss to jointly train and constrain the distance between different modal distributions. While these methods alleviate modal differences to some extent, their effectiveness is not ideal in scenarios with significant background variations. Therefore, this invention uses bicausal comparison loss as a constraint to guide the removal of identity-irrelevant information from identity-correlated features, thereby improving the expressive power of identity-correlated features. Simultaneously, it combines identity loss and weighted regularized triplet loss for joint training, ultimately improving the accuracy of cross-modal person re-identification. Summary of the Invention

[0006] This invention proposes a cross-modal re-identification method based on feature separation and causal comparison loss. A novel visible and infrared cross-modal pedestrian re-identification network structure, FSDCC (feature separation and double causal constraint loss), is designed. The sampling strategy for each training batch is as follows: P pedestrians are randomly selected from the training dataset, and then K visible light pedestrian images and K infrared pedestrian images are randomly selected for each pedestrian, forming a batch training data containing 2n (here n = P × K) pedestrian images. Finally, the 2n pedestrian images are fed into the network for training. Figure 1 As shown, firstly, a CNS nonlocal neural network is used to extract features f from infrared and visible light pedestrian images, with shared network parameters. Secondly, a feature separation module separates the previously obtained global features to obtain identity-related features f. + Features f that are weakly correlated with identity -Two types of features are used (such as clutter interference components). Finally, a bicausal comparison loss is used to constrain and guide the removal of identity-irrelevant information from identity-related features, thereby improving the expressive power of identity-related features. At the same time, the identity loss function and the adaptive weighted hardest batch sampling triplet loss function are combined for joint training, ultimately achieving the goal of improving the accuracy of cross-modal pedestrian re-identification.

[0007] A cross-modal re-identification method based on feature separation and causal comparison loss includes the following steps:

[0008] (1) First, the visible light and infrared pedestrian images, both with a size of 288×144, are fed into two networks with the same structure to obtain the visible light features f. rgb and infrared features f IR Then f rgb and f IR The feature f obtained by performing the concatenate operation rgb_ir Finally, f rgb_ir The data is fed into a CNS nonlocal neural network, and the final output is the pedestrian feature f.

[0009] (2) Use the feature separation module to perform feature separation on the feature f from step (1), and obtain the identity-related features (f) respectively. + ) and features weakly correlated with identity (f - Two types of characteristics.

[0010] (3) Using bicausal comparison loss as a constraint to guide the analysis of identity-related features (f + The algorithm removes identity-irrelevant information, thereby improving the expressive power of identity-related features. At the same time, it combines identity loss and weighted regularized triplet loss for joint training, ultimately improving the accuracy of cross-modal pedestrian re-identification. Attached Figure Description

[0011] Figure 1 This is a block diagram of a cross-modal re-identification method based on feature separation and causal comparison loss according to the present invention;

[0012] Figure 2 This is a diagram of the CNS nonlocal neural network structure of the present invention;

[0013] Figure 3 This is a structural diagram of the feature separation module of the present invention;

[0014] Figure 4 For the L of the present invention id L wrt and L dcc Schematic diagram of feature space constraints;

[0015] Figure 5 f of the present invention+ f and f - A visualization of the features. Detailed Implementation

[0016] The following is in conjunction with the appendix Figure 1 Appendix Figure 2 , attached Figure 3 , attached Figure 4 and attached Figure 5 Further explanation of the present invention:

[0017] Figure 1 The following is an explanation of the FSDCC network structure and principle:

[0018] The FSDCC network model framework is trained and tested in an end-to-end manner. The FSDCC network mainly consists of four parts: dual-stream input, feature extraction, feature separation and embedding, and loss constraints. In the dual-stream input part, the input visible light and infrared pedestrian images are both three-channel images with a height and width of 288×144. The visible light and infrared pedestrian images are fed into two structurally identical networks respectively, yielding visible light features f. rgb and infrared features f IR f rgb and f IR The dimensions are n×64×72×36. The two network parameters are not shared. The network structure consists of a convolutional layer, a batch normalization layer, a non-linear activation layer, and a max-pooling layer. Then f... rgb and f IR Perform the concatenate operation to obtain feature f rgb_ir f rgb_ir The dimension is 2n×64×72×36, then f rgb_ir The data is fed into a CNS nonlocal neural network to obtain pedestrian features f. In the feature separation and embedding section, the features f output by the CNS nonlocal neural network are first fed into the feature separation module for feature separation, yielding identity-related features (fi). + ) and features weakly correlated with identity (f - Two types of features are used. In the feature embedding part, the data is converted into a fixed-size feature representation for easier processing and computation. In the loss constraint part, a bicausal comparison loss is used as a constraint to guide the selection of identity-related features (f + The algorithm removes identity-irrelevant information, thereby enhancing the expressive power of identity-related features. In addition, it combines the identity loss function and the adaptive weighted most difficult batch sampling triplet loss function for joint training, ultimately improving the accuracy of cross-modal pedestrian re-identification.

[0019] (1) CNS Nonlocal Neural Network Module

[0020] Figure 2This invention presents a complete framework for the CNS nonlocal neural network module. The CNS module comprises four CNS components, each consisting of a 1×1 convolutional layer, a normalization layer, and a sigmoid nonlinear activation layer. The input and output images of the CNS module have the same shape. Figure 1 The diagram shows a network framework for a cross-modal re-identification method based on feature separation and causal comparison loss. This invention uses ResNet50 to extract pedestrian features and embeds CNS nonlocal neural network modules in all four stages of ResNet50. The ResNet50 feature extraction network with CNS nonlocal neural network modules can directly capture long-range dependencies by calculating the interaction between any two locations, without being limited to neighboring points. This is equivalent to constructing a convolutional kernel as large as the feature map size, thereby maintaining more information.

[0021] (2) shows the feature separation module.

[0022] The internal structure of the feature separation module (FSM) is as follows: Figure 3 As shown, the FSM includes a global average pooling layer and two pooling layers respectively composed of... A parameterized fully connected layer is followed by a ReLU activation function δ(·) and a sigmoid activation function σ(·). To reduce the number of parameters, a scaling factor τ is used and set to 16. Therefore, the input image features... FSM output separation mask vector α={α1,α2,α3,...,α k The calculation process is defined as follows:

[0023]

[0024] Features of the input module For all visible light image features and infrared image features, given the image features output by the base network. Then, the modality is unified in the metric space and decomposed into two components that are independent of the modality but strongly correlated with identity. Interference factors such as background that affect identity recognition in, represent Corresponding channels, and using a more stable "residual" method to correlate with the original features. Combination:

[0025]

[0026]

[0027] because It includes components strongly correlated with identity, and combines the original features through a more stable "residual" method. get This is then used as the vector for pedestrian identification during the final test. Compared to the original features... Because it contains a lot of interference components that are unrelated to identity, i.e. Theoretically, its ability to distinguish should be weakened.

[0028] Since visible light and infrared cross-modal person re-identification is a fine-grained instance retrieval method, the currently widely used max pooling or average pooling cannot capture the discriminative features of specific domains well. Therefore, a generalized mean pooling (GMP) method is adopted to... and Pooling is used to obtain f + f and f - General average pooling has hyperparameters, unlike typical average pooling or min (max) pooling methods, f + f and f - The calculation process is shown in the following formula:

[0029]

[0030]

[0031]

[0032] Here, p is a hyperparameter that can be learned through the network or set manually. It can be seen that commonly used max pooling and average pooling are special cases of general average pooling; p→∞ is equivalent to max pooling, while p=1 is equivalent to average pooling.

[0033] Because f + Derived from f, f - It also originates from f + Therefore f + f and f - All of them possess a certain degree of discriminative power and can be directly used as pedestrian features for pedestrian retrieval. Theoretically, f + It contains more identity-related components, while f - If it contains a large number of weakly correlated identity interference components, it means that f + It should have the strongest feature representation capability, f is the second strongest, f is the third strongest. - Slightly worse. Figure 5 It showed f + f and f - The visualization of features allows us to observe f. +The feature effect is significantly better than f - and f.

[0034] (3) Loss Function

[0035] First, use cross-entropy identity loss L id (Identical Loss) for f + Constraints are applied because only f is used. + As the final pedestrian feature, therefore only f + Imposing an identity loss constraint can achieve a better constraint effect. Identity loss L id This will significantly improve the overall compactness of the feature embedding space, as defined below:

[0036]

[0037] Secondly, in order to further improve f + f, f - The overall compactness of the feature space is improved by using weighted regularized tuple loss L. wrt (Weighted Regularization Triplet loss,WRT Loss) for f + f and f - All three features are constrained simultaneously. L wrt Similar to the conventional hard-sample triplet loss, it is used to bring together features of the same identity in the feature embedding space and disperse features of different identities in the feature space. However, L wrt By directly using feature distance to weight the "difficulty" of positive and negative samples, for positive sample pairs, the larger the feature distance, the greater the weight of the loss; for negative sample pairs, the smaller the feature distance, the greater the weight of the loss. No additional boundary margin hyperparameters need to be set, making it more flexible and applicable. wrt The definition is as follows:

[0038]

[0039] in:

[0040]

[0041] In Equations 8 and 9, 'a' represents the set of all anchor points in each training batch, 'p' represents the set of positive samples corresponding to the given identity, and 'n' represents the set of negative samples. and These represent the Euclidean distances between the anchor point and the positive (negative) sample, respectively.

[0042] Finally, to further facilitate the separation of strongly correlated and weakly correlated identity features, a dual causal comparison loss constraint L is designed by comparing the discriminative power of the recovered features. dcc (Dual Causality Compare Loss, DCC Loss). As mentioned earlier, compared to the original feature f, f + It includes components strongly related to identity. It should have stronger discrimination capabilities, while f - It contains a significant number of interference components that are weakly correlated with identity. Theoretically, its discrimination ability should be weakened, L dcc This further promotes the separation of interfering components in features and enhances the discriminative power of features with strong identity correlation, hence the name "causal comparison loss". When it is necessary to enhance f1 while restricting f2, the dual causal loss L... dcc For (f1, f2), the distance between positive sample pairs of f1 needs to be shortened, while the distance between positive sample pairs of f2 needs to be widened. Correspondingly, the distance between negative sample pairs of f1 needs to be widened, while the distance between negative sample pairs of f2 needs to be shortened. This ensures that feature f1 has a stronger discriminative power than f2 in the loss function. Let a1(a2) represent the set of all anchor points for f1(f2) in each training batch, p1(p2) represent the set of positive samples corresponding to the identity of f1(f2), and n1(n2) represent the set of negative samples for f1(f2). Let f1 and f2 represent the Euclidean distances between the anchor points f1 and f2 and the positive and negative samples, respectively. Then L dcc (x1,x2) is defined as:

[0043]

[0044] in, and This is used to measure the degree of enhancement to feature f1, indicating that a greater penalty is imposed on features with weaker discriminative power. and Then it is used to represent the degree of suppression of f2, indicating that a greater penalty is imposed on features with stronger discriminative power, as specifically defined as follows:

[0045]

[0046]

[0047] Since the number of pedestrian identities is usually large, relying solely on identity loss is insufficient to obtain features with sufficient discriminative power. Furthermore, model testing typically addresses the question of "Are these two people the same person?" rather than "Who is this person?". Therefore, measuring the relative distance between features becomes more important. As a variant of the hard sample sampling triplet loss function, the weighted regularized triplet loss L...wrt Additional constraints were applied to the relative distances between features to further shrink f. + f, f - Each feature space is used to mine hard samples and apply different levels of loss and penalty to them for further targeted training. dcc Mainly used for f + f, f - The comparative training of the three features, utilizing the cluttered background and modal differences in the features, is detrimental to pedestrian re-identification. Therefore, L is applied. dcc The constraints separate it out because it contains L dcc (f + ,f) and L dcc (f,f - Therefore, it is called "double causal comparison loss". Table 1 shows f + f, f - After comparing the combined training and testing performance, this invention ultimately selects f + f, f - For training, f + For testing purposes. Figure 4 As shown, it represents L id L wrt and L dcc Feature space constraint diagram. This invention combines identity loss, weighted regularized tuple loss, and the proposed dual causal comparison loss to constrain the overall restriction loss L of the entire model. total Defined as:

[0048]

[0049] Wherein, λ1 and λ2 are hyperparameters that adjust the contribution.

[0050] This invention conducted network structure ablation studies on the RegDB and SYSU-MM01 datasets, where Baseline represents the baseline network, RE stands for random erasure, CNS represents the CNS nonlocal neural network, FSM represents the feature separation module, and L... id Indicating loss of identity, L wrt L represents the weighted hardest triplet sampling loss function. dcc This represents the causal comparison loss. Furthermore, comparisons were made with some mainstream algorithms, using a single query setting for evaluation, and Rank-1, Rank-5, Rank-10, and mAP (mean accuracy) as evaluation metrics. Experimental results are shown in Tables 1, 2, 3, 4, and 5. The experimental accuracy is significantly improved compared to the baseline network and other comparative algorithms.

[0051] Table 1 f +f and f - Feature combination method test experiment

[0052]

[0053] Table 2 Ablation study on SYSU-MM01 data

[0054]

[0055] Table 3 Ablation studies on regDB data

[0056]

[0057] Table 4 compares the results with mainstream algorithms on the SYSU-MM01 dataset.

[0058]

[0059] Table 5 compares the results with mainstream algorithms on the RegDB dataset.

[0060]

[0061]

Claims

1. A cross-modal re-identification method based on feature separation and causal comparison loss, characterized by the following steps: (1) First, the image size is Visible light and infrared pedestrians are sent into two networks with the same structure respectively, and visible light features and infrared features are obtained respectively, then and are concatenated to obtain features , finally is sent into the CNS non-local neural network, and the pedestrian features are finally output; (2) using Feature Separation Module on the features of step (1) to obtain identity strongly related features (F1) and identity weakly related features (F2) respectively. ​​​ (3) Using Dual Causal Contrastive Loss as a constraint to guide the removal of identity-independent information from the identity strongly correlated features (IDSCF) ), so as to improve the expression ability of the identity strongly correlated features, and the identity loss and the weighted regularization triplet loss are combined for joint training, so as to finally improve the cross-modal pedestrian re-identification accuracy. The transformation alignment is that a feature separation module FSM is used in step (2), the FSM includes a global average pooling layer and two parameterized fully connected layers, followed by activation functions and sigmoid activation functions ; in order to reduce the number of parameters, the reduction rate is used and set to 16; therefore, the input image features , the Separate Block outputs a separation mask vector , and the calculation process is defined as: Features of the input module are all visible image features and infrared image features; Given image features output by a base network After, unify the modalities within the metric space and decompose them into two components that are strongly identity-related and independent of the modality And the interference component that affects identity discrimination Wherein, Represent Corresponding channels, and in a more stable "residual" way with the original features Combination: because It includes components strongly correlated with identity, and combines the original features through a more stable "residual" method. get This is used as a vector for pedestrian identification during the final test; compared to the original features , Because it contains a lot of interference components that are unrelated to identity, i.e. Theoretically, its resolution should be weakened; since visible light and infrared cross-modal person re-identification is a fine-grained instance retrieval, the currently widely used max pooling or average pooling cannot capture the discriminative features of specific domains well. Therefore, the general average pooling method GMP is adopted. , and Pooling is performed to obtain , and General average pooling has hyperparameters, unlike typical average pooling or minimum / maximum pooling methods. , and The calculation process is shown in the following formula: wherein is a hyper-parameter that can be learned through network or set artificially; it can be seen that the commonly used max-pooling and average-pooling are special cases of the general average-pooling, is equivalent to max-pooling, and is equivalent to average-pooling; Due to derived from , derived from therefore , and all have certain discriminability, which can be directly used as pedestrian features for pedestrian retrieval; while theoretically contains more identity-related components, and contains more identity-related interference components, which means should have the strongest feature representation ability, second, slightly worse.

2. The method according to claim 1, wherein the sampling strategy for each training batch in step (1) is: randomly selecting from the training dataset. There are 10 pedestrians, and then each pedestrian is randomly selected. Zhang Visible Light Pedestrian Images and Zhang Hong's pedestrian images constitute a collection of Batch training data of Zhang Xingren images, among which Then Images of pedestrians are fed into the network for training, and then visible light features are output. and infrared features , and All dimensions are Then and Obtained by performing a concatenate operation Its dimensions are ; Will The data is fed into a CNS nonlocal neural network to obtain pedestrian features. Its dimensions are ; The CNS non-local neural network used in step (1) comprises four CNS components, which are composed of one convolutional layer, a normalization layer and a sigmoid nonlinear activation layer; the image shape of the input and output of the CNS non-local neural network module is the same; ResNet50 is used to extract pedestrian features, and the CNS non-local neural network module is embedded in four stages of ResNet50; the ResNet50 feature extraction network using the CNS non-local neural network module can directly capture long-range dependencies by calculating the interaction between any two positions, without being limited to adjacent points, which is equivalent to constructing a convolution kernel with the same size as the feature map, so that more information can be maintained.

3. The method of claim 1, wherein In step (3), first, cross-entropy identity loss is used Constraints are imposed on Since only is used as the final pedestrian feature, only is subjected to identity loss constraints to achieve better constraint effects; identity loss will significantly improve the overall compactness of the feature embedding space, which is defined as follows: Second, in order to further improve 、 、 The overall compactness of the feature space, using weighted regularized tuple loss To 、 And Three kinds of features are constrained at the same time; Similar to the conventional hard sample triplet loss, it is used to pull the same identity features in the feature embedding space and disperse the different identity features in the feature space. However, Directly using the feature distance to weight the "difficulty" of positive and negative samples, for positive sample pairs, The greater the characteristic distance is, the greater the loss weight is. For negative sample pairs, the smaller the characteristic distance is, the greater the loss weight is. No additional boundary margin hyperparameter needs to be set, and therefore the method is more flexible and has stronger applicability. The following definitions apply: wherein: In formulas 8 and 9, denotes all anchor sets in each training batch, denotes the positive sample set of the corresponding identity, is the negative sample set, and denote the Euclidean distance between the anchor and the positive / negative samples, respectively. Finally, to further facilitate the separation of strongly correlated and weakly correlated identity features, a dual-causal comparison loss constraint was designed by comparing the discriminative power of the recovered features. As mentioned earlier, compared to the original features , It includes components strongly related to identity. It should have a stronger ability to distinguish, and It contains a significant number of interference components that are weakly correlated with identity. Theoretically, its ability to distinguish should be weakened. This will further promote the separation of interfering components in features and enhance the discriminative power of features strongly correlated with identity; hence the name "causal comparison loss"; when it is necessary to enhance... And restrictions At that time, double causal loss Then it is necessary to zoom in. The distance between positive sample pairs, and the expansion The distance between positive sample pairs needs to be increased accordingly. The distance between negative sample pairs is reduced, and the distance between them is shortened. The distance between negative sample pairs is used to represent features in the loss function. Compare It has stronger discrimination capabilities; using Indicates each training batch All anchor point sets, Then it means The positive sample set corresponding to the identity. That is negative sample set, , ( , ) respectively represent The Euclidean distance between the anchor point and the positive (negative) sample is then... Defined as: in, and Used to measure the characteristics The degree of enhancement indicates the greater the penalty imposed on features with weaker discriminative power, while and Then it is used to represent inhibition. The degree of penalty applied indicates the level of discrimination applied to features with higher discriminative power, as defined below: Because there are many different pedestrian identities, relying solely on identity loss is insufficient to obtain features with enough discriminative power. Moreover, during model testing, re-identification needs to address the question of "whether these two people are the same person" rather than "who this person is." Therefore, measuring the relative distance between features becomes more important. As a variant of the hard sample sampling triplet loss function, the weighted regularized triplet loss... Additional constraints were applied to the relative distances between features to further reduce the size. , , Each feature space is used to mine hard samples and apply different levels of loss and penalty to hard samples for further targeted training; Mainly used for , , The comparative training of the three features, utilizing the cluttered background and modal differences in the features, is detrimental to pedestrian re-identification. Therefore, an application is made... Constraints separate it out because it contains and Therefore, it is called "dual causal comparison loss"; combining identity loss, weighted regularized tuple loss, and the proposed dual causal comparison loss, it is used to constrain the overall limiting loss of the entire model. Defined as: wherein, and are hyperparameters that modulate the contribution.

Images